2024-09-10T21:37:52.227 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T21:37:52.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T21:37:52.346 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899695' kernel: flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy mgr-modules: - snap_schedule sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJuU25ZsKvAISSwvvqu5e9PSsEwQARHPlZSsrRi5TenL/NgBtVkBTHQVgIe551etWODf0sBr692KW93Fd2J/XU= smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAZLDLTRtMvySc+5/wvVEw43sWumnDcGPM2TyHxMI2TXkt6qA/iOrMtmhoel9yO+XLpyeJwCWYHSQkO4VLc7Ik= smithi171.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI+Xf26HFSxUbH3fVWRlefO5Dy9u5PiA6Smau0dxjF03GAya+H0C+9K3fMpnEsCXkcl98B2To1/JQmP86TNYrIM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T21:37:52.346 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T21:37:52.347 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T21:37:52.347 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T21:37:52.349 INFO:teuthology.task.internal:Checking packages... 2024-09-10T21:37:52.369 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T21:37:52.369 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T21:37:52.370 INFO:teuthology.packaging:ref: None 2024-09-10T21:37:52.370 INFO:teuthology.packaging:tag: None 2024-09-10T21:37:52.370 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:37:52.370 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:37:52.370 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:37:52.523 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T21:37:52.525 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T21:37:52.532 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T21:37:52.532 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T21:37:52.570 INFO:teuthology.task.internal:Saving configuration 2024-09-10T21:37:52.589 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T21:37:52.597 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T21:37:52.629 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:32:13.410758', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJuU25ZsKvAISSwvvqu5e9PSsEwQARHPlZSsrRi5TenL/NgBtVkBTHQVgIe551etWODf0sBr692KW93Fd2J/XU='} 2024-09-10T21:37:52.660 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:32:13.409507', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOAZLDLTRtMvySc+5/wvVEw43sWumnDcGPM2TyHxMI2TXkt6qA/iOrMtmhoel9yO+XLpyeJwCWYHSQkO4VLc7Ik='} 2024-09-10T21:37:52.689 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi171.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:32:13.411807', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI+Xf26HFSxUbH3fVWRlefO5Dy9u5PiA6Smau0dxjF03GAya+H0C+9K3fMpnEsCXkcl98B2To1/JQmP86TNYrIM='} 2024-09-10T21:37:52.689 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T21:37:52.697 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T21:37:52.697 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T21:37:52.697 INFO:teuthology.task.internal:roles: ubuntu@smithi171.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T21:37:52.697 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T21:37:52.808 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdc8e6f3be0>, signals=[15]) 2024-09-10T21:37:52.808 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T21:37:52.814 INFO:teuthology.task.internal:Opening connections... 2024-09-10T21:37:52.814 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-09-10T21:37:52.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:37:52.918 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-09-10T21:37:52.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:37:52.999 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi171.front.sepia.ceph.com 2024-09-10T21:37:53.000 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:37:53.081 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T21:37:53.089 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-09-10T21:37:53.107 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-09-10T21:37:53.107 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-09-10T21:37:53.165 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-09-10T21:37:53.165 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-09-10T21:37:53.165 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-09-10T21:37:53.165 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T21:37:53.166 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T21:37:53.167 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-09-10T21:37:53.192 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-09-10T21:37:53.211 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-09-10T21:37:53.211 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-09-10T21:37:53.269 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2024-09-10T21:37:53.269 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2024-09-10T21:37:53.269 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T21:37:53.270 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T21:37:53.270 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-09-10T21:37:53.301 DEBUG:teuthology.orchestra.run.smithi171:> uname -m 2024-09-10T21:37:53.320 INFO:teuthology.orchestra.run.smithi171.stdout:x86_64 2024-09-10T21:37:53.320 DEBUG:teuthology.orchestra.run.smithi171:> cat /etc/os-release 2024-09-10T21:37:53.377 INFO:teuthology.orchestra.run.smithi171.stdout:NAME="CentOS Stream" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION="9" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:ID="centos" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:ID_LIKE="rhel fedora" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:VERSION_ID="9" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:PLATFORM_ID="platform:el9" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:ANSI_COLOR="0;31" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:LOGO="fedora-logo-icon" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T21:37:53.378 INFO:teuthology.orchestra.run.smithi171.stdout:HOME_URL="https://centos.org/" 2024-09-10T21:37:53.379 INFO:teuthology.orchestra.run.smithi171.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T21:37:53.379 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T21:37:53.379 INFO:teuthology.orchestra.run.smithi171.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T21:37:53.379 INFO:teuthology.lock.ops:Updating smithi171.front.sepia.ceph.com on lock server 2024-09-10T21:37:53.409 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T21:37:53.420 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T21:37:53.428 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T21:37:53.428 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:37:53.430 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:37:53.432 DEBUG:teuthology.orchestra.run.smithi171:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:37:53.449 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T21:37:53.457 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T21:37:53.457 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:37:53.488 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:37:53.490 DEBUG:teuthology.orchestra.run.smithi171:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:37:53.521 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T21:37:53.632 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T21:37:53.646 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T21:37:53.647 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'mon.c': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, timeout 300 2024-09-10T21:37:53.648 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:37:53.649 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:37:53.650 DEBUG:teuthology.orchestra.run.smithi171:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:37:53.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:37:53.667 DEBUG:teuthology.task.kernel:sha1 for mon.b is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:37:53.667 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:37:53.667 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-09-10T21:37:53.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:37:53.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:37:53.669 DEBUG:teuthology.task.kernel:sha1 for mon.c is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:37:53.669 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:37:53.669 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-10T21:37:53.669 DEBUG:teuthology.task.kernel:sha1 for mon.a is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:37:53.669 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:37:53.670 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-09-10T21:37:53.724 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-503.el9.x86_64 2024-09-10T21:37:53.725 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:37:53.725 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:37:53.725 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:37:53.737 INFO:teuthology.orchestra.run.smithi171.stdout:5.14.0-503.el9.x86_64 2024-09-10T21:37:53.737 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-503.el9.x86_64 2024-09-10T21:37:53.737 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:37:53.738 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:37:53.738 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:37:53.739 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:37:53.739 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:37:53.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:37:53.884 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum upgrade -y linux-firmware 2024-09-10T21:37:53.885 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum upgrade -y linux-firmware 2024-09-10T21:37:53.900 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum upgrade -y linux-firmware 2024-09-10T21:37:57.513 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-09-10T21:37:57.709 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.2 MB 00:02 2024-09-10T21:37:59.664 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-10T21:38:00.112 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.2 MB 00:04 2024-09-10T21:38:02.114 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-10T21:38:05.048 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.5 MB 00:01 2024-09-10T21:38:06.649 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-10T21:38:06.809 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-10T21:38:08.015 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.5 MB 00:02 2024-09-10T21:38:08.136 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-10T21:38:09.433 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 89 kB/s | 19 kB 00:00 2024-09-10T21:38:10.905 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-10T21:38:13.068 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.5 MB 00:02 2024-09-10T21:38:13.525 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-09-10T21:38:15.252 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-10T21:38:15.666 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-10T21:38:15.667 INFO:teuthology.orchestra.run.smithi171.stdout:Nothing to do. 2024-09-10T21:38:15.667 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-10T21:38:15.780 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.c... 2024-09-10T21:38:15.780 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:38:15.941 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T21:38:15.942 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T21:38:15.944 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi171.front.sepia.ceph.com to complete... 2024-09-10T21:38:16.337 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T21:38:16.472 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-10T21:38:18.496 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-09-10T21:38:18.497 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-09-10T21:38:18.497 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-09-10T21:38:18.623 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T21:38:18.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:38:18.799 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T21:38:18.799 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T21:38:18.801 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi090.front.sepia.ceph.com to complete... 2024-09-10T21:38:21.870 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T21:38:23.983 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-09-10T21:38:23.984 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-09-10T21:38:23.984 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-09-10T21:38:24.086 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T21:38:24.087 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:38:24.276 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T21:38:24.277 DEBUG:teuthology.orchestra.run.smithi154:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T21:38:24.279 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi154.front.sepia.ceph.com to complete... 2024-09-10T21:38:26.136 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10 21:38:26 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T21:38:26.136 INFO:teuthology.orchestra.run.smithi171.stderr:FINISHED --2024-09-10 21:38:26-- 2024-09-10T21:38:26.136 INFO:teuthology.orchestra.run.smithi171.stderr:Total wall clock time: 10s 2024-09-10T21:38:26.137 INFO:teuthology.orchestra.run.smithi171.stderr:Downloaded: 1 files, 343M in 9.9s (34.7 MB/s) 2024-09-10T21:38:26.139 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.c... 2024-09-10T21:38:26.139 DEBUG:teuthology.orchestra.run.smithi171:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T21:38:28.270 INFO:teuthology.orchestra.run.smithi171.stdout:Verifying... ######################################## 2024-09-10T21:38:28.548 INFO:teuthology.orchestra.run.smithi171.stdout:Preparing... ######################################## 2024-09-10T21:38:28.663 INFO:teuthology.orchestra.run.smithi171.stdout:Updating / installing... 2024-09-10T21:38:36.707 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T21:38:44.713 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10 21:38:44 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T21:38:44.714 INFO:teuthology.orchestra.run.smithi154.stderr:FINISHED --2024-09-10 21:38:44-- 2024-09-10T21:38:44.714 INFO:teuthology.orchestra.run.smithi154.stderr:Total wall clock time: 20s 2024-09-10T21:38:44.714 INFO:teuthology.orchestra.run.smithi154.stderr:Downloaded: 1 files, 343M in 20s (17.3 MB/s) 2024-09-10T21:38:44.717 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T21:38:44.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T21:38:46.825 INFO:teuthology.orchestra.run.smithi154.stdout:Verifying... ######################################## 2024-09-10T21:38:47.067 INFO:teuthology.orchestra.run.smithi154.stdout:Preparing... ######################################## 2024-09-10T21:38:47.170 INFO:teuthology.orchestra.run.smithi154.stdout:Updating / installing... 2024-09-10T21:38:48.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10 21:38:48 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T21:38:48.965 INFO:teuthology.orchestra.run.smithi090.stderr:FINISHED --2024-09-10 21:38:48-- 2024-09-10T21:38:48.965 INFO:teuthology.orchestra.run.smithi090.stderr:Total wall clock time: 30s 2024-09-10T21:38:48.965 INFO:teuthology.orchestra.run.smithi090.stderr:Downloaded: 1 files, 343M in 30s (11.5 MB/s) 2024-09-10T21:38:48.968 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T21:38:48.968 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T21:38:51.097 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying... ######################################## 2024-09-10T21:38:51.729 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing... ######################################## 2024-09-10T21:38:53.048 INFO:teuthology.orchestra.run.smithi090.stdout:Updating / installing... 2024-09-10T21:38:55.271 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T21:39:01.033 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T21:39:17.521 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi171.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T21:39:17.522 DEBUG:teuthology.orchestra.run.smithi171:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T21:39:17.602 INFO:teuthology.orchestra.run.smithi171.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:17.602 INFO:teuthology.orchestra.run.smithi171.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:17.602 INFO:teuthology.orchestra.run.smithi171.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:17.602 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T21:39:17.603 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T21:39:17.604 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:39:17.605 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:39:17.606 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:39:17.607 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:39:17.608 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:39:17.609 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:39:17.610 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:39:17.611 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:39:17.612 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:39:17.613 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T21:39:17.614 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:39:17.615 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:39:17.616 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:39:17.617 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:39:17.618 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T21:39:17.619 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T21:39:17.620 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:39:17.621 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:39:17.622 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:39:17.623 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:39:17.624 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:39:17.625 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:39:17.626 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:39:17.627 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:39:17.628 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:39:17.629 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:39:17.630 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:39:17.631 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:39:17.632 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:39:17.633 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:39:17.634 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:39:17.635 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:39:17.636 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:39:17.637 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:39:17.638 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T21:39:17.639 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:39:17.640 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:39:17.641 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:39:17.642 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:39:17.643 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:39:17.644 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:39:17.645 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:39:17.646 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T21:39:17.647 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:39:17.648 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T21:39:17.649 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T21:39:17.650 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:39:17.651 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T21:39:17.652 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T21:39:17.653 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T21:39:17.654 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T21:39:17.655 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:39:17.656 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T21:39:17.657 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:39:17.658 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:39:17.659 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:39:17.660 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:39:17.661 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:39:17.662 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:39:17.663 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:39:17.664 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:39:17.665 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:39:17.666 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:39:17.667 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:39:17.668 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T21:39:17.669 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T21:39:17.670 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T21:39:17.671 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:39:17.672 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:39:17.673 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:39:17.674 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:39:17.675 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:39:17.676 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T21:39:17.677 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:39:17.678 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:39:17.679 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:39:17.680 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T21:39:17.681 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T21:39:17.682 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:39:17.683 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:39:17.684 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T21:39:17.685 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:39:17.686 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T21:39:17.687 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:39:17.688 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T21:39:17.689 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:39:17.690 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:39:17.691 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:39:17.692 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:39:17.693 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:39:17.694 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T21:39:17.695 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:39:17.696 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:39:17.697 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:39:17.698 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:39:17.699 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:39:17.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:39:17.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:39:17.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:39:17.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:39:17.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:39:17.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:39:17.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:39:17.700 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:39:17.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:39:17.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T21:39:17.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T21:39:17.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:39:17.742 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T21:39:17.743 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:39:17.744 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T21:39:17.745 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:39:17.746 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:39:17.747 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:39:17.748 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:39:17.749 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:39:17.750 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T21:39:17.751 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T21:39:17.752 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T21:39:17.753 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T21:39:17.754 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T21:39:17.755 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T21:39:17.756 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T21:39:17.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:39:17.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:39:17.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:39:17.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:39:17.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:39:17.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:39:17.757 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:39:17.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:39:17.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T21:39:17.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:39:17.759 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:39:17.760 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:39:17.761 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:39:17.762 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:39:17.763 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:39:17.764 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:39:17.765 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:39:17.766 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:39:17.767 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:39:17.768 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:39:17.769 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:39:17.770 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T21:39:17.771 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:39:17.772 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:39:17.773 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:39:17.774 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:39:17.775 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:39:17.776 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:39:17.777 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:39:17.778 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:39:17.779 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:39:17.780 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:39:17.781 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:39:17.782 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:39:17.783 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:39:17.784 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:39:17.785 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:39:17.786 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:39:17.787 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T21:39:17.788 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:39:17.789 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:39:17.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T21:39:17.790 INFO:teuthology.orchestra.run.smithi171.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:39:17.794 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:17.794 DEBUG:teuthology.orchestra.run.smithi171:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout:fi 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout:for file in vmlinuz System.map config; do 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout: fi 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout:done 2024-09-10T21:39:17.817 INFO:teuthology.orchestra.run.smithi171.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout:fi 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout:fi 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:17.818 INFO:teuthology.orchestra.run.smithi171.stdout:fi 2024-09-10T21:39:17.819 DEBUG:teuthology.orchestra.run.smithi171:> sudo rpm -qi grub2-tools 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Name : grub2-tools 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Epoch : 1 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Version : 2.06 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Release : 82.el9 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Architecture: x86_64 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Group : Unspecified 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Size : 8274670 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:License : GPLv3+ 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T21:39:17.891 INFO:teuthology.orchestra.run.smithi171.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:Packager : builder@centos.org 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:Vendor : CentOS 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:Summary : Support tools for GRUB. 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:Description : 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout:hardware devices. 2024-09-10T21:39:17.892 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:39:17.893 INFO:teuthology.orchestra.run.smithi171.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T21:39:17.893 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T21:39:17.894 INFO:teuthology.task.kernel:Updating grub on smithi171 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:17.894 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T21:39:18.440 INFO:teuthology.orchestra.run.smithi171.stderr:Generating grub configuration file ... 2024-09-10T21:39:19.822 INFO:teuthology.orchestra.run.smithi171.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T21:39:19.850 INFO:teuthology.orchestra.run.smithi171.stderr:done 2024-09-10T21:39:19.853 DEBUG:teuthology.orchestra.run.smithi171:> mktemp 2024-09-10T21:39:19.868 INFO:teuthology.orchestra.run.smithi171.stdout:/tmp/tmp.XjtvnFonb5 2024-09-10T21:39:19.868 DEBUG:teuthology.orchestra.run.smithi171:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XjtvnFonb5 2024-09-10T21:39:19.934 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 0666 /tmp/tmp.XjtvnFonb5 2024-09-10T21:39:20.071 DEBUG:teuthology.orchestra.remote:smithi171:/tmp/tmp.XjtvnFonb5 is 6KB 2024-09-10T21:39:20.119 DEBUG:teuthology.orchestra.run.smithi171:> rm -fr /tmp/tmp.XjtvnFonb5 2024-09-10T21:39:20.135 DEBUG:teuthology.orchestra.run.smithi171:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T21:39:20.197 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T21:39:20.198 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T21:39:20.198 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T21:39:20.198 INFO:teuthology.orchestra.run.smithi171.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T21:39:20.199 DEBUG:teuthology.orchestra.run.smithi171:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:20.362 DEBUG:teuthology.orchestra.run.smithi171:> sudo shutdown -r now 2024-09-10T21:39:35.094 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T21:39:35.095 DEBUG:teuthology.orchestra.run.smithi154:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T21:39:35.181 INFO:teuthology.orchestra.run.smithi154.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:35.181 INFO:teuthology.orchestra.run.smithi154.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:39:35.182 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:39:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:39:35.184 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:39:35.185 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:39:35.186 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:39:35.187 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:39:35.188 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:39:35.189 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:39:35.190 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:39:35.191 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:39:35.192 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T21:39:35.193 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T21:39:35.194 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:39:35.195 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:39:35.196 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T21:39:35.197 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T21:39:35.198 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:39:35.199 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:39:35.200 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:39:35.201 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:39:35.202 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:39:35.203 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:39:35.204 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:39:35.205 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:39:35.206 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:39:35.207 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:39:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:39:35.209 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:39:35.210 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:39:35.211 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:39:35.212 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:39:35.213 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:39:35.213 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:39:35.213 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:39:35.213 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:39:35.213 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:39:35.213 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:39:35.213 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:39:35.213 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:39:35.213 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:39:35.214 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T21:39:35.215 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:39:35.216 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:39:35.217 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:39:35.218 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:39:35.219 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T21:39:35.220 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:39:35.221 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:39:35.222 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:39:35.223 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:39:35.224 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T21:39:35.225 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T21:39:35.226 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:39:35.227 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T21:39:35.228 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T21:39:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T21:39:35.230 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T21:39:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:39:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T21:39:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:39:35.234 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T21:39:35.235 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:39:35.236 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:39:35.237 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:39:35.238 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:39:35.239 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T21:39:35.240 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:39:35.241 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T21:39:35.242 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:39:35.243 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:39:35.244 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:39:35.245 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:39:35.246 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:39:35.247 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:39:35.248 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T21:39:35.249 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:39:35.250 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:39:35.251 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T21:39:35.252 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:39:35.253 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:39:35.254 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:39:35.255 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:39:35.256 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T21:39:35.257 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:39:35.258 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T21:39:35.259 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T21:39:35.260 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T21:39:35.261 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:39:35.262 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:39:35.263 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T21:39:35.264 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:39:35.265 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:39:35.266 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:39:35.267 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:39:35.268 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:39:35.269 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:39:35.270 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:39:35.271 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:39:35.272 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T21:39:35.273 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:39:35.274 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:39:35.275 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:39:35.276 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:39:35.277 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T21:39:35.278 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:39:35.279 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:39:35.280 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:39:35.281 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:39:35.282 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:39:35.283 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:39:35.284 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:39:35.285 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:39:35.286 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:39:35.287 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T21:39:35.288 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:39:35.289 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T21:39:35.290 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:39:35.291 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:39:35.292 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:39:35.293 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:39:35.294 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:39:35.295 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:39:35.296 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:39:35.297 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:39:35.298 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:39:35.299 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:39:35.300 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:39:35.301 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T21:39:35.302 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:39:35.303 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:39:35.304 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:39:35.305 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:39:35.306 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:39:35.307 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:39:35.308 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:39:35.309 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:39:35.310 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:39:35.311 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:39:35.312 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T21:39:35.313 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:39:35.314 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:39:35.315 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:39:35.316 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T21:39:35.317 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:39:35.318 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:39:35.319 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T21:39:35.320 INFO:teuthology.orchestra.run.smithi154.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:39:35.324 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:35.324 DEBUG:teuthology.orchestra.run.smithi154:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T21:39:35.346 INFO:teuthology.orchestra.run.smithi154.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T21:39:35.346 INFO:teuthology.orchestra.run.smithi154.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout:fi 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout:for file in vmlinuz System.map config; do 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: fi 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout:done 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout:fi 2024-09-10T21:39:35.347 INFO:teuthology.orchestra.run.smithi154.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T21:39:35.348 INFO:teuthology.orchestra.run.smithi154.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:35.348 INFO:teuthology.orchestra.run.smithi154.stdout:fi 2024-09-10T21:39:35.348 INFO:teuthology.orchestra.run.smithi154.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T21:39:35.348 INFO:teuthology.orchestra.run.smithi154.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:39:35.348 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:35.348 INFO:teuthology.orchestra.run.smithi154.stdout:fi 2024-09-10T21:39:35.349 DEBUG:teuthology.orchestra.run.smithi154:> sudo rpm -qi grub2-tools 2024-09-10T21:39:35.421 INFO:teuthology.orchestra.run.smithi154.stdout:Name : grub2-tools 2024-09-10T21:39:35.421 INFO:teuthology.orchestra.run.smithi154.stdout:Epoch : 1 2024-09-10T21:39:35.421 INFO:teuthology.orchestra.run.smithi154.stdout:Version : 2.06 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Release : 82.el9 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: x86_64 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Group : Unspecified 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Size : 8274670 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:License : GPLv3+ 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Packager : builder@centos.org 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Vendor : CentOS 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T21:39:35.422 INFO:teuthology.orchestra.run.smithi154.stdout:Summary : Support tools for GRUB. 2024-09-10T21:39:35.423 INFO:teuthology.orchestra.run.smithi154.stdout:Description : 2024-09-10T21:39:35.423 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:39:35.423 INFO:teuthology.orchestra.run.smithi154.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T21:39:35.423 INFO:teuthology.orchestra.run.smithi154.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T21:39:35.423 INFO:teuthology.orchestra.run.smithi154.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T21:39:35.423 INFO:teuthology.orchestra.run.smithi154.stdout:hardware devices. 2024-09-10T21:39:35.423 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:39:35.423 INFO:teuthology.orchestra.run.smithi154.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T21:39:35.424 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T21:39:35.424 INFO:teuthology.task.kernel:Updating grub on smithi154 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:35.424 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T21:39:36.082 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2024-09-10T21:39:37.401 INFO:teuthology.orchestra.run.smithi154.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T21:39:37.441 INFO:teuthology.orchestra.run.smithi154.stderr:done 2024-09-10T21:39:37.444 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-09-10T21:39:37.459 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.12MtYvnntl 2024-09-10T21:39:37.459 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.12MtYvnntl 2024-09-10T21:39:37.526 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.12MtYvnntl 2024-09-10T21:39:37.676 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.12MtYvnntl is 6KB 2024-09-10T21:39:37.696 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.12MtYvnntl 2024-09-10T21:39:37.711 DEBUG:teuthology.orchestra.run.smithi154:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T21:39:37.774 INFO:teuthology.orchestra.run.smithi154.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T21:39:37.774 INFO:teuthology.orchestra.run.smithi154.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T21:39:37.774 INFO:teuthology.orchestra.run.smithi154.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T21:39:37.774 INFO:teuthology.orchestra.run.smithi154.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T21:39:37.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:37.952 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2024-09-10T21:39:43.989 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T21:39:43.989 DEBUG:teuthology.orchestra.run.smithi090:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T21:39:44.121 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:44.121 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:44.121 INFO:teuthology.orchestra.run.smithi090.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:44.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:44.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T21:39:44.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T21:39:44.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T21:39:44.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T21:39:44.121 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:39:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:39:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:39:44.124 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:39:44.125 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:39:44.126 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:39:44.127 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:39:44.128 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:39:44.129 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:39:44.130 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:39:44.131 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T21:39:44.132 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:39:44.133 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:39:44.134 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:39:44.135 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T21:39:44.136 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T21:39:44.137 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:39:44.138 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:39:44.139 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:39:44.140 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:39:44.141 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:39:44.142 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:39:44.143 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:39:44.144 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:39:44.145 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:39:44.146 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:39:44.147 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:39:44.148 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:39:44.149 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:39:44.150 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:39:44.151 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:39:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:39:44.153 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:39:44.154 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:39:44.155 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:39:44.156 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:39:44.157 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T21:39:44.158 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:39:44.159 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:39:44.160 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:39:44.161 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:39:44.162 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T21:39:44.163 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:39:44.164 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:39:44.165 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T21:39:44.166 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:39:44.167 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T21:39:44.168 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T21:39:44.169 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:39:44.170 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:39:44.171 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T21:39:44.172 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T21:39:44.173 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T21:39:44.174 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:39:44.175 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T21:39:44.176 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:39:44.177 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:39:44.178 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T21:39:44.179 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T21:39:44.180 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:39:44.181 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:39:44.182 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:39:44.183 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:39:44.184 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T21:39:44.185 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:39:44.186 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:39:44.187 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T21:39:44.188 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:39:44.189 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:39:44.190 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:39:44.191 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:39:44.192 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T21:39:44.193 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:39:44.194 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:39:44.195 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:39:44.196 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T21:39:44.197 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:39:44.198 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:39:44.199 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:39:44.200 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T21:39:44.201 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:39:44.202 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:39:44.203 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:39:44.204 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:39:44.205 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:39:44.206 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:39:44.207 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:39:44.208 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:39:44.209 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:39:44.210 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T21:39:44.211 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:39:44.212 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:39:44.213 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T21:39:44.214 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:39:44.215 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T21:39:44.216 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:39:44.217 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:39:44.218 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:39:44.219 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:39:44.220 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:39:44.221 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:39:44.222 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:39:44.223 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:39:44.224 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T21:39:44.225 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:39:44.226 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:39:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:39:44.228 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:39:44.229 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:39:44.230 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T21:39:44.231 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:39:44.232 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:39:44.233 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:39:44.234 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:39:44.235 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:39:44.236 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:39:44.237 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:39:44.238 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T21:39:44.239 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:39:44.240 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:39:44.241 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:39:44.242 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:39:44.243 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:39:44.244 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:39:44.245 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:39:44.246 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:39:44.247 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:39:44.248 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:39:44.249 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T21:39:44.250 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T21:39:44.251 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:39:44.252 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:39:44.253 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T21:39:44.254 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T21:39:44.255 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:39:44.256 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T21:39:44.257 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:39:44.261 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:44.261 DEBUG:teuthology.orchestra.run.smithi090:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T21:39:44.287 INFO:teuthology.orchestra.run.smithi090.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout:for file in vmlinuz System.map config; do 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout: fi 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout:done 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:39:44.288 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-09-10T21:39:44.289 INFO:teuthology.orchestra.run.smithi090.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T21:39:44.289 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:44.289 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-09-10T21:39:44.289 INFO:teuthology.orchestra.run.smithi090.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T21:39:44.289 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:39:44.289 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:44.289 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-09-10T21:39:44.290 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -qi grub2-tools 2024-09-10T21:39:44.367 INFO:teuthology.orchestra.run.smithi090.stdout:Name : grub2-tools 2024-09-10T21:39:44.367 INFO:teuthology.orchestra.run.smithi090.stdout:Epoch : 1 2024-09-10T21:39:44.367 INFO:teuthology.orchestra.run.smithi090.stdout:Version : 2.06 2024-09-10T21:39:44.367 INFO:teuthology.orchestra.run.smithi090.stdout:Release : 82.el9 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: x86_64 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Group : Unspecified 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Size : 8274670 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:License : GPLv3+ 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Packager : builder@centos.org 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Vendor : CentOS 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Summary : Support tools for GRUB. 2024-09-10T21:39:44.368 INFO:teuthology.orchestra.run.smithi090.stdout:Description : 2024-09-10T21:39:44.369 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:39:44.369 INFO:teuthology.orchestra.run.smithi090.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T21:39:44.369 INFO:teuthology.orchestra.run.smithi090.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T21:39:44.369 INFO:teuthology.orchestra.run.smithi090.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T21:39:44.369 INFO:teuthology.orchestra.run.smithi090.stdout:hardware devices. 2024-09-10T21:39:44.369 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:39:44.369 INFO:teuthology.orchestra.run.smithi090.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T21:39:44.370 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T21:39:44.370 INFO:teuthology.task.kernel:Updating grub on smithi090 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:44.370 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T21:39:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2024-09-10T21:39:46.959 INFO:teuthology.orchestra.run.smithi090.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T21:39:46.981 INFO:teuthology.orchestra.run.smithi090.stderr:done 2024-09-10T21:39:46.983 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-09-10T21:39:47.011 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.2RUbz17Uog 2024-09-10T21:39:47.012 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.2RUbz17Uog 2024-09-10T21:39:47.082 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.2RUbz17Uog 2024-09-10T21:39:47.231 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.2RUbz17Uog is 6KB 2024-09-10T21:39:47.280 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.2RUbz17Uog 2024-09-10T21:39:47.295 DEBUG:teuthology.orchestra.run.smithi090:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T21:39:47.373 INFO:teuthology.orchestra.run.smithi090.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T21:39:47.373 INFO:teuthology.orchestra.run.smithi090.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T21:39:47.374 INFO:teuthology.orchestra.run.smithi090.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T21:39:47.374 INFO:teuthology.orchestra.run.smithi090.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T21:39:47.375 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:39:47.531 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2024-09-10T21:39:50.365 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-10T21:39:50.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-10T21:39:50.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:40:07.973 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T21:40:07.973 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-10T21:40:07.974 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:40:17.545 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T21:40:17.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-10T21:40:17.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:40:25.021 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-10T21:40:26.301 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-09-10T21:40:34.030 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-10T21:40:34.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:40:35.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-10T21:40:35.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:40:36.029 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-09-10T21:40:38.365 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-09-10T21:40:45.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-10T21:40:45.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:40:48.093 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-09-10T21:40:50.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-10T21:40:50.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:40:52.417 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-10T21:41:00.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-10T21:41:00.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:41:03.197 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-09-10T21:41:04.420 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-10T21:41:04.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:41:07.485 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.171 2024-09-10T21:41:18.205 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-10T21:41:18.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:41:22.490 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-10T21:41:22.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:41:22.837 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-10T21:41:23.457 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-10T21:41:23.457 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:41:23.457 DEBUG:teuthology.orchestra.run.smithi171:> uname -r 2024-09-10T21:41:23.491 INFO:teuthology.orchestra.run.smithi171.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:41:23.491 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:41:23.491 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:41:23.491 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T21:41:24.492 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-10T21:41:24.492 DEBUG:teuthology.orchestra.run.smithi171:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T21:41:24.636 INFO:teuthology.orchestra.run.smithi171.stdout:ttyS1 2024-09-10T21:41:24.663 DEBUG:teuthology.parallel:result is None 2024-09-10T21:41:50.330 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-09-10T21:41:50.378 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T21:41:51.267 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-10T21:41:51.267 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:41:51.267 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-09-10T21:41:51.302 INFO:teuthology.orchestra.run.smithi090.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:41:51.383 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:41:51.384 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:41:51.384 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T21:41:52.385 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T21:41:52.385 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T21:41:52.632 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2024-09-10T21:41:52.662 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:05.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-10T21:42:05.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:42:05.714 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-09-10T21:42:06.243 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-10T21:42:06.243 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:42:06.243 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-09-10T21:42:06.277 INFO:teuthology.orchestra.run.smithi154.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:42:06.278 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:42:06.278 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:42:06.278 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T21:42:07.278 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T21:42:07.279 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T21:42:07.383 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2024-09-10T21:42:07.423 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:07.423 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T21:42:07.430 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T21:42:07.430 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:42:07.435 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:42:07.439 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:42:07.478 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T21:42:07.484 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T21:42:07.493 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T21:42:07.496 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:42:07.513 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:42:07.516 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:42:07.576 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T21:42:07.582 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T21:42:07.583 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:42:07.587 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:42:07.602 DEBUG:teuthology.orchestra.run.smithi171:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:42:07.670 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:42:07.675 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:42:07.693 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:42:07.697 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:42:07.757 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:42:07.781 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:42:07.784 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T21:42:07.791 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T21:42:07.791 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:42:07.795 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:42:07.799 DEBUG:teuthology.orchestra.run.smithi171:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:42:07.856 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T21:42:07.865 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T21:42:07.865 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:42:07.869 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:42:07.873 DEBUG:teuthology.orchestra.run.smithi171:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:42:07.907 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:42:08.015 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:42:08.085 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:42:08.158 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:42:08.218 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:42:08.218 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:42:08.272 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:42:08.345 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:42:08.442 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:42:08.511 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:42:08.574 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:42:08.574 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:42:08.628 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:42:08.700 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:42:08.798 DEBUG:teuthology.orchestra.run.smithi171:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:42:08.868 DEBUG:teuthology.orchestra.run.smithi171:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:42:08.931 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:42:08.931 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:42:08.986 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-09-10T21:42:08.990 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-09-10T21:42:08.994 DEBUG:teuthology.orchestra.run.smithi171:> sudo service rsyslog restart 2024-09-10T21:42:09.072 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T21:42:09.074 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T21:42:09.108 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T21:42:09.601 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T21:42:09.609 INFO:teuthology.task.internal:Starting timer... 2024-09-10T21:42:09.609 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T21:42:09.620 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T21:42:09.628 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T21:42:09.629 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-09-10T21:42:09.721 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-09-10T21:42:09.726 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-09-10T21:42:09.818 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-09-10T21:42:09.821 DEBUG:teuthology.orchestra.run.smithi171:> sudo service auditd rotate 2024-09-10T21:42:09.952 INFO:teuthology.orchestra.run.smithi171.stdout:Rotating logs: 2024-09-10T21:42:09.955 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T21:42:09.955 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-09-10T21:42:09.999 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-09-10T21:42:10.000 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-09-10T21:42:10.041 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-09-10T21:42:10.042 DEBUG:teuthology.orchestra.run.smithi171:> /usr/sbin/getenforce 2024-09-10T21:42:10.080 INFO:teuthology.orchestra.run.smithi171.stdout:Permissive 2024-09-10T21:42:10.080 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi154.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi171.front.sepia.ceph.com': 'permissive'} 2024-09-10T21:42:10.081 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T21:42:10.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:42:10.138 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T21:42:10.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:42:10.198 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T21:42:10.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:42:10.257 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T21:42:10.257 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-09-10T21:42:10.340 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-09-10T21:42:10.407 DEBUG:teuthology.orchestra.run.smithi171:> sudo /usr/sbin/setenforce permissive 2024-09-10T21:42:10.480 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T21:42:10.489 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T21:42:10.500 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T21:42:10.501 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi154.front.sepia.ceph.com,smithi171.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T21:46:56.113 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi154.front.sepia.ceph.com'), Remote(name='ubuntu@smithi171.front.sepia.ceph.com')] 2024-09-10T21:46:56.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-10T21:46:56.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:46:56.210 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-09-10T21:46:56.310 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-09-10T21:46:56.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-10T21:46:56.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:46:56.404 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-09-10T21:46:56.515 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-09-10T21:46:56.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-10T21:46:56.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi171.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:46:56.607 DEBUG:teuthology.orchestra.run.smithi171:> true 2024-09-10T21:46:56.719 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi171.front.sepia.ceph.com' 2024-09-10T21:46:56.720 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T21:46:56.731 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T21:46:56.731 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:46:56.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:46:56.736 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:46:56.736 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:46:56.740 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:46:56.740 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:46:56.794 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T21:46:56.797 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T21:46:56.820 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T21:46:56.823 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T21:46:56.827 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T21:46:56.850 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T21:46:56.874 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-09-10T21:46:56.878 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-09-10T21:46:56.898 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-09-10T21:46:56.899 INFO:teuthology.orchestra.run.smithi171.stderr:sudo: ntpd: command not found 2024-09-10T21:46:56.901 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-09-10T21:46:56.923 INFO:teuthology.orchestra.run.smithi171.stdout:506 Cannot talk to daemon 2024-09-10T21:46:56.927 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T21:46:56.931 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T21:46:56.952 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T21:46:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T21:46:56.960 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T21:46:56.982 INFO:teuthology.orchestra.run.smithi171.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T21:46:57.043 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-09-10T21:46:57.050 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T21:46:57.050 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-09-10T21:46:57.050 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.050 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.050 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.050 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.056 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-09-10T21:46:57.063 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T21:46:57.063 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-09-10T21:46:57.063 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.063 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.063 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.063 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.074 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-10T21:46:57.081 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T21:46:57.081 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-10T21:46:57.082 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.082 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.082 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.082 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:46:57.083 INFO:teuthology.run_tasks:Running task install... 2024-09-10T21:46:57.091 DEBUG:teuthology.task.install:project ceph 2024-09-10T21:46:57.091 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T21:46:57.091 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T21:46:57.091 INFO:teuthology.task.install:Using flavor: default 2024-09-10T21:46:57.097 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T21:46:57.097 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T21:46:57.097 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T21:46:57.098 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:46:57.100 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T21:46:57.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:46:57.101 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T21:46:57.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:46:57.257 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T21:46:57.257 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T21:46:57.270 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T21:46:57.270 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T21:46:57.271 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T21:46:57.271 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T21:46:57.383 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T21:46:57.383 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:46:57.383 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T21:46:57.390 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T21:46:57.390 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:46:57.391 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T21:46:57.398 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T21:46:57.399 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:46:57.399 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T21:46:57.440 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T21:46:57.440 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T21:46:57.447 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T21:46:57.447 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T21:46:57.456 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T21:46:57.456 DEBUG:teuthology.orchestra.run.smithi171:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T21:46:57.537 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T21:46:57.545 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T21:46:57.553 DEBUG:teuthology.orchestra.run.smithi171:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T21:46:57.614 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T21:46:57.623 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T21:46:57.631 DEBUG:teuthology.orchestra.run.smithi171:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T21:46:57.669 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-09-10T21:46:57.672 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-09-10T21:46:57.678 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-09-10T21:46:57.681 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-09-10T21:46:57.687 INFO:teuthology.orchestra.run.smithi171.stdout:check_obsoletes = 1 2024-09-10T21:46:57.690 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum clean all 2024-09-10T21:46:57.942 INFO:teuthology.orchestra.run.smithi090.stdout:45 files removed 2024-09-10T21:46:57.956 INFO:teuthology.orchestra.run.smithi154.stdout:45 files removed 2024-09-10T21:46:57.968 INFO:teuthology.orchestra.run.smithi171.stdout:45 files removed 2024-09-10T21:46:57.975 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T21:46:57.989 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T21:46:58.001 DEBUG:teuthology.orchestra.run.smithi171:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 289 kB/s | 84 kB 00:00 2024-09-10T21:46:58.692 INFO:teuthology.orchestra.run.smithi171.stdout:ceph packages for x86_64 290 kB/s | 84 kB 00:00 2024-09-10T21:46:58.700 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 268 kB/s | 84 kB 00:00 2024-09-10T21:46:58.969 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-09-10T21:46:59.002 INFO:teuthology.orchestra.run.smithi171.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-09-10T21:46:59.002 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-09-10T21:46:59.253 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T21:46:59.268 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-10T21:46:59.276 INFO:teuthology.orchestra.run.smithi171.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-10T21:46:59.768 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-09-10T21:46:59.877 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-10T21:47:01.296 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.2 MB 00:02 2024-09-10T21:47:01.927 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-10T21:47:03.520 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 20 MB 00:02 2024-09-10T21:47:04.156 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-10T21:47:08.205 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - CRB 9.8 MB/s | 6.5 MB 00:00 2024-09-10T21:47:09.736 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.5 MB 00:03 2024-09-10T21:47:10.546 INFO:teuthology.orchestra.run.smithi171.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-10T21:47:11.899 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.5 MB 00:03 2024-09-10T21:47:12.078 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-10T21:47:12.244 INFO:teuthology.orchestra.run.smithi171.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-10T21:47:13.391 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-10T21:47:13.651 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-10T21:47:14.947 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-10T21:47:18.692 INFO:teuthology.orchestra.run.smithi171.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-10T21:47:19.763 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-10T21:47:21.213 INFO:teuthology.orchestra.run.smithi171.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:47:21.214 INFO:teuthology.orchestra.run.smithi171.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:47:21.225 INFO:teuthology.orchestra.run.smithi171.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T21:47:21.226 INFO:teuthology.orchestra.run.smithi171.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T21:47:21.227 INFO:teuthology.orchestra.run.smithi171.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T21:47:21.227 INFO:teuthology.orchestra.run.smithi171.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T21:47:21.229 INFO:teuthology.orchestra.run.smithi171.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T21:47:21.232 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T21:47:21.232 INFO:teuthology.orchestra.run.smithi171.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T21:47:21.298 INFO:teuthology.orchestra.run.smithi171.stdout:Dependencies resolved. 2024-09-10T21:47:21.317 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: Package Arch Version Repository Size 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout:Installing: 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T21:47:21.318 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T21:47:21.319 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T21:47:21.320 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrading: 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T21:47:21.321 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout:Installing dependencies: 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T21:47:21.322 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T21:47:21.323 INFO:teuthology.orchestra.run.smithi171.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T21:47:21.324 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T21:47:21.325 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T21:47:21.326 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T21:47:21.327 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T21:47:21.328 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T21:47:21.329 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout:Installing weak dependencies: 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction Summary 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout:======================================================================================= 2024-09-10T21:47:21.330 INFO:teuthology.orchestra.run.smithi171.stdout:Install 149 Packages 2024-09-10T21:47:21.331 INFO:teuthology.orchestra.run.smithi171.stdout:Upgrade 8 Packages 2024-09-10T21:47:21.331 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:47:21.331 INFO:teuthology.orchestra.run.smithi171.stdout:Total download size: 240 M 2024-09-10T21:47:21.331 INFO:teuthology.orchestra.run.smithi171.stdout:Downloading Packages: 2024-09-10T21:47:21.346 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T21:47:22.011 INFO:teuthology.orchestra.run.smithi171.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 44 kB/s | 6.6 kB 00:00 2024-09-10T21:47:22.177 INFO:teuthology.orchestra.run.smithi171.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 5.3 MB/s | 900 kB 00:00 2024-09-10T21:47:22.278 INFO:teuthology.orchestra.run.smithi171.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.4 MB/s | 145 kB 00:00 2024-09-10T21:47:22.318 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:47:22.319 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:47:22.332 INFO:teuthology.orchestra.run.smithi154.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T21:47:22.334 INFO:teuthology.orchestra.run.smithi154.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T21:47:22.334 INFO:teuthology.orchestra.run.smithi154.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T21:47:22.335 INFO:teuthology.orchestra.run.smithi154.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T21:47:22.337 INFO:teuthology.orchestra.run.smithi154.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T21:47:22.340 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T21:47:22.341 INFO:teuthology.orchestra.run.smithi154.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T21:47:22.370 INFO:teuthology.orchestra.run.smithi171.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 11 MB/s | 5.5 MB 00:00 2024-09-10T21:47:22.407 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-09-10T21:47:22.433 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-09-10T21:47:22.433 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-09-10T21:47:22.433 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T21:47:22.434 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T21:47:22.435 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T21:47:22.435 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T21:47:22.435 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T21:47:22.435 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T21:47:22.435 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T21:47:22.435 INFO:teuthology.orchestra.run.smithi154.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T21:47:22.435 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T21:47:22.435 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T21:47:22.436 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T21:47:22.436 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T21:47:22.436 INFO:teuthology.orchestra.run.smithi154.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T21:47:22.436 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T21:47:22.436 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T21:47:22.436 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T21:47:22.436 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T21:47:22.436 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T21:47:22.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T21:47:22.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T21:47:22.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T21:47:22.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T21:47:22.437 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T21:47:22.437 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:47:22.437 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T21:47:22.438 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T21:47:22.438 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-09-10T21:47:22.438 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T21:47:22.438 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T21:47:22.438 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T21:47:22.438 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T21:47:22.438 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T21:47:22.438 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:47:22.439 INFO:teuthology.orchestra.run.smithi154.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T21:47:22.439 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T21:47:22.439 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-09-10T21:47:22.439 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T21:47:22.439 INFO:teuthology.orchestra.run.smithi154.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T21:47:22.439 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T21:47:22.439 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T21:47:22.439 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T21:47:22.440 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T21:47:22.440 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T21:47:22.440 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T21:47:22.440 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T21:47:22.440 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T21:47:22.440 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T21:47:22.440 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T21:47:22.440 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T21:47:22.441 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T21:47:22.441 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T21:47:22.441 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T21:47:22.441 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T21:47:22.441 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T21:47:22.441 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T21:47:22.441 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T21:47:22.442 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T21:47:22.442 INFO:teuthology.orchestra.run.smithi154.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T21:47:22.442 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T21:47:22.442 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T21:47:22.442 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T21:47:22.442 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T21:47:22.442 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T21:47:22.442 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T21:47:22.443 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:47:22.443 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T21:47:22.443 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T21:47:22.443 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T21:47:22.443 INFO:teuthology.orchestra.run.smithi154.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T21:47:22.443 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T21:47:22.443 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T21:47:22.443 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T21:47:22.443 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T21:47:22.444 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T21:47:22.444 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T21:47:22.444 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T21:47:22.444 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T21:47:22.444 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T21:47:22.444 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T21:47:22.444 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T21:47:22.445 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T21:47:22.445 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T21:47:22.445 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T21:47:22.445 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T21:47:22.445 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T21:47:22.445 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T21:47:22.445 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T21:47:22.445 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T21:47:22.446 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T21:47:22.446 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T21:47:22.446 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T21:47:22.446 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T21:47:22.446 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T21:47:22.446 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T21:47:22.446 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T21:47:22.446 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T21:47:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T21:47:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T21:47:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T21:47:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T21:47:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T21:47:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T21:47:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T21:47:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T21:47:22.448 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T21:47:22.448 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T21:47:22.448 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T21:47:22.448 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T21:47:22.448 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T21:47:22.448 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T21:47:22.448 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T21:47:22.448 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T21:47:22.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T21:47:22.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T21:47:22.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T21:47:22.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T21:47:22.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T21:47:22.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T21:47:22.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T21:47:22.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T21:47:22.449 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T21:47:22.450 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2024-09-10T21:47:22.451 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T21:47:22.452 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:47:22.452 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-09-10T21:47:22.452 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-09-10T21:47:22.452 INFO:teuthology.orchestra.run.smithi154.stdout:Install 149 Packages 2024-09-10T21:47:22.452 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 8 Packages 2024-09-10T21:47:22.452 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:47:22.452 INFO:teuthology.orchestra.run.smithi171.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 14 MB/s | 2.4 MB 00:00 2024-09-10T21:47:22.453 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 240 M 2024-09-10T21:47:22.453 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-09-10T21:47:22.504 INFO:teuthology.orchestra.run.smithi171.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 11 MB/s | 1.5 MB 00:00 2024-09-10T21:47:22.881 INFO:teuthology.orchestra.run.smithi154.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 46 kB/s | 6.6 kB 00:00 2024-09-10T21:47:22.888 INFO:teuthology.orchestra.run.smithi171.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 11 MB/s | 4.8 MB 00:00 2024-09-10T21:47:23.247 INFO:teuthology.orchestra.run.smithi171.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 16 MB/s | 22 MB 00:01 2024-09-10T21:47:23.281 INFO:teuthology.orchestra.run.smithi154.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 2.2 MB/s | 900 kB 00:00 2024-09-10T21:47:23.313 INFO:teuthology.orchestra.run.smithi171.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 379 kB/s | 25 kB 00:00 2024-09-10T21:47:23.348 INFO:teuthology.orchestra.run.smithi154.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 2.1 MB/s | 145 kB 00:00 2024-09-10T21:47:23.572 INFO:teuthology.orchestra.run.smithi171.stdout:(10/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 16 MB/s | 17 MB 00:01 2024-09-10T21:47:23.631 INFO:teuthology.orchestra.run.smithi171.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 3.8 MB/s | 225 kB 00:00 2024-09-10T21:47:23.765 INFO:teuthology.orchestra.run.smithi171.stdout:(12/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 12 MB/s | 11 MB 00:00 2024-09-10T21:47:23.782 INFO:teuthology.orchestra.run.smithi154.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 5.5 MB/s | 2.4 MB 00:00 2024-09-10T21:47:23.806 INFO:teuthology.orchestra.run.smithi171.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 154 kB/s | 27 kB 00:00 2024-09-10T21:47:23.849 INFO:teuthology.orchestra.run.smithi171.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 380 kB/s | 32 kB 00:00 2024-09-10T21:47:23.884 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:47:23.885 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:47:23.891 INFO:teuthology.orchestra.run.smithi171.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 8.8 MB/s | 755 kB 00:00 2024-09-10T21:47:23.896 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T21:47:23.897 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T21:47:23.898 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T21:47:23.899 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T21:47:23.901 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T21:47:23.903 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T21:47:23.904 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T21:47:23.925 INFO:teuthology.orchestra.run.smithi171.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.1 MB/s | 162 kB 00:00 2024-09-10T21:47:23.949 INFO:teuthology.orchestra.run.smithi154.stdout:(5/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 4.6 MB/s | 5.5 MB 00:01 2024-09-10T21:47:23.972 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-09-10T21:47:23.977 INFO:teuthology.orchestra.run.smithi171.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 1.4 MB/s | 127 kB 00:00 2024-09-10T21:47:23.992 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-09-10T21:47:23.992 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-09-10T21:47:23.992 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T21:47:23.995 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T21:47:23.998 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T21:47:23.999 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T21:47:24.000 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T21:47:24.001 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T21:47:24.002 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T21:47:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T21:47:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T21:47:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T21:47:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T21:47:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T21:47:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T21:47:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T21:47:24.003 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T21:47:24.004 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-09-10T21:47:24.005 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-09-10T21:47:24.006 INFO:teuthology.orchestra.run.smithi090.stdout:Install 149 Packages 2024-09-10T21:47:24.006 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 8 Packages 2024-09-10T21:47:24.006 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:47:24.006 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 240 M 2024-09-10T21:47:24.007 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-09-10T21:47:24.017 INFO:teuthology.orchestra.run.smithi171.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 5.4 MB/s | 507 kB 00:00 2024-09-10T21:47:24.057 INFO:teuthology.orchestra.run.smithi154.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-09-10T21:47:24.093 INFO:teuthology.orchestra.run.smithi171.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 602 kB/s | 45 kB 00:00 2024-09-10T21:47:24.185 INFO:teuthology.orchestra.run.smithi171.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.5 MB/s | 143 kB 00:00 2024-09-10T21:47:24.327 INFO:teuthology.orchestra.run.smithi171.stdout:(21/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 16 MB/s | 5.5 MB 00:00 2024-09-10T21:47:24.360 INFO:teuthology.orchestra.run.smithi171.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 921 kB/s | 161 kB 00:00 2024-09-10T21:47:24.419 INFO:teuthology.orchestra.run.smithi171.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 3.4 MB/s | 321 kB 00:00 2024-09-10T21:47:24.444 INFO:teuthology.orchestra.run.smithi171.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 3.6 MB/s | 306 kB 00:00 2024-09-10T21:47:24.495 INFO:teuthology.orchestra.run.smithi171.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.3 MB/s | 99 kB 00:00 2024-09-10T21:47:24.528 INFO:teuthology.orchestra.run.smithi171.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.0 MB/s | 85 kB 00:00 2024-09-10T21:47:24.574 INFO:teuthology.orchestra.run.smithi154.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 7.7 MB/s | 4.8 MB 00:00 2024-09-10T21:47:24.612 INFO:teuthology.orchestra.run.smithi171.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.0 MB/s | 171 kB 00:00 2024-09-10T21:47:24.730 INFO:teuthology.orchestra.run.smithi171.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 293 kB/s | 34 kB 00:00 2024-09-10T21:47:24.809 INFO:teuthology.orchestra.run.smithi171.stdout:(29/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 10 MB/s | 3.2 MB 00:00 2024-09-10T21:47:24.839 INFO:teuthology.orchestra.run.smithi171.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 1.4 MB/s | 154 kB 00:00 2024-09-10T21:47:24.940 INFO:teuthology.orchestra.run.smithi090.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-09-10T21:47:25.425 INFO:teuthology.orchestra.run.smithi154.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 8.1 MB/s | 22 MB 00:02 2024-09-10T21:47:25.516 INFO:teuthology.orchestra.run.smithi154.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 276 kB/s | 25 kB 00:00 2024-09-10T21:47:25.932 INFO:teuthology.orchestra.run.smithi090.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 908 kB/s | 900 kB 00:00 2024-09-10T21:47:25.947 INFO:teuthology.orchestra.run.smithi171.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 6.7 MB/s | 7.4 MB 00:01 2024-09-10T21:47:26.041 INFO:teuthology.orchestra.run.smithi090.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.3 MB/s | 145 kB 00:00 2024-09-10T21:47:26.123 INFO:teuthology.orchestra.run.smithi154.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 7.1 MB/s | 11 MB 00:01 2024-09-10T21:47:26.184 INFO:teuthology.orchestra.run.smithi154.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 3.7 MB/s | 225 kB 00:00 2024-09-10T21:47:26.214 INFO:teuthology.orchestra.run.smithi171.stdout:(32/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 10 MB/s | 14 MB 00:01 2024-09-10T21:47:26.242 INFO:teuthology.orchestra.run.smithi154.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 464 kB/s | 27 kB 00:00 2024-09-10T21:47:26.248 INFO:teuthology.orchestra.run.smithi171.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4854.g8f 968 kB/s | 291 kB 00:00 2024-09-10T21:47:26.298 INFO:teuthology.orchestra.run.smithi171.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 591 kB/s | 49 kB 00:00 2024-09-10T21:47:26.309 INFO:teuthology.orchestra.run.smithi154.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 477 kB/s | 32 kB 00:00 2024-09-10T21:47:26.340 INFO:teuthology.orchestra.run.smithi171.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 179 kB/s | 16 kB 00:00 2024-09-10T21:47:26.366 INFO:teuthology.orchestra.run.smithi171.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 4.2 MB/s | 285 kB 00:00 2024-09-10T21:47:26.443 INFO:teuthology.orchestra.run.smithi154.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 5.5 MB/s | 755 kB 00:00 2024-09-10T21:47:26.450 INFO:teuthology.orchestra.run.smithi171.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 217 kB/s | 18 kB 00:00 2024-09-10T21:47:26.483 INFO:teuthology.orchestra.run.smithi171.stdout:(38/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 5.4 MB/s | 783 kB 00:00 2024-09-10T21:47:26.526 INFO:teuthology.orchestra.run.smithi154.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 1.9 MB/s | 162 kB 00:00 2024-09-10T21:47:26.827 INFO:teuthology.orchestra.run.smithi154.stdout:(16/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 6.2 MB/s | 17 MB 00:02 2024-09-10T21:47:26.852 INFO:teuthology.orchestra.run.smithi154.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 389 kB/s | 127 kB 00:00 2024-09-10T21:47:26.857 INFO:teuthology.orchestra.run.smithi090.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 2.4 MB 00:00 2024-09-10T21:47:26.944 INFO:teuthology.orchestra.run.smithi154.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 4.3 MB/s | 507 kB 00:00 2024-09-10T21:47:26.968 INFO:teuthology.orchestra.run.smithi171.stdout:(39/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 14 MB/s | 50 MB 00:03 2024-09-10T21:47:27.019 INFO:teuthology.orchestra.run.smithi154.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 603 kB/s | 45 kB 00:00 2024-09-10T21:47:27.095 INFO:teuthology.orchestra.run.smithi154.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.9 MB/s | 143 kB 00:00 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi154.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.9 MB/s | 161 kB 00:00 2024-09-10T21:47:27.262 INFO:teuthology.orchestra.run.smithi154.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 3.8 MB/s | 321 kB 00:00 2024-09-10T21:47:27.354 INFO:teuthology.orchestra.run.smithi154.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 3.3 MB/s | 306 kB 00:00 2024-09-10T21:47:27.504 INFO:teuthology.orchestra.run.smithi154.stdout:(24/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 8.5 MB/s | 5.5 MB 00:00 2024-09-10T21:47:27.526 INFO:teuthology.orchestra.run.smithi171.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 74 kB/s | 80 kB 00:01 2024-09-10T21:47:27.529 INFO:teuthology.orchestra.run.smithi154.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 567 kB/s | 99 kB 00:00 2024-09-10T21:47:27.563 INFO:teuthology.orchestra.run.smithi154.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.4 MB/s | 85 kB 00:00 2024-09-10T21:47:27.630 INFO:teuthology.orchestra.run.smithi154.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.5 MB/s | 171 kB 00:00 2024-09-10T21:47:27.680 INFO:teuthology.orchestra.run.smithi154.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 686 kB/s | 34 kB 00:00 2024-09-10T21:47:27.699 INFO:teuthology.orchestra.run.smithi090.stdout:(5/157): ceph-common-19.3.0-4854.g8f219c47.el9. 7.0 MB/s | 22 MB 00:03 2024-09-10T21:47:27.747 INFO:teuthology.orchestra.run.smithi154.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.3 MB/s | 154 kB 00:00 2024-09-10T21:47:27.768 INFO:teuthology.orchestra.run.smithi171.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 239 kB/s | 191 kB 00:00 2024-09-10T21:47:27.793 INFO:teuthology.orchestra.run.smithi090.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 1.6 MB/s | 1.5 MB 00:00 2024-09-10T21:47:27.989 INFO:teuthology.orchestra.run.smithi154.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 6.9 MB/s | 3.2 MB 00:00 2024-09-10T21:47:28.075 INFO:teuthology.orchestra.run.smithi090.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 13 MB/s | 4.8 MB 00:00 2024-09-10T21:47:28.485 INFO:teuthology.orchestra.run.smithi171.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 400 kB/s | 801 kB 00:02 2024-09-10T21:47:28.510 INFO:teuthology.orchestra.run.smithi171.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 257 kB/s | 253 kB 00:00 2024-09-10T21:47:28.894 INFO:teuthology.orchestra.run.smithi090.stdout:(8/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 13 MB/s | 11 MB 00:00 2024-09-10T21:47:28.961 INFO:teuthology.orchestra.run.smithi090.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 380 kB/s | 25 kB 00:00 2024-09-10T21:47:29.161 INFO:teuthology.orchestra.run.smithi171.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-10T21:47:29.253 INFO:teuthology.orchestra.run.smithi171.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 182 kB/s | 135 kB 00:00 2024-09-10T21:47:29.398 INFO:teuthology.orchestra.run.smithi154.stdout:(31/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 8.6 MB/s | 14 MB 00:01 2024-09-10T21:47:29.498 INFO:teuthology.orchestra.run.smithi154.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 2.8 MB/s | 291 kB 00:00 2024-09-10T21:47:29.608 INFO:teuthology.orchestra.run.smithi154.stdout:(33/157): ceph-mgr-diskprediction-local-19.3.0- 4.6 MB/s | 7.4 MB 00:01 2024-09-10T21:47:29.632 INFO:teuthology.orchestra.run.smithi154.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 369 kB/s | 49 kB 00:00 2024-09-10T21:47:29.690 INFO:teuthology.orchestra.run.smithi154.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 197 kB/s | 16 kB 00:00 2024-09-10T21:47:29.724 INFO:teuthology.orchestra.run.smithi154.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 3.0 MB/s | 285 kB 00:00 2024-09-10T21:47:29.816 INFO:teuthology.orchestra.run.smithi154.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 199 kB/s | 18 kB 00:00 2024-09-10T21:47:29.924 INFO:teuthology.orchestra.run.smithi154.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 738 kB/s | 80 kB 00:00 2024-09-10T21:47:29.928 INFO:teuthology.orchestra.run.smithi171.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 163 kB/s | 125 kB 00:00 2024-09-10T21:47:29.958 INFO:teuthology.orchestra.run.smithi154.stdout:(39/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 2.9 MB/s | 783 kB 00:00 2024-09-10T21:47:29.970 INFO:teuthology.orchestra.run.smithi171.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 573 kB/s | 1.2 MB 00:02 2024-09-10T21:47:30.075 INFO:teuthology.orchestra.run.smithi154.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-10T21:47:30.087 INFO:teuthology.orchestra.run.smithi171.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 689 kB 00:00 2024-09-10T21:47:30.101 INFO:teuthology.orchestra.run.smithi154.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.4 MB/s | 801 kB 00:00 2024-09-10T21:47:30.129 INFO:teuthology.orchestra.run.smithi171.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 5.8 MB/s | 943 kB 00:00 2024-09-10T21:47:30.163 INFO:teuthology.orchestra.run.smithi171.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-10T21:47:30.180 INFO:teuthology.orchestra.run.smithi171.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 436 kB/s | 22 kB 00:00 2024-09-10T21:47:30.201 INFO:teuthology.orchestra.run.smithi154.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-09-10T21:47:30.214 INFO:teuthology.orchestra.run.smithi171.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-10T21:47:30.239 INFO:teuthology.orchestra.run.smithi171.stdout:(53/157): python3-urllib3-1.26.5-6.el9.noarch.r 218 kB/s | 215 kB 00:00 2024-09-10T21:47:30.256 INFO:teuthology.orchestra.run.smithi171.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 438 kB/s | 33 kB 00:00 2024-09-10T21:47:30.259 INFO:teuthology.orchestra.run.smithi154.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T21:47:30.335 INFO:teuthology.orchestra.run.smithi154.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-10T21:47:30.356 INFO:teuthology.orchestra.run.smithi171.stdout:(55/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-09-10T21:47:30.377 INFO:teuthology.orchestra.run.smithi154.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 4.4 MB/s | 1.2 MB 00:00 2024-09-10T21:47:30.381 INFO:teuthology.orchestra.run.smithi171.stdout:(56/157): flexiblas-openblas-openmp-3.0.4-8.el9 150 kB/s | 18 kB 00:00 2024-09-10T21:47:30.402 INFO:teuthology.orchestra.run.smithi154.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-10T21:47:30.407 INFO:teuthology.orchestra.run.smithi171.stdout:(57/157): libaio-devel-0.3.111-13.el9.x86_64.rp 76 kB/s | 11 kB 00:00 2024-09-10T21:47:30.440 INFO:teuthology.orchestra.run.smithi171.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.4 MB/s | 45 kB 00:00 2024-09-10T21:47:30.461 INFO:teuthology.orchestra.run.smithi154.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-10T21:47:30.474 INFO:teuthology.orchestra.run.smithi171.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-10T21:47:30.500 INFO:teuthology.orchestra.run.smithi171.stdout:(60/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.8 MB/s | 171 kB 00:00 2024-09-10T21:47:30.533 INFO:teuthology.orchestra.run.smithi171.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-10T21:47:30.567 INFO:teuthology.orchestra.run.smithi171.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 6.4 MB/s | 581 kB 00:00 2024-09-10T21:47:30.592 INFO:teuthology.orchestra.run.smithi171.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 434 kB/s | 24 kB 00:00 2024-09-10T21:47:30.611 INFO:teuthology.orchestra.run.smithi154.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-10T21:47:30.617 INFO:teuthology.orchestra.run.smithi171.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 582 kB/s | 47 kB 00:00 2024-09-10T21:47:30.643 INFO:teuthology.orchestra.run.smithi171.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-10T21:47:30.668 INFO:teuthology.orchestra.run.smithi171.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 786 kB/s | 38 kB 00:00 2024-09-10T21:47:30.669 INFO:teuthology.orchestra.run.smithi154.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-10T21:47:30.711 INFO:teuthology.orchestra.run.smithi154.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-09-10T21:47:30.735 INFO:teuthology.orchestra.run.smithi171.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-10T21:47:30.745 INFO:teuthology.orchestra.run.smithi154.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-10T21:47:30.771 INFO:teuthology.orchestra.run.smithi154.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-10T21:47:30.796 INFO:teuthology.orchestra.run.smithi154.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-10T21:47:30.827 INFO:teuthology.orchestra.run.smithi171.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-10T21:47:30.838 INFO:teuthology.orchestra.run.smithi154.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-10T21:47:30.888 INFO:teuthology.orchestra.run.smithi154.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-10T21:47:30.903 INFO:teuthology.orchestra.run.smithi171.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-10T21:47:30.945 INFO:teuthology.orchestra.run.smithi171.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 7.8 MB/s | 1.6 MB 00:00 2024-09-10T21:47:30.955 INFO:teuthology.orchestra.run.smithi154.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-10T21:47:30.969 INFO:teuthology.orchestra.run.smithi090.stdout:(10/157): ceph-base-19.3.0-4854.g8f219c47.el9.x 884 kB/s | 5.5 MB 00:06 2024-09-10T21:47:30.970 INFO:teuthology.orchestra.run.smithi171.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 6.5 MB/s | 910 kB 00:00 2024-09-10T21:47:30.995 INFO:teuthology.orchestra.run.smithi171.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-10T21:47:30.997 INFO:teuthology.orchestra.run.smithi154.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-10T21:47:31.031 INFO:teuthology.orchestra.run.smithi154.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-10T21:47:31.079 INFO:teuthology.orchestra.run.smithi171.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-10T21:47:31.079 INFO:teuthology.orchestra.run.smithi154.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-09-10T21:47:31.106 INFO:teuthology.orchestra.run.smithi154.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-09-10T21:47:31.140 INFO:teuthology.orchestra.run.smithi154.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 7.6 MB/s | 581 kB 00:00 2024-09-10T21:47:31.147 INFO:teuthology.orchestra.run.smithi171.stdout:(74/157): postgresql-server-13.16-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-09-10T21:47:31.165 INFO:teuthology.orchestra.run.smithi154.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-10T21:47:31.171 INFO:teuthology.orchestra.run.smithi171.stdout:(75/157): python3-attrs-20.3.0-7.el9.noarch.rpm 503 kB/s | 87 kB 00:00 2024-09-10T21:47:31.190 INFO:teuthology.orchestra.run.smithi154.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-09-10T21:47:31.211 INFO:teuthology.orchestra.run.smithi090.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 932 kB/s | 225 kB 00:00 2024-09-10T21:47:31.215 INFO:teuthology.orchestra.run.smithi154.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T21:47:31.241 INFO:teuthology.orchestra.run.smithi154.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T21:47:31.266 INFO:teuthology.orchestra.run.smithi154.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-10T21:47:31.272 INFO:teuthology.orchestra.run.smithi171.stdout:(76/157): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 6.0 MB 00:00 2024-09-10T21:47:31.295 INFO:teuthology.orchestra.run.smithi090.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 326 kB/s | 27 kB 00:00 2024-09-10T21:47:31.297 INFO:teuthology.orchestra.run.smithi171.stdout:(77/157): python3-colorama-0.4.6-3.el9.noarch.r 507 kB/s | 61 kB 00:00 2024-09-10T21:47:31.322 INFO:teuthology.orchestra.run.smithi171.stdout:(78/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-10T21:47:31.348 INFO:teuthology.orchestra.run.smithi171.stdout:(79/157): python3-jinja2-2.11.3-6.el9.noarch.rp 3.3 MB/s | 248 kB 00:00 2024-09-10T21:47:31.373 INFO:teuthology.orchestra.run.smithi171.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 517 kB/s | 26 kB 00:00 2024-09-10T21:47:31.378 INFO:teuthology.orchestra.run.smithi090.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 382 kB/s | 32 kB 00:00 2024-09-10T21:47:31.398 INFO:teuthology.orchestra.run.smithi171.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T21:47:31.409 INFO:teuthology.orchestra.run.smithi154.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-09-10T21:47:31.424 INFO:teuthology.orchestra.run.smithi171.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 698 kB/s | 35 kB 00:00 2024-09-10T21:47:31.501 INFO:teuthology.orchestra.run.smithi154.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 19 MB/s | 4.9 MB 00:00 2024-09-10T21:47:31.516 INFO:teuthology.orchestra.run.smithi171.stdout:(83/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 43 MB/s | 5.1 MB 00:00 2024-09-10T21:47:31.541 INFO:teuthology.orchestra.run.smithi171.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.2 MB/s | 250 kB 00:00 2024-09-10T21:47:31.551 INFO:teuthology.orchestra.run.smithi154.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-10T21:47:31.592 INFO:teuthology.orchestra.run.smithi171.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T21:47:31.593 INFO:teuthology.orchestra.run.smithi154.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-10T21:47:31.617 INFO:teuthology.orchestra.run.smithi171.stdout:(86/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-10T21:47:31.618 INFO:teuthology.orchestra.run.smithi154.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-10T21:47:31.642 INFO:teuthology.orchestra.run.smithi171.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 854 kB/s | 42 kB 00:00 2024-09-10T21:47:31.668 INFO:teuthology.orchestra.run.smithi171.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-10T21:47:31.685 INFO:teuthology.orchestra.run.smithi154.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-10T21:47:31.693 INFO:teuthology.orchestra.run.smithi171.stdout:(89/157): python3-jsonpointer-2.0-4.el9.noarch. 52 kB/s | 19 kB 00:00 2024-09-10T21:47:31.718 INFO:teuthology.orchestra.run.smithi171.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-10T21:47:31.744 INFO:teuthology.orchestra.run.smithi171.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-10T21:47:31.769 INFO:teuthology.orchestra.run.smithi171.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T21:47:31.794 INFO:teuthology.orchestra.run.smithi171.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 722 kB/s | 54 kB 00:00 2024-09-10T21:47:31.837 INFO:teuthology.orchestra.run.smithi171.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 42 kB 00:00 2024-09-10T21:47:31.862 INFO:teuthology.orchestra.run.smithi171.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 708 kB/s | 44 kB 00:00 2024-09-10T21:47:31.892 INFO:teuthology.orchestra.run.smithi090.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 1.4 MB/s | 755 kB 00:00 2024-09-10T21:47:31.904 INFO:teuthology.orchestra.run.smithi171.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-09-10T21:47:31.937 INFO:teuthology.orchestra.run.smithi171.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 503 kB/s | 36 kB 00:00 2024-09-10T21:47:31.994 INFO:teuthology.orchestra.run.smithi171.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 549 kB/s | 47 kB 00:00 2024-09-10T21:47:32.046 INFO:teuthology.orchestra.run.smithi090.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 1.0 MB/s | 162 kB 00:00 2024-09-10T21:47:32.169 INFO:teuthology.orchestra.run.smithi154.stdout:(73/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 7.6 MB/s | 50 MB 00:06 2024-09-10T21:47:32.180 INFO:teuthology.orchestra.run.smithi171.stdout:(99/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-10T21:47:32.196 INFO:teuthology.orchestra.run.smithi090.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 845 kB/s | 127 kB 00:00 2024-09-10T21:47:32.205 INFO:teuthology.orchestra.run.smithi171.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 1.1 MB/s | 304 kB 00:00 2024-09-10T21:47:32.216 INFO:teuthology.orchestra.run.smithi154.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 165 kB/s | 87 kB 00:00 2024-09-10T21:47:32.230 INFO:teuthology.orchestra.run.smithi171.stdout:(101/157): uuid-1.6.2-55.el9.x86_64.rpm 249 kB/s | 58 kB 00:00 2024-09-10T21:47:32.256 INFO:teuthology.orchestra.run.smithi171.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T21:47:32.281 INFO:teuthology.orchestra.run.smithi171.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 848 kB/s | 64 kB 00:00 2024-09-10T21:47:32.306 INFO:teuthology.orchestra.run.smithi171.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 597 kB/s | 45 kB 00:00 2024-09-10T21:47:32.311 INFO:teuthology.orchestra.run.smithi154.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 8.0 MB/s | 5.8 MB 00:00 2024-09-10T21:47:32.336 INFO:teuthology.orchestra.run.smithi154.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 511 kB/s | 61 kB 00:00 2024-09-10T21:47:32.361 INFO:teuthology.orchestra.run.smithi154.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-10T21:47:32.373 INFO:teuthology.orchestra.run.smithi171.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-10T21:47:32.387 INFO:teuthology.orchestra.run.smithi154.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-10T21:47:32.412 INFO:teuthology.orchestra.run.smithi154.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-10T21:47:32.432 INFO:teuthology.orchestra.run.smithi171.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-09-10T21:47:32.437 INFO:teuthology.orchestra.run.smithi154.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-09-10T21:47:32.463 INFO:teuthology.orchestra.run.smithi154.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-10T21:47:32.474 INFO:teuthology.orchestra.run.smithi171.stdout:(107/157): lua-devel-5.4.4-4.el9.x86_64.rpm 116 kB/s | 22 kB 00:00 2024-09-10T21:47:32.488 INFO:teuthology.orchestra.run.smithi154.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-10T21:47:32.516 INFO:teuthology.orchestra.run.smithi171.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 231 kB/s | 19 kB 00:00 2024-09-10T21:47:32.538 INFO:teuthology.orchestra.run.smithi154.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-10T21:47:32.567 INFO:teuthology.orchestra.run.smithi171.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 644 kB/s | 26 kB 00:00 2024-09-10T21:47:32.609 INFO:teuthology.orchestra.run.smithi171.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2024-09-10T21:47:32.634 INFO:teuthology.orchestra.run.smithi171.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-10T21:47:32.664 INFO:teuthology.orchestra.run.smithi154.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-09-10T21:47:32.668 INFO:teuthology.orchestra.run.smithi171.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-10T21:47:32.689 INFO:teuthology.orchestra.run.smithi154.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-10T21:47:32.710 INFO:teuthology.orchestra.run.smithi171.stdout:(113/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.0 MB/s | 863 kB 00:00 2024-09-10T21:47:32.714 INFO:teuthology.orchestra.run.smithi154.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T21:47:32.740 INFO:teuthology.orchestra.run.smithi154.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-10T21:47:32.773 INFO:teuthology.orchestra.run.smithi154.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-10T21:47:32.777 INFO:teuthology.orchestra.run.smithi171.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-10T21:47:32.798 INFO:teuthology.orchestra.run.smithi154.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-10T21:47:32.823 INFO:teuthology.orchestra.run.smithi154.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T21:47:32.849 INFO:teuthology.orchestra.run.smithi154.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T21:47:32.850 INFO:teuthology.orchestra.run.smithi171.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 594 kB/s | 43 kB 00:00 2024-09-10T21:47:32.874 INFO:teuthology.orchestra.run.smithi154.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T21:47:32.911 INFO:teuthology.orchestra.run.smithi171.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 533 kB/s | 32 kB 00:00 2024-09-10T21:47:32.961 INFO:teuthology.orchestra.run.smithi171.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 283 kB/s | 14 kB 00:00 2024-09-10T21:47:32.966 INFO:teuthology.orchestra.run.smithi154.stdout:(93/157): python3-babel-2.9.1-2.el9.noarch.rpm 7.5 MB/s | 6.0 MB 00:00 2024-09-10T21:47:32.991 INFO:teuthology.orchestra.run.smithi154.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-09-10T21:47:33.004 INFO:teuthology.orchestra.run.smithi171.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-09-10T21:47:33.005 INFO:teuthology.orchestra.run.smithi090.stdout:(17/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 12 MB/s | 50 MB 00:04 2024-09-10T21:47:33.029 INFO:teuthology.orchestra.run.smithi171.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-10T21:47:33.039 INFO:teuthology.orchestra.run.smithi090.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 603 kB/s | 507 kB 00:00 2024-09-10T21:47:33.041 INFO:teuthology.orchestra.run.smithi154.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 590 kB/s | 44 kB 00:00 2024-09-10T21:47:33.067 INFO:teuthology.orchestra.run.smithi154.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-10T21:47:33.071 INFO:teuthology.orchestra.run.smithi171.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.1 MB/s | 303 kB 00:00 2024-09-10T21:47:33.092 INFO:teuthology.orchestra.run.smithi154.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 712 kB/s | 36 kB 00:00 2024-09-10T21:47:33.096 INFO:teuthology.orchestra.run.smithi171.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-10T21:47:33.118 INFO:teuthology.orchestra.run.smithi154.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-10T21:47:33.131 INFO:teuthology.orchestra.run.smithi090.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 494 kB/s | 45 kB 00:00 2024-09-10T21:47:33.143 INFO:teuthology.orchestra.run.smithi154.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-10T21:47:33.168 INFO:teuthology.orchestra.run.smithi154.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T21:47:33.180 INFO:teuthology.orchestra.run.smithi171.stdout:(122/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-09-10T21:47:33.193 INFO:teuthology.orchestra.run.smithi154.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T21:47:33.205 INFO:teuthology.orchestra.run.smithi171.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 1.6 MB/s | 220 kB 00:00 2024-09-10T21:47:33.256 INFO:teuthology.orchestra.run.smithi171.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-10T21:47:33.298 INFO:teuthology.orchestra.run.smithi171.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-09-10T21:47:33.336 INFO:teuthology.orchestra.run.smithi154.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 381 kB/s | 64 kB 00:00 2024-09-10T21:47:33.340 INFO:teuthology.orchestra.run.smithi171.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 210 kB/s | 18 kB 00:00 2024-09-10T21:47:33.361 INFO:teuthology.orchestra.run.smithi154.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 268 kB/s | 45 kB 00:00 2024-09-10T21:47:33.382 INFO:teuthology.orchestra.run.smithi171.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-09-10T21:47:33.424 INFO:teuthology.orchestra.run.smithi171.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-09-10T21:47:33.466 INFO:teuthology.orchestra.run.smithi171.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 234 kB/s | 20 kB 00:00 2024-09-10T21:47:33.478 INFO:teuthology.orchestra.run.smithi154.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-09-10T21:47:33.554 INFO:teuthology.orchestra.run.smithi154.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-09-10T21:47:33.592 INFO:teuthology.orchestra.run.smithi171.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-09-10T21:47:33.595 INFO:teuthology.orchestra.run.smithi154.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-09-10T21:47:33.606 INFO:teuthology.orchestra.run.smithi090.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 301 kB/s | 143 kB 00:00 2024-09-10T21:47:33.617 INFO:teuthology.orchestra.run.smithi171.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T21:47:33.620 INFO:teuthology.orchestra.run.smithi154.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-10T21:47:33.685 INFO:teuthology.orchestra.run.smithi171.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 866 kB/s | 58 kB 00:00 2024-09-10T21:47:33.735 INFO:teuthology.orchestra.run.smithi171.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-09-10T21:47:33.760 INFO:teuthology.orchestra.run.smithi171.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-10T21:47:33.794 INFO:teuthology.orchestra.run.smithi090.stdout:(21/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 7.0 MB/s | 5.5 MB 00:00 2024-09-10T21:47:33.802 INFO:teuthology.orchestra.run.smithi171.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 244 kB/s | 16 kB 00:00 2024-09-10T21:47:33.823 INFO:teuthology.orchestra.run.smithi090.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 745 kB/s | 161 kB 00:00 2024-09-10T21:47:33.828 INFO:teuthology.orchestra.run.smithi171.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-10T21:47:33.828 INFO:teuthology.orchestra.run.smithi154.stdout:(108/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-10T21:47:33.870 INFO:teuthology.orchestra.run.smithi171.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 419 kB/s | 28 kB 00:00 2024-09-10T21:47:33.871 INFO:teuthology.orchestra.run.smithi154.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-09-10T21:47:33.890 INFO:teuthology.orchestra.run.smithi090.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 3.3 MB/s | 321 kB 00:00 2024-09-10T21:47:33.912 INFO:teuthology.orchestra.run.smithi171.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-09-10T21:47:33.930 INFO:teuthology.orchestra.run.smithi154.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 258 kB/s | 26 kB 00:00 2024-09-10T21:47:33.937 INFO:teuthology.orchestra.run.smithi171.stdout:(139/157): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-09-10T21:47:33.955 INFO:teuthology.orchestra.run.smithi154.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 658 kB/s | 49 kB 00:00 2024-09-10T21:47:33.980 INFO:teuthology.orchestra.run.smithi171.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 878 kB/s | 59 kB 00:00 2024-09-10T21:47:33.980 INFO:teuthology.orchestra.run.smithi154.stdout:(112/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T21:47:33.991 INFO:teuthology.orchestra.run.smithi090.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 994 kB/s | 99 kB 00:00 2024-09-10T21:47:34.005 INFO:teuthology.orchestra.run.smithi154.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-10T21:47:34.056 INFO:teuthology.orchestra.run.smithi154.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-10T21:47:34.066 INFO:teuthology.orchestra.run.smithi090.stdout:(25/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.1 MB/s | 85 kB 00:00 2024-09-10T21:47:34.070 INFO:teuthology.orchestra.run.smithi171.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 270 kB/s | 36 kB 00:00 2024-09-10T21:47:34.081 INFO:teuthology.orchestra.run.smithi154.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T21:47:34.116 INFO:teuthology.orchestra.run.smithi154.stdout:(116/157): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-09-10T21:47:34.122 INFO:teuthology.orchestra.run.smithi171.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 525 kB/s | 74 kB 00:00 2024-09-10T21:47:34.125 INFO:teuthology.orchestra.run.smithi090.stdout:(26/157): python3-rbd-19.3.0-4854.g8f219c47.el9 1.0 MB/s | 306 kB 00:00 2024-09-10T21:47:34.140 INFO:teuthology.orchestra.run.smithi154.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 547 kB/s | 32 kB 00:00 2024-09-10T21:47:34.166 INFO:teuthology.orchestra.run.smithi154.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 283 kB/s | 14 kB 00:00 2024-09-10T21:47:34.170 INFO:teuthology.orchestra.run.smithi171.stdout:(143/157): python3-grpcio-1.46.7-10.el9.x86_64. 1.9 MB/s | 2.0 MB 00:01 2024-09-10T21:47:34.191 INFO:teuthology.orchestra.run.smithi154.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T21:47:34.196 INFO:teuthology.orchestra.run.smithi171.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-10T21:47:34.230 INFO:teuthology.orchestra.run.smithi171.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-09-10T21:47:34.241 INFO:teuthology.orchestra.run.smithi154.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-09-10T21:47:34.255 INFO:teuthology.orchestra.run.smithi171.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-10T21:47:34.281 INFO:teuthology.orchestra.run.smithi171.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T21:47:34.283 INFO:teuthology.orchestra.run.smithi154.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-10T21:47:34.283 INFO:teuthology.orchestra.run.smithi090.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 1.1 MB/s | 171 kB 00:00 2024-09-10T21:47:34.308 INFO:teuthology.orchestra.run.smithi154.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-10T21:47:34.316 INFO:teuthology.orchestra.run.smithi171.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-10T21:47:34.342 INFO:teuthology.orchestra.run.smithi154.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-10T21:47:34.368 INFO:teuthology.orchestra.run.smithi154.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-09-10T21:47:34.381 INFO:teuthology.orchestra.run.smithi171.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-10T21:47:34.392 INFO:teuthology.orchestra.run.smithi090.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 316 kB/s | 34 kB 00:00 2024-09-10T21:47:34.451 INFO:teuthology.orchestra.run.smithi154.stdout:(125/157): libarrow-9.0.0-11.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-09-10T21:47:34.476 INFO:teuthology.orchestra.run.smithi154.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-09-10T21:47:34.502 INFO:teuthology.orchestra.run.smithi154.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 459 kB/s | 23 kB 00:00 2024-09-10T21:47:34.527 INFO:teuthology.orchestra.run.smithi154.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 385 kB/s | 19 kB 00:00 2024-09-10T21:47:34.552 INFO:teuthology.orchestra.run.smithi154.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-10T21:47:34.577 INFO:teuthology.orchestra.run.smithi154.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T21:47:34.643 INFO:teuthology.orchestra.run.smithi154.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-09-10T21:47:34.651 INFO:teuthology.orchestra.run.smithi090.stdout:(29/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 5.4 MB/s | 3.2 MB 00:00 2024-09-10T21:47:34.678 INFO:teuthology.orchestra.run.smithi154.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.6 MB/s | 58 kB 00:00 2024-09-10T21:47:34.686 INFO:teuthology.orchestra.run.smithi090.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 526 kB/s | 154 kB 00:00 2024-09-10T21:47:34.720 INFO:teuthology.orchestra.run.smithi154.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-10T21:47:34.745 INFO:teuthology.orchestra.run.smithi154.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-09-10T21:47:34.771 INFO:teuthology.orchestra.run.smithi154.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-09-10T21:47:34.821 INFO:teuthology.orchestra.run.smithi154.stdout:(136/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-09-10T21:47:34.846 INFO:teuthology.orchestra.run.smithi154.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 896 kB/s | 90 kB 00:00 2024-09-10T21:47:34.863 INFO:teuthology.orchestra.run.smithi154.stdout:(138/157): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-09-10T21:47:34.880 INFO:teuthology.orchestra.run.smithi154.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 1.9 MB/s | 31 kB 00:00 2024-09-10T21:47:34.897 INFO:teuthology.orchestra.run.smithi154.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-10T21:47:34.914 INFO:teuthology.orchestra.run.smithi154.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-10T21:47:34.939 INFO:teuthology.orchestra.run.smithi154.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-10T21:47:34.964 INFO:teuthology.orchestra.run.smithi154.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-09-10T21:47:34.990 INFO:teuthology.orchestra.run.smithi154.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T21:47:35.015 INFO:teuthology.orchestra.run.smithi171.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 4.6 MB/s | 3.2 MB 00:00 2024-09-10T21:47:35.038 INFO:teuthology.orchestra.run.smithi154.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-10T21:47:35.065 INFO:teuthology.orchestra.run.smithi154.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 266 kB/s | 20 kB 00:00 2024-09-10T21:47:35.091 INFO:teuthology.orchestra.run.smithi154.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.6 MB/s | 191 kB 00:00 2024-09-10T21:47:35.116 INFO:teuthology.orchestra.run.smithi154.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-10T21:47:35.183 INFO:teuthology.orchestra.run.smithi154.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-10T21:47:35.733 INFO:teuthology.orchestra.run.smithi154.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 4.2 MB/s | 2.3 MB 00:00 2024-09-10T21:47:36.145 INFO:teuthology.orchestra.run.smithi090.stdout:(31/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 2.1 MB/s | 17 MB 00:08 2024-09-10T21:47:36.256 INFO:teuthology.orchestra.run.smithi154.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 3.5 MB/s | 1.8 MB 00:00 2024-09-10T21:47:36.325 INFO:teuthology.orchestra.run.smithi154.stdout:(152/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 2.7 MB/s | 3.2 MB 00:01 2024-09-10T21:47:36.337 INFO:teuthology.orchestra.run.smithi090.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 1.5 MB/s | 291 kB 00:00 2024-09-10T21:47:36.342 INFO:teuthology.orchestra.run.smithi154.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 474 kB/s | 41 kB 00:00 2024-09-10T21:47:36.421 INFO:teuthology.orchestra.run.smithi090.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 593 kB/s | 49 kB 00:00 2024-09-10T21:47:36.443 INFO:teuthology.orchestra.run.smithi154.stdout:(154/157): librados2-19.3.0-4854.g8f219c47.el9. 2.6 MB/s | 3.5 MB 00:01 2024-09-10T21:47:36.501 INFO:teuthology.orchestra.run.smithi154.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 1.4 MB/s | 249 kB 00:00 2024-09-10T21:47:36.504 INFO:teuthology.orchestra.run.smithi090.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 197 kB/s | 16 kB 00:00 2024-09-10T21:47:36.663 INFO:teuthology.orchestra.run.smithi090.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 1.8 MB/s | 285 kB 00:00 2024-09-10T21:47:36.785 INFO:teuthology.orchestra.run.smithi154.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 3.1 MB/s | 1.4 MB 00:00 2024-09-10T21:47:37.005 INFO:teuthology.orchestra.run.smithi090.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 2.2 MB/s | 783 kB 00:00 2024-09-10T21:47:37.075 INFO:teuthology.orchestra.run.smithi154.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 2.4 MB 00:00 2024-09-10T21:47:37.081 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-09-10T21:47:37.081 INFO:teuthology.orchestra.run.smithi154.stdout:Total 16 MB/s | 240 MB 00:14 2024-09-10T21:47:37.097 INFO:teuthology.orchestra.run.smithi090.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 199 kB/s | 18 kB 00:00 2024-09-10T21:47:37.239 INFO:teuthology.orchestra.run.smithi090.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 565 kB/s | 80 kB 00:00 2024-09-10T21:47:37.390 INFO:teuthology.orchestra.run.smithi171.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 789 kB/s | 2.3 MB 00:03 2024-09-10T21:47:37.398 INFO:teuthology.orchestra.run.smithi090.stdout:(39/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.9 MB/s | 801 kB 00:00 2024-09-10T21:47:37.460 INFO:teuthology.orchestra.run.smithi171.stdout:(152/157): NetworkManager-libnm-1.49.90-1.el9.x 756 kB/s | 1.8 MB 00:02 2024-09-10T21:47:37.481 INFO:teuthology.orchestra.run.smithi090.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-10T21:47:37.549 INFO:teuthology.orchestra.run.smithi090.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-10T21:47:37.657 INFO:teuthology.orchestra.run.smithi090.stdout:(42/157): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-10T21:47:37.716 INFO:teuthology.orchestra.run.smithi090.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-10T21:47:37.775 INFO:teuthology.orchestra.run.smithi090.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-10T21:47:37.833 INFO:teuthology.orchestra.run.smithi090.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-10T21:47:37.901 INFO:teuthology.orchestra.run.smithi090.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-09-10T21:47:37.933 INFO:teuthology.orchestra.run.smithi171.stdout:(153/157): librados2-19.3.0-4854.g8f219c47.el9. 971 kB/s | 3.5 MB 00:03 2024-09-10T21:47:37.983 INFO:teuthology.orchestra.run.smithi171.stdout:(154/157): NetworkManager-team-1.49.90-1.el9.x8 69 kB/s | 41 kB 00:00 2024-09-10T21:47:38.209 INFO:teuthology.orchestra.run.smithi090.stdout:(47/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 4.0 MB/s | 14 MB 00:03 2024-09-10T21:47:38.243 INFO:teuthology.orchestra.run.smithi090.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-09-10T21:47:38.325 INFO:teuthology.orchestra.run.smithi171.stdout:(155/157): NetworkManager-tui-1.49.90-1.el9.x86 288 kB/s | 249 kB 00:00 2024-09-10T21:47:38.326 INFO:teuthology.orchestra.run.smithi090.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 107 kB 00:00 2024-09-10T21:47:38.385 INFO:teuthology.orchestra.run.smithi090.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 373 kB/s | 22 kB 00:00 2024-09-10T21:47:38.527 INFO:teuthology.orchestra.run.smithi090.stdout:(51/157): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-09-10T21:47:38.587 INFO:teuthology.orchestra.run.smithi090.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 549 kB/s | 33 kB 00:00 2024-09-10T21:47:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:(53/157): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-09-10T21:47:38.696 INFO:teuthology.orchestra.run.smithi090.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 312 kB/s | 18 kB 00:00 2024-09-10T21:47:38.746 INFO:teuthology.orchestra.run.smithi090.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-09-10T21:47:38.855 INFO:teuthology.orchestra.run.smithi090.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-10T21:47:38.964 INFO:teuthology.orchestra.run.smithi090.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-10T21:47:39.014 INFO:teuthology.orchestra.run.smithi090.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-09-10T21:47:39.153 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-09-10T21:47:39.273 INFO:teuthology.orchestra.run.smithi090.stdout:(59/157): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-09-10T21:47:39.337 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-09-10T21:47:39.338 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-09-10T21:47:39.367 INFO:teuthology.orchestra.run.smithi171.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 967 kB/s | 1.4 MB 00:01 2024-09-10T21:47:39.532 INFO:teuthology.orchestra.run.smithi090.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 2.2 MB/s | 581 kB 00:00 2024-09-10T21:47:39.590 INFO:teuthology.orchestra.run.smithi090.stdout:(61/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.0 MB/s | 3.0 MB 00:01 2024-09-10T21:47:39.616 INFO:teuthology.orchestra.run.smithi090.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 293 kB/s | 24 kB 00:00 2024-09-10T21:47:39.645 INFO:teuthology.orchestra.run.smithi090.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-10T21:47:39.725 INFO:teuthology.orchestra.run.smithi090.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T21:47:39.750 INFO:teuthology.orchestra.run.smithi090.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-10T21:47:39.775 INFO:teuthology.orchestra.run.smithi090.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-10T21:47:40.543 INFO:teuthology.orchestra.run.smithi090.stdout:(67/157): ceph-mgr-diskprediction-local-19.3.0- 1.3 MB/s | 7.4 MB 00:05 2024-09-10T21:47:40.984 INFO:teuthology.orchestra.run.smithi171.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 816 kB/s | 2.4 MB 00:02 2024-09-10T21:47:40.994 INFO:teuthology.orchestra.run.smithi171.stdout:-------------------------------------------------------------------------------- 2024-09-10T21:47:40.994 INFO:teuthology.orchestra.run.smithi171.stdout:Total 12 MB/s | 240 MB 00:19 2024-09-10T21:47:41.285 INFO:teuthology.orchestra.run.smithi090.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.9 MB/s | 4.4 MB 00:01 2024-09-10T21:47:41.360 INFO:teuthology.orchestra.run.smithi090.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.1 MB/s | 4.9 MB 00:01 2024-09-10T21:47:41.410 INFO:teuthology.orchestra.run.smithi090.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-09-10T21:47:41.435 INFO:teuthology.orchestra.run.smithi090.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-10T21:47:41.611 INFO:teuthology.orchestra.run.smithi090.stdout:(72/157): postgresql-contrib-13.16-1.el9.x86_64 2.7 MB/s | 910 kB 00:00 2024-09-10T21:47:41.736 INFO:teuthology.orchestra.run.smithi090.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2024-09-10T21:47:41.787 INFO:teuthology.orchestra.run.smithi090.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 499 kB/s | 87 kB 00:00 2024-09-10T21:47:41.808 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-09-10T21:47:41.809 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-09-10T21:47:41.847 INFO:teuthology.orchestra.run.smithi090.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-10T21:47:41.938 INFO:teuthology.orchestra.run.smithi090.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-10T21:47:42.022 INFO:teuthology.orchestra.run.smithi090.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-10T21:47:42.081 INFO:teuthology.orchestra.run.smithi090.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-09-10T21:47:42.131 INFO:teuthology.orchestra.run.smithi090.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 386 kB/s | 19 kB 00:00 2024-09-10T21:47:42.206 INFO:teuthology.orchestra.run.smithi090.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-10T21:47:42.257 INFO:teuthology.orchestra.run.smithi090.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 695 kB/s | 35 kB 00:00 2024-09-10T21:47:42.732 INFO:teuthology.orchestra.run.smithi090.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 4.4 MB/s | 5.8 MB 00:01 2024-09-10T21:47:42.833 INFO:teuthology.orchestra.run.smithi090.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 5.4 MB/s | 6.0 MB 00:01 2024-09-10T21:47:42.912 INFO:teuthology.orchestra.run.smithi090.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 250 kB 00:00 2024-09-10T21:47:42.942 INFO:teuthology.orchestra.run.smithi090.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-10T21:47:42.967 INFO:teuthology.orchestra.run.smithi090.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 77 kB 00:00 2024-09-10T21:47:42.992 INFO:teuthology.orchestra.run.smithi090.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-10T21:47:43.051 INFO:teuthology.orchestra.run.smithi090.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-10T21:47:43.083 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction check 2024-09-10T21:47:43.085 INFO:teuthology.orchestra.run.smithi090.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-10T21:47:43.143 INFO:teuthology.orchestra.run.smithi090.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-10T21:47:43.169 INFO:teuthology.orchestra.run.smithi090.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 611 kB/s | 51 kB 00:00 2024-09-10T21:47:43.202 INFO:teuthology.orchestra.run.smithi090.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-09-10T21:47:43.261 INFO:teuthology.orchestra.run.smithi090.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 715 kB/s | 42 kB 00:00 2024-09-10T21:47:43.267 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction check succeeded. 2024-09-10T21:47:43.267 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction test 2024-09-10T21:47:43.320 INFO:teuthology.orchestra.run.smithi090.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-09-10T21:47:43.378 INFO:teuthology.orchestra.run.smithi090.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-10T21:47:43.495 INFO:teuthology.orchestra.run.smithi090.stdout:(96/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-09-10T21:47:43.521 INFO:teuthology.orchestra.run.smithi090.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 252 kB/s | 36 kB 00:00 2024-09-10T21:47:43.554 INFO:teuthology.orchestra.run.smithi090.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-10T21:47:43.613 INFO:teuthology.orchestra.run.smithi090.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-10T21:47:43.640 INFO:teuthology.orchestra.run.smithi090.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 683 kB/s | 58 kB 00:00 2024-09-10T21:47:43.665 INFO:teuthology.orchestra.run.smithi090.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.8 MB/s | 96 kB 00:00 2024-09-10T21:47:43.732 INFO:teuthology.orchestra.run.smithi090.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 688 kB/s | 64 kB 00:00 2024-09-10T21:47:43.766 INFO:teuthology.orchestra.run.smithi090.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 447 kB/s | 45 kB 00:00 2024-09-10T21:47:43.842 INFO:teuthology.orchestra.run.smithi090.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-09-10T21:47:43.908 INFO:teuthology.orchestra.run.smithi090.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-10T21:47:43.950 INFO:teuthology.orchestra.run.smithi090.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-09-10T21:47:43.992 INFO:teuthology.orchestra.run.smithi090.stdout:(107/157): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-09-10T21:47:44.017 INFO:teuthology.orchestra.run.smithi090.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-10T21:47:44.043 INFO:teuthology.orchestra.run.smithi090.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T21:47:44.068 INFO:teuthology.orchestra.run.smithi090.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T21:47:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-10T21:47:44.152 INFO:teuthology.orchestra.run.smithi090.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-10T21:47:44.227 INFO:teuthology.orchestra.run.smithi090.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-10T21:47:44.286 INFO:teuthology.orchestra.run.smithi090.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-10T21:47:44.362 INFO:teuthology.orchestra.run.smithi090.stdout:(115/157): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-10T21:47:44.387 INFO:teuthology.orchestra.run.smithi090.stdout:(116/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 433 kB/s | 43 kB 00:00 2024-09-10T21:47:44.412 INFO:teuthology.orchestra.run.smithi090.stdout:(117/157): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-09-10T21:47:44.438 INFO:teuthology.orchestra.run.smithi090.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-10T21:47:44.463 INFO:teuthology.orchestra.run.smithi090.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T21:47:44.505 INFO:teuthology.orchestra.run.smithi090.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T21:47:44.564 INFO:teuthology.orchestra.run.smithi090.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-09-10T21:47:44.597 INFO:teuthology.orchestra.run.smithi090.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 2.3 MB/s | 220 kB 00:00 2024-09-10T21:47:44.632 INFO:teuthology.orchestra.run.smithi090.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.1 MB/s | 144 kB 00:00 2024-09-10T21:47:44.666 INFO:teuthology.orchestra.run.smithi090.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 321 kB/s | 11 kB 00:00 2024-09-10T21:47:44.691 INFO:teuthology.orchestra.run.smithi090.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 712 kB/s | 18 kB 00:00 2024-09-10T21:47:44.717 INFO:teuthology.orchestra.run.smithi090.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-10T21:47:44.742 INFO:teuthology.orchestra.run.smithi090.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-09-10T21:47:44.768 INFO:teuthology.orchestra.run.smithi090.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-10T21:47:44.843 INFO:teuthology.orchestra.run.smithi090.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 7.3 MB/s | 2.0 MB 00:00 2024-09-10T21:47:44.885 INFO:teuthology.orchestra.run.smithi090.stdout:(130/157): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-10T21:47:44.910 INFO:teuthology.orchestra.run.smithi090.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-09-10T21:47:44.969 INFO:teuthology.orchestra.run.smithi090.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-09-10T21:47:45.003 INFO:teuthology.orchestra.run.smithi090.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 627 kB/s | 58 kB 00:00 2024-09-10T21:47:45.036 INFO:teuthology.orchestra.run.smithi090.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 495 kB/s | 16 kB 00:00 2024-09-10T21:47:45.070 INFO:teuthology.orchestra.run.smithi090.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-09-10T21:47:45.095 INFO:teuthology.orchestra.run.smithi090.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-10T21:47:45.121 INFO:teuthology.orchestra.run.smithi090.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-10T21:47:45.146 INFO:teuthology.orchestra.run.smithi090.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-10T21:47:45.171 INFO:teuthology.orchestra.run.smithi090.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T21:47:45.205 INFO:teuthology.orchestra.run.smithi090.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-10T21:47:45.230 INFO:teuthology.orchestra.run.smithi090.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-09-10T21:47:45.256 INFO:teuthology.orchestra.run.smithi090.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-10T21:47:45.281 INFO:teuthology.orchestra.run.smithi090.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T21:47:45.306 INFO:teuthology.orchestra.run.smithi090.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T21:47:45.334 INFO:teuthology.orchestra.run.smithi090.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 743 kB/s | 20 kB 00:00 2024-09-10T21:47:45.542 INFO:teuthology.orchestra.run.smithi090.stdout:(146/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.8 MB/s | 16 MB 00:02 2024-09-10T21:47:45.576 INFO:teuthology.orchestra.run.smithi090.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 791 kB/s | 191 kB 00:00 2024-09-10T21:47:45.609 INFO:teuthology.orchestra.run.smithi090.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 1.3 MB/s | 427 kB 00:00 2024-09-10T21:47:45.685 INFO:teuthology.orchestra.run.smithi090.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-10T21:47:45.781 INFO:teuthology.orchestra.run.smithi171.stdout:Transaction test succeeded. 2024-09-10T21:47:45.782 INFO:teuthology.orchestra.run.smithi171.stdout:Running transaction 2024-09-10T21:47:45.977 INFO:teuthology.orchestra.run.smithi090.stdout:(150/157): NetworkManager-1.49.90-1.el9.x86_64. 8.0 MB/s | 2.3 MB 00:00 2024-09-10T21:47:46.214 INFO:teuthology.orchestra.run.smithi090.stdout:(151/157): NetworkManager-libnm-1.49.90-1.el9.x 7.6 MB/s | 1.8 MB 00:00 2024-09-10T21:47:46.278 INFO:teuthology.orchestra.run.smithi090.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 641 kB/s | 41 kB 00:00 2024-09-10T21:47:46.345 INFO:teuthology.orchestra.run.smithi090.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 3.7 MB/s | 249 kB 00:00 2024-09-10T21:47:46.445 INFO:teuthology.orchestra.run.smithi090.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-10T21:47:46.653 INFO:teuthology.orchestra.run.smithi090.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-09-10T21:47:46.929 INFO:teuthology.orchestra.run.smithi090.stdout:(156/157): librados2-19.3.0-4854.g8f219c47.el9. 2.6 MB/s | 3.5 MB 00:01 2024-09-10T21:47:47.221 INFO:teuthology.orchestra.run.smithi090.stdout:(157/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 2.0 MB/s | 3.2 MB 00:01 2024-09-10T21:47:47.227 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-09-10T21:47:47.227 INFO:teuthology.orchestra.run.smithi090.stdout:Total 10 MB/s | 240 MB 00:23 2024-09-10T21:47:48.806 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-09-10T21:47:48.990 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T21:47:49.206 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T21:47:49.297 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-09-10T21:47:49.307 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T21:47:49.350 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T21:47:49.390 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T21:47:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T21:47:49.479 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-09-10T21:47:49.479 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-09-10T21:47:49.817 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T21:47:49.932 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:47:50.102 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:47:50.142 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:47:50.246 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:47:50.289 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:47:50.330 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T21:47:50.559 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T21:47:50.620 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T21:47:50.646 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:47:50.764 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:47:50.812 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:47:50.930 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:47:50.957 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:47:50.996 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:47:51.234 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:47:51.454 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:47:51.695 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T21:47:51.839 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:47:51.974 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:47:52.185 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-09-10T21:47:52.186 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-09-10T21:47:52.291 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:47:52.437 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T21:47:52.496 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T21:47:52.541 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T21:47:52.624 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T21:47:52.677 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T21:47:52.716 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T21:47:52.779 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T21:47:52.817 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T21:47:52.851 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T21:47:52.942 INFO:teuthology.orchestra.run.smithi171.stdout: Preparing : 1/1 2024-09-10T21:47:52.965 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T21:47:53.028 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T21:47:53.063 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T21:47:53.110 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T21:47:53.114 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T21:47:53.164 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T21:47:53.199 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T21:47:53.230 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T21:47:53.242 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T21:47:53.283 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T21:47:53.310 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T21:47:53.315 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T21:47:53.339 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T21:47:53.377 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T21:47:53.429 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T21:47:53.483 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T21:47:53.573 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T21:47:53.735 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T21:47:53.793 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T21:47:53.802 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T21:47:53.933 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:47:54.088 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:47:54.126 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:47:54.232 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:47:54.282 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:47:54.322 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T21:47:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T21:47:54.546 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T21:47:54.562 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T21:47:54.597 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T21:47:54.618 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T21:47:54.650 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T21:47:54.666 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:47:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T21:47:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T21:47:54.795 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:47:54.826 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:47:54.900 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:47:54.928 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:47:54.933 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T21:47:54.968 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:47:55.007 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T21:47:55.113 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T21:47:55.197 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T21:47:55.230 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:47:55.281 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T21:47:55.329 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T21:47:55.366 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T21:47:55.421 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:47:55.424 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T21:47:55.465 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T21:47:55.510 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T21:47:55.569 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T21:47:55.605 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T21:47:55.732 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:47:55.751 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T21:47:55.811 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T21:47:55.853 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T21:47:55.855 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:47:55.970 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T21:47:56.010 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T21:47:56.053 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T21:47:56.116 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T21:47:56.140 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T21:47:56.163 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:47:56.268 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T21:47:56.319 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T21:47:56.359 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T21:47:56.406 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T21:47:56.425 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T21:47:56.492 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T21:47:56.513 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T21:47:56.536 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T21:47:56.568 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T21:47:56.573 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T21:47:56.607 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T21:47:56.655 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T21:47:56.661 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T21:47:56.666 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T21:47:56.700 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T21:47:56.703 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T21:47:56.734 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T21:47:56.738 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T21:47:56.803 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T21:47:56.823 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T21:47:56.865 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T21:47:56.904 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T21:47:56.922 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T21:47:56.947 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T21:47:56.993 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T21:47:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T21:47:57.073 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T21:47:57.075 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T21:47:57.109 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T21:47:57.110 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T21:47:57.152 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T21:47:57.231 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T21:47:57.271 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T21:47:57.326 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T21:47:57.337 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T21:47:57.417 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T21:47:57.578 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T21:47:57.647 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T21:47:58.242 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T21:47:58.300 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T21:47:58.304 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T21:47:58.333 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T21:47:58.406 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T21:47:58.445 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T21:47:58.450 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T21:47:58.495 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T21:47:58.498 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T21:47:58.541 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T21:47:58.560 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T21:47:58.613 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T21:47:58.664 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T21:47:58.701 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T21:47:58.804 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T21:47:58.871 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T21:47:58.938 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T21:47:58.973 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T21:47:58.985 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T21:47:59.028 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-09-10T21:47:59.057 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T21:47:59.068 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T21:47:59.160 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T21:47:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T21:47:59.198 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T21:47:59.234 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T21:47:59.293 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T21:47:59.368 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T21:47:59.413 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T21:47:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T21:47:59.472 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T21:47:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T21:47:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T21:47:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T21:47:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T21:47:59.653 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T21:47:59.705 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T21:47:59.749 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T21:47:59.871 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T21:47:59.890 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T21:47:59.937 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T21:47:59.987 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T21:48:00.014 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T21:48:00.017 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T21:48:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T21:48:00.041 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T21:48:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:48:00.184 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T21:48:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:48:00.344 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T21:48:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:48:00.385 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T21:48:00.422 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T21:48:00.458 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T21:48:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:48:00.505 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T21:48:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:48:00.544 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T21:48:00.566 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T21:48:00.612 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T21:48:00.684 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T21:48:00.717 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T21:48:00.761 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T21:48:00.793 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T21:48:00.832 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T21:48:00.894 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T21:48:00.914 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T21:48:00.940 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:48:00.952 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T21:48:00.989 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T21:48:01.066 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:48:01.100 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:48:01.137 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T21:48:01.182 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:48:01.210 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:48:01.251 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:48:01.536 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T21:48:01.615 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T21:48:01.621 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:48:01.796 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T21:48:01.839 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:48:01.840 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T21:48:01.882 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T21:48:02.036 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T21:48:02.219 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:48:02.234 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T21:48:02.285 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:48:02.330 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:48:02.420 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T21:48:02.447 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:48:02.480 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T21:48:02.483 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T21:48:02.515 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T21:48:02.532 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T21:48:02.602 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:48:02.633 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T21:48:02.698 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T21:48:02.744 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T21:48:02.751 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T21:48:02.799 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T21:48:02.847 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T21:48:02.857 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T21:48:02.893 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T21:48:02.929 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T21:48:02.983 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T21:48:03.022 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T21:48:03.076 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T21:48:03.107 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T21:48:03.123 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T21:48:03.131 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T21:48:03.138 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:48:03.187 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T21:48:03.195 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T21:48:03.232 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:48:03.257 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T21:48:03.287 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T21:48:03.289 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T21:48:03.353 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T21:48:03.388 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T21:48:03.434 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T21:48:03.517 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T21:48:03.559 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T21:48:03.636 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T21:48:03.673 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T21:48:03.764 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T21:48:03.817 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T21:48:03.909 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T21:48:04.076 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T21:48:04.100 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T21:48:04.147 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T21:48:04.247 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T21:48:04.784 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T21:48:04.897 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T21:48:04.934 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T21:48:04.978 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T21:48:05.016 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T21:48:05.061 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T21:48:05.262 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T21:48:05.380 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T21:48:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T21:48:05.566 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T21:48:05.659 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T21:48:05.716 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T21:48:05.761 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T21:48:05.809 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T21:48:05.820 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T21:48:05.837 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:48:05.861 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T21:48:05.878 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:48:05.915 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T21:48:05.918 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T21:48:05.954 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T21:48:05.964 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T21:48:05.987 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T21:48:06.136 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T21:48:06.190 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T21:48:06.242 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T21:48:06.340 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T21:48:06.367 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T21:48:06.390 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:48:06.420 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T21:48:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T21:48:06.534 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:48:06.544 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T21:48:06.562 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:48:06.569 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T21:48:06.597 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T21:48:06.606 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:48:06.646 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T21:48:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T21:48:06.737 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:48:06.737 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T21:48:06.738 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T21:48:06.738 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:06.791 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:48:06.855 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T21:48:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T21:48:06.932 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T21:48:06.993 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T21:48:07.058 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T21:48:07.097 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T21:48:07.136 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T21:48:07.170 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T21:48:07.199 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T21:48:07.203 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T21:48:07.244 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:48:07.247 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T21:48:07.305 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T21:48:07.319 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:48:07.364 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T21:48:07.412 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T21:48:07.472 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T21:48:07.509 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T21:48:07.639 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T21:48:07.730 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T21:48:08.608 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T21:48:08.685 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T21:48:08.719 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T21:48:08.832 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T21:48:08.895 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T21:48:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T21:48:09.069 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T21:48:09.115 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T21:48:09.347 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T21:48:09.388 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T21:48:09.478 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T21:48:09.975 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:48:10.020 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:48:10.672 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T21:48:10.686 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:48:10.733 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:48:10.811 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T21:48:10.896 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:48:10.897 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T21:48:10.897 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T21:48:10.897 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:10.942 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:48:12.386 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T21:48:12.483 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T21:48:12.519 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T21:48:12.553 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T21:48:12.904 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T21:48:12.956 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:48:13.077 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:48:13.112 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T21:48:13.197 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T21:48:13.732 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T21:48:13.769 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:48:13.860 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:48:13.888 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T21:48:16.510 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:48:16.553 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:48:17.212 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:48:17.299 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:48:17.434 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:48:17.434 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T21:48:17.434 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T21:48:17.434 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:17.467 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:48:19.498 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:48:19.498 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-09-10T21:48:19.499 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-09-10T21:48:19.499 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-09-10T21:48:19.499 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-09-10T21:48:19.499 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-09-10T21:48:19.499 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2024-09-10T21:48:19.499 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-09-10T21:48:19.499 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:19.719 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:48:19.756 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:48:19.756 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:19.756 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T21:48:19.756 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:21.389 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:48:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:48:21.447 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:21.447 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T21:48:21.447 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:48:21.447 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:48:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:21.697 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:48:21.749 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:48:21.749 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:21.749 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T21:48:21.749 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:48:21.749 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:48:21.749 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:22.240 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:48:22.290 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:48:22.290 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:22.290 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T21:48:22.291 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:48:22.291 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:48:22.291 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:22.440 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T21:48:22.534 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T21:48:22.606 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T21:48:22.833 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T21:48:22.919 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T21:48:23.012 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T21:48:23.130 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T21:48:23.287 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T21:48:23.348 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T21:48:23.392 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T21:48:23.417 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T21:48:23.565 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:48:23.565 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /sys 2024-09-10T21:48:23.565 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /proc 2024-09-10T21:48:23.565 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /mnt 2024-09-10T21:48:23.565 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /var/tmp 2024-09-10T21:48:23.565 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /home 2024-09-10T21:48:23.565 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /root 2024-09-10T21:48:23.565 INFO:teuthology.orchestra.run.smithi171.stdout:skipping the directory /tmp 2024-09-10T21:48:23.566 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:23.800 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:48:23.836 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:48:23.836 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:23.836 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T21:48:23.837 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:24.110 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:48:24.142 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:48:24.175 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T21:48:24.264 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T21:48:24.297 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:48:25.458 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:48:25.515 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:48:25.515 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:25.515 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T21:48:25.515 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:48:25.515 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:48:25.516 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:25.756 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:48:25.805 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:48:25.806 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:25.806 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T21:48:25.806 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:48:25.806 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:48:25.806 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:26.331 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:48:26.382 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:48:26.382 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:26.382 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T21:48:26.382 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:48:26.383 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:48:26.383 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:26.524 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T21:48:26.583 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T21:48:26.648 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T21:48:26.874 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T21:48:26.937 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T21:48:27.031 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T21:48:27.149 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T21:48:27.317 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T21:48:27.376 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T21:48:27.419 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T21:48:27.445 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T21:48:27.943 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:48:28.012 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:48:28.071 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:48:28.113 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:48:28.155 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T21:48:28.235 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T21:48:28.287 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:48:29.099 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:48:29.128 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:48:29.280 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:48:29.468 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T21:48:29.497 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:48:29.548 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:48:29.549 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:29.549 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T21:48:29.549 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:48:29.549 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:48:29.549 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:29.623 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:48:29.649 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:48:29.756 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T21:48:29.837 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T21:48:30.942 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:48:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:48:30.995 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:30.995 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T21:48:30.995 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:48:30.995 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:48:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:48:31.101 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:48:31.101 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:31.101 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T21:48:31.101 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:31.160 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:48:31.211 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:48:31.211 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:31.212 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T21:48:31.212 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:48:31.212 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:48:31.212 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:31.552 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:48:31.603 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:48:31.603 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:31.603 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T21:48:31.604 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:48:31.604 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:48:31.604 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:31.914 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:48:31.993 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:48:32.941 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:48:32.941 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-09-10T21:48:32.941 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-09-10T21:48:32.941 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-09-10T21:48:32.941 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-09-10T21:48:32.941 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-09-10T21:48:32.941 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-09-10T21:48:32.941 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-09-10T21:48:32.942 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:33.081 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:48:33.118 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:48:33.155 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:48:33.189 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:48:33.189 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:33.189 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T21:48:33.189 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:33.271 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:48:33.453 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T21:48:33.505 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:48:33.557 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:48:33.557 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:33.558 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T21:48:33.558 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:48:33.558 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:48:33.558 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:33.622 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:48:33.649 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:48:33.761 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T21:48:33.861 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T21:48:34.861 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:48:34.901 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T21:48:34.919 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:48:34.919 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:34.919 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T21:48:34.920 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:48:34.920 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:48:34.920 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:34.934 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:48:34.987 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:48:34.987 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:34.987 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T21:48:34.987 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:48:34.987 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:48:34.987 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:35.078 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:48:35.128 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:48:35.128 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:35.128 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T21:48:35.129 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:35.173 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:48:35.194 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:48:35.220 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:48:35.220 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:35.220 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T21:48:35.220 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:48:35.221 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:48:35.221 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:48:35.246 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:35.246 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T21:48:35.246 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:48:35.246 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:48:35.246 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:35.596 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:48:35.646 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:48:35.646 INFO:teuthology.orchestra.run.smithi171.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:35.646 INFO:teuthology.orchestra.run.smithi171.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T21:48:35.646 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:48:35.647 INFO:teuthology.orchestra.run.smithi171.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:48:35.647 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:35.704 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:48:35.755 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:48:35.755 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:35.755 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T21:48:35.755 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:48:35.755 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:48:35.756 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:35.888 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T21:48:35.956 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T21:48:36.029 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T21:48:36.254 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T21:48:36.319 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T21:48:36.405 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T21:48:36.521 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T21:48:36.708 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T21:48:36.744 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T21:48:36.749 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T21:48:36.793 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T21:48:36.794 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T21:48:36.821 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T21:48:36.834 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T21:48:36.857 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T21:48:36.860 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T21:48:37.108 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T21:48:37.162 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T21:48:37.193 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T21:48:37.232 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T21:48:37.284 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T21:48:37.507 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:48:37.545 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:48:37.587 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T21:48:37.667 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T21:48:37.701 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:48:38.111 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:48:38.157 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:48:38.200 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:48:39.485 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T21:48:39.604 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T21:48:39.671 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T21:48:39.787 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T21:48:39.824 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T21:48:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T21:48:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T21:48:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:48:40.043 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:48:40.043 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:48:40.185 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:48:40.209 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T21:48:40.229 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:48:40.229 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:48:40.462 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:48:40.462 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:48:40.613 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:48:40.696 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:48:40.758 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T21:48:40.811 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T21:48:40.855 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T21:48:40.915 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T21:48:40.948 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:48:40.948 INFO:teuthology.orchestra.run.smithi154.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T21:48:40.948 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:40.953 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T21:48:41.193 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T21:48:41.261 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T21:48:41.291 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T21:48:41.294 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:48:41.360 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:48:41.364 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T21:48:41.408 INFO:teuthology.orchestra.run.smithi171.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T21:48:42.207 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:48:42.258 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:48:42.300 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:48:42.445 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:48:42.485 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:48:42.636 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:48:42.816 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T21:48:42.855 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:48:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:48:42.906 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:42.906 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T21:48:42.906 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:48:42.906 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:48:42.906 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:42.966 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:48:42.992 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:48:43.111 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T21:48:43.197 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T21:48:43.587 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T21:48:43.693 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T21:48:43.755 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T21:48:43.862 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T21:48:43.896 INFO:teuthology.orchestra.run.smithi171.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T21:48:43.930 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T21:48:43.975 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T21:48:43.976 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:48:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:48:44.091 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:48:44.209 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:48:44.237 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T21:48:44.268 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T21:48:44.277 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:48:44.284 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:48:44.284 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:48:44.327 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:48:44.328 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:44.328 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T21:48:44.328 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:48:44.328 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:48:44.328 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:44.386 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:48:44.435 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:48:44.435 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:44.435 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T21:48:44.436 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:44.495 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:48:44.528 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:48:44.528 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:48:44.545 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:48:44.545 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:44.545 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T21:48:44.546 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:48:44.546 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:48:44.546 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:44.646 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:48:44.746 INFO:teuthology.orchestra.run.smithi171.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:48:44.896 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:48:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:48:44.941 INFO:teuthology.orchestra.run.smithi171.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T21:48:44.941 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:44.946 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:48:44.946 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:48:44.946 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T21:48:44.946 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:48:44.946 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:48:44.947 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:47.321 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T21:48:47.321 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T21:48:47.321 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T21:48:47.322 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T21:48:47.323 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T21:48:47.335 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T21:48:47.336 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T21:48:47.337 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T21:48:47.338 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T21:48:47.339 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T21:48:47.340 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T21:48:47.341 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T21:48:47.342 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T21:48:47.343 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T21:48:47.344 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T21:48:47.345 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T21:48:49.659 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-09-10T21:48:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T21:48:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T21:48:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T21:48:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T21:48:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T21:48:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T21:48:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T21:48:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T21:48:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T21:48:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T21:48:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T21:48:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T21:48:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T21:48:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T21:48:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T21:48:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T21:48:49.672 INFO:teuthology.orchestra.run.smithi154.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T21:48:49.672 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:48:49.672 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-09-10T21:48:50.006 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T21:48:50.056 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T21:48:50.116 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T21:48:50.155 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T21:48:50.190 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T21:48:50.311 DEBUG:teuthology.parallel:result is None 2024-09-10T21:48:50.418 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T21:48:50.517 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T21:48:50.544 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T21:48:50.575 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T21:48:50.637 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T21:48:51.438 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:48:51.496 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:48:51.500 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T21:48:51.500 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T21:48:51.500 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T21:48:51.500 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T21:48:51.500 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T21:48:51.501 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T21:48:51.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T21:48:51.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T21:48:51.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T21:48:51.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T21:48:51.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T21:48:51.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T21:48:51.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T21:48:51.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T21:48:51.502 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T21:48:51.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T21:48:51.503 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T21:48:51.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T21:48:51.514 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T21:48:51.515 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T21:48:51.516 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T21:48:51.517 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T21:48:51.518 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T21:48:51.519 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T21:48:51.520 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T21:48:51.521 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T21:48:51.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T21:48:51.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T21:48:51.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T21:48:51.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T21:48:51.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T21:48:51.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T21:48:51.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T21:48:51.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T21:48:51.522 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T21:48:51.523 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T21:48:51.524 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T21:48:51.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T21:48:51.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T21:48:51.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T21:48:51.525 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T21:48:51.538 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:48:52.801 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T21:48:52.914 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T21:48:53.028 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T21:48:53.142 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T21:48:53.185 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T21:48:53.219 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T21:48:53.255 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T21:48:53.255 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:48:53.408 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:48:53.409 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:48:53.522 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:48:53.555 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T21:48:53.571 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:48:53.571 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:48:53.826 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:48:53.826 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:48:53.886 INFO:teuthology.orchestra.run.smithi171.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout:Upgraded: 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T21:48:53.887 INFO:teuthology.orchestra.run.smithi171.stdout:Installed: 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.888 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.889 INFO:teuthology.orchestra.run.smithi171.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T21:48:53.890 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.891 INFO:teuthology.orchestra.run.smithi171.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T21:48:53.892 INFO:teuthology.orchestra.run.smithi171.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.893 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T21:48:53.894 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T21:48:53.895 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T21:48:53.896 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T21:48:53.897 INFO:teuthology.orchestra.run.smithi171.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.898 INFO:teuthology.orchestra.run.smithi171.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:48:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T21:48:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T21:48:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T21:48:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T21:48:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T21:48:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T21:48:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T21:48:53.899 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:48:53.899 INFO:teuthology.orchestra.run.smithi171.stdout:Complete! 2024-09-10T21:48:53.935 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:48:54.008 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:48:54.213 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:48:54.213 INFO:teuthology.orchestra.run.smithi090.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T21:48:54.213 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:48:54.654 DEBUG:teuthology.parallel:result is None 2024-09-10T21:49:02.387 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T21:49:02.387 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T21:49:02.387 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T21:49:02.389 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T21:49:02.404 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T21:49:02.404 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T21:49:02.404 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T21:49:02.404 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T21:49:02.404 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T21:49:02.406 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T21:49:02.407 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T21:49:02.408 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T21:49:02.409 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T21:49:02.410 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T21:49:02.411 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T21:49:02.412 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T21:49:02.413 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T21:49:02.414 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T21:49:02.415 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T21:49:02.415 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T21:49:02.415 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T21:49:02.415 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T21:49:02.415 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T21:49:02.415 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T21:49:02.415 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T21:49:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:49:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:49:04.638 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-09-10T21:49:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T21:49:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T21:49:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T21:49:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T21:49:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.638 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.639 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.640 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T21:49:04.641 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T21:49:04.642 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T21:49:04.643 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T21:49:04.644 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T21:49:04.645 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T21:49:04.646 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T21:49:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T21:49:04.648 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.649 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:49:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T21:49:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T21:49:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T21:49:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T21:49:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T21:49:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T21:49:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T21:49:04.650 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:49:04.651 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-09-10T21:49:05.377 DEBUG:teuthology.parallel:result is None 2024-09-10T21:49:05.378 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:49:05.526 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T21:49:05.571 INFO:teuthology.orchestra.run.smithi090.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T21:49:05.572 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T21:49:05.572 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T21:49:05.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:49:05.718 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T21:49:05.769 INFO:teuthology.orchestra.run.smithi154.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T21:49:05.769 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T21:49:05.769 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T21:49:05.771 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:49:05.916 DEBUG:teuthology.orchestra.run.smithi171:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T21:49:05.966 INFO:teuthology.orchestra.run.smithi171.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T21:49:05.967 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T21:49:05.967 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T21:49:05.969 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T21:49:05.969 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:49:05.969 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:49:06.034 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:49:06.034 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:49:06.102 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:49:06.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:49:06.165 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T21:49:06.165 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:49:06.165 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:49:06.233 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:49:06.335 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:49:06.335 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:49:06.394 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:49:06.490 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:49:06.490 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:49:06.550 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:49:06.646 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T21:49:06.647 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:49:06.647 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:49:06.706 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:49:06.803 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:49:06.803 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:49:06.859 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:49:06.954 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:49:06.954 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:49:07.010 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:49:07.105 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T21:49:07.106 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:49:07.106 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:49:07.164 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:49:07.260 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:49:07.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:49:07.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:49:07.410 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:49:07.410 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:49:07.466 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:49:07.560 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T21:49:07.652 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'mgr-modules': ['snap_schedule'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-10T21:49:07.652 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:49:07.652 INFO:tasks.cephadm:Cluster fsid is 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:49:07.652 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T21:49:07.652 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.90', 'mon.b': '172.21.15.154', 'mon.c': '172.21.15.171'} 2024-09-10T21:49:07.653 INFO:tasks.cephadm:First mon is mon.a on smithi090 2024-09-10T21:49:07.653 INFO:tasks.cephadm:First mgr is x 2024-09-10T21:49:07.653 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T21:49:07.653 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-09-10T21:49:07.710 DEBUG:teuthology.orchestra.run.smithi154:> sudo hostname $(hostname -s) 2024-09-10T21:49:07.767 DEBUG:teuthology.orchestra.run.smithi171:> sudo hostname $(hostname -s) 2024-09-10T21:49:07.823 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T21:49:07.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:49:07.972 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82938/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4854-g8f219c47', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4854.g8f219c47'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 09:27:13.215997', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-09-10-0723', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-10T21:49:08.115 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-hemanth-testing-2024-09-10-0723, sha1 8f219c476cb4f26fc5719c661c0323719a45cb30 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:49:08.116 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T21:49:08.117 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T21:49:08.117 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T21:49:08.556 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 21:49 /home/ubuntu/cephtest/cephadm 2024-09-10T21:49:08.557 DEBUG:teuthology.orchestra.run.smithi154:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T21:49:08.961 INFO:teuthology.orchestra.run.smithi154.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 21:49 /home/ubuntu/cephtest/cephadm 2024-09-10T21:49:08.962 DEBUG:teuthology.orchestra.run.smithi171:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T21:49:09.382 INFO:teuthology.orchestra.run.smithi171.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 21:49 /home/ubuntu/cephtest/cephadm 2024-09-10T21:49:09.383 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T21:49:09.424 DEBUG:teuthology.orchestra.run.smithi154:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T21:49:09.466 DEBUG:teuthology.orchestra.run.smithi171:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T21:49:09.522 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 on all hosts... 2024-09-10T21:49:09.522 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T21:49:09.526 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T21:49:09.530 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T21:49:11.517 INFO:teuthology.orchestra.run.smithi154.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T21:49:11.793 INFO:teuthology.orchestra.run.smithi171.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T21:49:12.538 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T21:49:50.326 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-10T21:49:50.326 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T21:49:50.327 INFO:teuthology.orchestra.run.smithi154.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T21:49:50.327 INFO:teuthology.orchestra.run.smithi154.stdout: "repo_digests": [ 2024-09-10T21:49:50.327 INFO:teuthology.orchestra.run.smithi154.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T21:49:50.327 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-09-10T21:49:50.327 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-10T21:49:55.262 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T21:49:55.263 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T21:49:55.263 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T21:49:55.263 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-09-10T21:49:55.263 INFO:teuthology.orchestra.run.smithi090.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T21:49:55.263 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T21:49:55.263 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T21:50:00.036 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-10T21:50:00.036 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T21:50:00.036 INFO:teuthology.orchestra.run.smithi171.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T21:50:00.036 INFO:teuthology.orchestra.run.smithi171.stdout: "repo_digests": [ 2024-09-10T21:50:00.036 INFO:teuthology.orchestra.run.smithi171.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T21:50:00.037 INFO:teuthology.orchestra.run.smithi171.stdout: ] 2024-09-10T21:50:00.037 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-10T21:50:00.073 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-09-10T21:50:00.139 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph 2024-09-10T21:50:00.200 DEBUG:teuthology.orchestra.run.smithi171:> sudo mkdir -p /etc/ceph 2024-09-10T21:50:00.260 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-09-10T21:50:00.318 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /etc/ceph 2024-09-10T21:50:00.375 DEBUG:teuthology.orchestra.run.smithi171:> sudo chmod 777 /etc/ceph 2024-09-10T21:50:00.432 INFO:tasks.cephadm:Writing seed config... 2024-09-10T21:50:00.433 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-10T21:50:00.433 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T21:50:00.433 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T21:50:00.434 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T21:50:00.435 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T21:50:00.435 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T21:50:00.435 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T21:50:00.435 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T21:50:00.435 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T21:50:00.435 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T21:50:00.435 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T21:50:00.436 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:50:00.436 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T21:50:00.471 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 815bde3e-6fbe-11ef-bcea-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_bal_replicate_threshold = 1 2024-09-10T21:50:00.472 DEBUG:teuthology.orchestra.run.smithi090:mon.a> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T21:50:00.516 DEBUG:teuthology.orchestra.run.smithi090:mgr.x> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T21:50:00.560 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T21:50:00.561 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 -v bootstrap --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.90 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T21:50:00.819 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-09-10T21:50:00.819 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', '-v', 'bootstrap', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.90', '--skip-admin-label'] 2024-09-10T21:50:00.820 INFO:teuthology.orchestra.run.smithi090.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T21:50:00.820 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying podman|docker is present... 2024-09-10T21:50:00.865 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 5.2.2 2024-09-10T21:50:00.865 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying lvm2 is present... 2024-09-10T21:50:00.866 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying time synchronization is in place... 2024-09-10T21:50:00.879 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T21:50:00.879 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T21:50:00.892 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T21:50:00.892 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-10T21:50:00.905 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-09-10T21:50:00.918 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-09-10T21:50:00.918 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-09-10T21:50:00.918 INFO:teuthology.orchestra.run.smithi090.stdout:Repeating the final host check... 2024-09-10T21:50:00.971 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 5.2.2 2024-09-10T21:50:00.971 INFO:teuthology.orchestra.run.smithi090.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T21:50:00.971 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl is present 2024-09-10T21:50:00.971 INFO:teuthology.orchestra.run.smithi090.stdout:lvcreate is present 2024-09-10T21:50:00.984 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T21:50:00.984 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T21:50:00.996 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T21:50:00.996 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-10T21:50:01.009 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-09-10T21:50:01.022 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-09-10T21:50:01.022 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-09-10T21:50:01.022 INFO:teuthology.orchestra.run.smithi090.stdout:Host looks OK 2024-09-10T21:50:01.022 INFO:teuthology.orchestra.run.smithi090.stdout:Cluster fsid: 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:01.023 INFO:teuthology.orchestra.run.smithi090.stdout:Acquiring lock 140458080854896 on /run/cephadm/815bde3e-6fbe-11ef-bcea-c7b262605968.lock 2024-09-10T21:50:01.023 INFO:teuthology.orchestra.run.smithi090.stdout:Lock 140458080854896 acquired on /run/cephadm/815bde3e-6fbe-11ef-bcea-c7b262605968.lock 2024-09-10T21:50:01.023 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 3300 ... 2024-09-10T21:50:01.023 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 6789 ... 2024-09-10T21:50:01.023 INFO:teuthology.orchestra.run.smithi090.stdout:Base mon IP(s) is [172.21.15.90:3300, 172.21.15.90:6789], mon addrv is [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-09-10T21:50:01.030 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.90 metric 100 2024-09-10T21:50:01.030 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.90 metric 100 2024-09-10T21:50:01.036 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T21:50:01.036 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T21:50:01.036 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T21:50:01.042 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T21:50:01.042 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T21:50:01.042 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T21:50:01.042 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T21:50:01.042 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d767/64 scope link noprefixroute 2024-09-10T21:50:01.042 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T21:50:01.043 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-09-10T21:50:01.043 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-09-10T21:50:01.043 INFO:teuthology.orchestra.run.smithi090.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T21:50:01.043 INFO:teuthology.orchestra.run.smithi090.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T21:50:01.044 INFO:teuthology.orchestra.run.smithi090.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T21:50:01.575 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T21:50:01.576 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T21:50:01.576 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T21:50:01.576 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:edd8cbe4deb828f809e8d76f308301dd93e971948e909d25cc03015fed1fd796 2024-09-10T21:50:01.576 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T21:50:01.576 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying config sha256:589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T21:50:01.576 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T21:50:02.676 INFO:teuthology.orchestra.run.smithi090.stdout:ceph: stdout ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:50:02.676 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph version: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T21:50:02.676 INFO:teuthology.orchestra.run.smithi090.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T21:50:03.813 INFO:teuthology.orchestra.run.smithi090.stdout:stat: stdout 167 167 2024-09-10T21:50:03.814 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial keys... 2024-09-10T21:50:04.805 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQAMv+BmDxA9DhAAJgnEc2lNqcz1ek2KAU65rw== 2024-09-10T21:50:06.048 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQANv+BmTO04DBAAltw52pWHFl2OXFsYAxHjhA== 2024-09-10T21:50:07.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQAOv+BmbqGqGhAA10JgaH+A7MiA15xf7jKBYg== 2024-09-10T21:50:07.205 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial monmap... 2024-09-10T21:50:08.145 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T21:50:08.145 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T21:50:08.146 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:08.146 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T21:50:08.146 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool for a [v2:172.21.15.90:3300,v1:172.21.15.90:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T21:50:08.146 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = quincy 2024-09-10T21:50:08.146 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: set fsid to 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:08.146 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T21:50:08.146 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:50:08.146 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mon... 2024-09-10T21:50:09.180 INFO:teuthology.orchestra.run.smithi090.stdout:create mon.a on 2024-09-10T21:50:09.463 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T21:50:09.690 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T21:50:10.005 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968.target. 2024-09-10T21:50:10.005 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968.target. 2024-09-10T21:50:10.323 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.a 2024-09-10T21:50:10.323 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T21:50:10.585 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968.target.wants/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@.service. 2024-09-10T21:50:11.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:10 smithi090 podman[29961]: 2024-09-10 21:50:10.970445092 +0000 UTC m=+0.163639485 container create d5f2def3f1b91c807c74272d09a18626857ae5801a795ed5b60dadf0d968db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T21:50:11.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:11 smithi090 podman[29961]: 2024-09-10 21:50:11.20594738 +0000 UTC m=+0.399141793 container init d5f2def3f1b91c807c74272d09a18626857ae5801a795ed5b60dadf0d968db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T21:50:11.308 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T21:50:11.308 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-09-10T21:50:11.321 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T21:50:11.322 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-10T21:50:11.322 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-09-10T21:50:11.322 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T21:50:11.322 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon to start... 2024-09-10T21:50:11.322 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon... 2024-09-10T21:50:11.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:11 smithi090 podman[29961]: 2024-09-10 21:50:11.212535067 +0000 UTC m=+0.405729461 container start d5f2def3f1b91c807c74272d09a18626857ae5801a795ed5b60dadf0d968db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T21:50:11.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:11 smithi090 ceph-mon[29980]: mkfs 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:11.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:11 smithi090 ceph-mon[29980]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T21:50:11.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:11 smithi090 bash[29961]: d5f2def3f1b91c807c74272d09a18626857ae5801a795ed5b60dadf0d968db58 2024-09-10T21:50:11.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:11 smithi090 systemd[1]: Started Ceph mon.a for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:50:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T21:50:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: monmap epoch 1 2024-09-10T21:50:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: last_changed 2024-09-10T21:50:07.568893+0000 2024-09-10T21:50:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: created 2024-09-10T21:50:07.568893+0000 2024-09-10T21:50:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: min_mon_release 19 (squid) 2024-09-10T21:50:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: election_strategy: 1 2024-09-10T21:50:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-09-10T21:50:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: fsmap 2024-09-10T21:50:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T21:50:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: mgrmap e1: no daemons active 2024-09-10T21:50:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:12 smithi090 ceph-mon[29980]: from='client.? 172.21.15.90:0/3674591873' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout id: 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout services: 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.587485s) 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout data: 2024-09-10T21:50:12.607 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T21:50:12.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T21:50:12.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T21:50:12.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T21:50:12.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:12.608 INFO:teuthology.orchestra.run.smithi090.stdout:mon is available 2024-09-10T21:50:12.608 INFO:teuthology.orchestra.run.smithi090.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T21:50:13.530 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:13 smithi090 ceph-mon[29980]: from='client.? 172.21.15.90:0/48126818' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T21:50:13.530 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:13 smithi090 ceph-mon[29980]: from='client.? 172.21.15.90:0/48126818' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T21:50:13.825 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T21:50:13.826 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T21:50:13.827 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T21:50:13.827 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T21:50:13.827 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T21:50:13.827 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T21:50:13.827 INFO:teuthology.orchestra.run.smithi090.stdout:Generating new minimal ceph.conf... 2024-09-10T21:50:24.660 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:24 smithi090 ceph-mon[29980]: from='client.? 172.21.15.90:0/2861352741' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:50:28.489 INFO:teuthology.orchestra.run.smithi090.stdout:Restarting the monitor... 2024-09-10T21:50:28.839 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:28 smithi090 systemd[1]: Stopping Ceph mon.a for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T21:50:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:28 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a[29976]: 2024-09-10T21:50:28.837+0000 7f9611600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:50:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:28 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a[29976]: 2024-09-10T21:50:28.837+0000 7f9611600640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T21:50:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:28 smithi090 podman[30231]: 2024-09-10 21:50:28.879377504 +0000 UTC m=+0.158899443 container died d5f2def3f1b91c807c74272d09a18626857ae5801a795ed5b60dadf0d968db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:50:29.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:29 smithi090 podman[30231]: 2024-09-10 21:50:29.275811961 +0000 UTC m=+0.555333909 container cleanup d5f2def3f1b91c807c74272d09a18626857ae5801a795ed5b60dadf0d968db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T21:50:29.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:29 smithi090 bash[30231]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a 2024-09-10T21:50:29.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:29 smithi090 podman[30242]: 2024-09-10 21:50:29.518500611 +0000 UTC m=+0.628537789 container remove d5f2def3f1b91c807c74272d09a18626857ae5801a795ed5b60dadf0d968db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T21:50:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:29 smithi090 systemd[1]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T21:50:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:29 smithi090 systemd[1]: Stopped Ceph mon.a for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:50:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:29 smithi090 systemd[1]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.211s CPU time. 2024-09-10T21:50:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:29 smithi090 systemd[1]: Starting Ceph mon.a for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T21:50:30.240 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:29 smithi090 podman[30332]: 2024-09-10 21:50:29.89294417 +0000 UTC m=+0.027064853 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:50:30.241 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:29 smithi090 podman[30332]: 2024-09-10 21:50:29.994612918 +0000 UTC m=+0.128733586 container create ebe243d48873048e3afe33be4664c0d09e07d38bba69980f2dc888198be36372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:50:30.241 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 podman[30332]: 2024-09-10 21:50:30.240383922 +0000 UTC m=+0.374504603 container init ebe243d48873048e3afe33be4664c0d09e07d38bba69980f2dc888198be36372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T21:50:30.315 INFO:teuthology.orchestra.run.smithi090.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T21:50:30.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 podman[30332]: 2024-09-10 21:50:30.247048547 +0000 UTC m=+0.381169219 container start ebe243d48873048e3afe33be4664c0d09e07d38bba69980f2dc888198be36372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T21:50:30.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T21:50:30.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T21:50:30.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: pidfile_write: ignore empty --pid-file 2024-09-10T21:50:30.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: load: jerasure load: lrc 2024-09-10T21:50:30.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: RocksDB version: 7.9.2 2024-09-10T21:50:30.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Git sha 0 2024-09-10T21:50:30.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T21:50:30.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: DB SUMMARY 2024-09-10T21:50:30.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: DB Session ID: V9CXDXOCUHG9BTMS2HV4 2024-09-10T21:50:30.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: CURRENT file: CURRENT 2024-09-10T21:50:30.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T21:50:30.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T21:50:30.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T21:50:30.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95158 ; 2024-09-10T21:50:30.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.error_if_exists: 0 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.create_if_missing: 0 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.paranoid_checks: 1 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.env: 0x5621ff20e180 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.info_log: 0x562237fbb6e0 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.statistics: (nil) 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.use_fsync: 0 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_log_file_size: 0 2024-09-10T21:50:30.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T21:50:30.574 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T21:50:30.574 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T21:50:30.574 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T21:50:30.574 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.allow_fallocate: 1 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.use_direct_reads: 0 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.db_log_dir: 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.wal_dir: 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T21:50:30.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.write_buffer_manager: 0x562237fbfb80 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T21:50:30.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.unordered_write: 0 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.row_cache: None 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.wal_filter: None 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.two_write_queues: 0 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T21:50:30.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.wal_compression: 0 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.atomic_flush: 0 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.log_readahead_size: 0 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_background_jobs: 2 2024-09-10T21:50:30.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_background_compactions: -1 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_subcompactions: 1 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_open_files: -1 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T21:50:30.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_background_flushes: -1 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Compression algorithms supported: 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: kZSTD supported: 0 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: kXpressCompression supported: 0 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: kBZip2Compression supported: 0 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: kLZ4Compression supported: 1 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: kZlibCompression supported: 1 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: kSnappyCompression supported: 1 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T21:50:30.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.merge_operator: 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_filter: None 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562237fbb300) 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-09-10T21:50:30.582 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_type: 0 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_index_type: 0 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_shortening: 1 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: checksum: 4 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: no_block_cache: 0 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache: 0x562237faf610 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_options: 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: capacity : 536870912 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_shard_bits : 4 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: strict_capacity_limit : 0 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-09-10T21:50:30.583 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_compressed: (nil) 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: persistent_cache: (nil) 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size: 4096 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size_deviation: 10 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_restart_interval: 16 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_block_restart_interval: 1 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: metadata_block_size: 4096 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: partition_filters: 0 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: use_delta_encoding: 1 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: filter_policy: bloomfilter 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: whole_key_filtering: 1 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: verify_compression: 0 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-09-10T21:50:30.584 INFO:journalctl@ceph.mon.a.smithi090.stdout: format_version: 5 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_index_compression: 1 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_align: 0 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout: max_auto_readahead_size: 262144 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout: prepopulate_block_cache: 0 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout: initial_auto_readahead_size: 8192 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression: NoCompression 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T21:50:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.num_levels: 7 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T21:50:30.586 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T21:50:30.587 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T21:50:30.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.inplace_update_support: 0 2024-09-10T21:50:30.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.bloom_locality: 0 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.max_successive_merges: 0 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.ttl: 2592000 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T21:50:30.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.enable_blob_files: false 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.min_blob_size: 0 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 298a3096-6519-41d4-899c-d96c6f51f2f7 2024-09-10T21:50:30.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726005030288922, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726005030290644, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91365, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 285, "table_properties": {"data_size": 89406, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11684, "raw_average_key_size": 47, "raw_value_size": 82781, "raw_average_value_size": 339, "num_data_blocks": 12, "num_entries": 244, "num_filter_entries": 244, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726005030, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "298a3096-6519-41d4-899c-d96c6f51f2f7", "db_session_id": "V9CXDXOCUHG9BTMS2HV4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726005030290819, "job": 1, "event": "recovery_finished"} 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562237fe0e00 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: DB pointer 0x5622380e0000 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** DB Stats ** 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T21:50:30.592 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: L0 2/0 91.08 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: Sum 2/0 91.08 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 74.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-09-10T21:50:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 74.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative compaction: 0.00 GB write, 15.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval compaction: 0.00 GB write, 15.31 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T21:50:30.594 INFO:journalctl@ceph.mon.a.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache BinnedLRUCache@0x562237faf610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: starting mon.a rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mon.a@-1(???) e1 preinit fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mon.a@-1(???).mds e1 new map 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mon.a@-1(???).mds e1 print_map 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: e1 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: btime 2024-09-10T21:50:11:271199+0000 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: legacy client fscid: -1 2024-09-10T21:50:30.595 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout: No filesystems configured 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 bash[30332]: ebe243d48873048e3afe33be4664c0d09e07d38bba69980f2dc888198be36372 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 systemd[1]: Started Ceph mon.a for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: monmap epoch 1 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: last_changed 2024-09-10T21:50:07.568893+0000 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: created 2024-09-10T21:50:07.568893+0000 2024-09-10T21:50:30.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: min_mon_release 19 (squid) 2024-09-10T21:50:30.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: election_strategy: 1 2024-09-10T21:50:30.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-09-10T21:50:30.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: fsmap 2024-09-10T21:50:30.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T21:50:30.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:30 smithi090 ceph-mon[30350]: mgrmap e1: no daemons active 2024-09-10T21:50:31.522 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T21:50:31.548 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T21:50:31.549 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mgr... 2024-09-10T21:50:31.549 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T21:50:31.549 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T21:50:31.893 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.x 2024-09-10T21:50:31.893 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T21:50:32.127 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968.target.wants/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@.service. 2024-09-10T21:50:32.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:31 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/778566794' entity='client.admin' 2024-09-10T21:50:32.463 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:32 smithi090 systemd[1]: Starting Ceph mgr.x for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T21:50:32.746 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:32 smithi090 podman[30560]: 2024-09-10 21:50:32.366947022 +0000 UTC m=+0.027020662 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:50:32.746 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:32 smithi090 podman[30560]: 2024-09-10 21:50:32.493379669 +0000 UTC m=+0.153453308 container create bd728928651d2b5e5ca7c48d61af2025b0321c6c7514dea9a0e4b4905ad8a622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T21:50:32.746 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:32 smithi090 podman[30560]: 2024-09-10 21:50:32.746389328 +0000 UTC m=+0.406462992 container init bd728928651d2b5e5ca7c48d61af2025b0321c6c7514dea9a0e4b4905ad8a622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T21:50:32.860 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T21:50:32.861 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-09-10T21:50:32.875 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T21:50:32.875 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-10T21:50:32.875 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-09-10T21:50:32.876 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T21:50:32.896 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T21:50:32.896 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-09-10T21:50:32.911 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T21:50:32.911 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-10T21:50:32.911 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-09-10T21:50:32.911 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T21:50:32.911 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr to start... 2024-09-10T21:50:32.912 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr... 2024-09-10T21:50:33.039 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:32 smithi090 podman[30560]: 2024-09-10 21:50:32.752439243 +0000 UTC m=+0.412512882 container start bd728928651d2b5e5ca7c48d61af2025b0321c6c7514dea9a0e4b4905ad8a622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T21:50:33.039 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:32 smithi090 bash[30560]: bd728928651d2b5e5ca7c48d61af2025b0321c6c7514dea9a0e4b4905ad8a622 2024-09-10T21:50:33.039 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:32 smithi090 systemd[1]: Started Ceph mgr.x for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:50:33.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:33 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1766264548' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T21:50:34.446 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:34.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-10T21:50:34.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T21:50:34.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T21:50:34.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T21:50:34.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T21:50:34.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T21:50:34.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:34.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T21:50:34.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:34.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T21:50:34.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T21:50:34.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T21:50:34.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T21:50:34.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T21:50:34.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T21:50:34.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:34.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T21:50:34.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T21:50:34.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T21:50:34.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T21:50:34.454 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T21:50:11:271199+0000", 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T21:50:34.455 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T21:50:11.271935+0000", 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-10T21:50:34.456 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (1/15)... 2024-09-10T21:50:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:37 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3397969124' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T21:50:37.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:37.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-10T21:50:37.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T21:50:37.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T21:50:37.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T21:50:37.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T21:50:37.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:37.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T21:50:37.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:37.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T21:50:37.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T21:50:37.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T21:50:37.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T21:50:37.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T21:50:37.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T21:50:37.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:37.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T21:50:37.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T21:50:37.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T21:50:37.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T21:50:37.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T21:50:11:271199+0000", 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T21:50:37.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T21:50:11.271935+0000", 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-10T21:50:37.796 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (2/15)... 2024-09-10T21:50:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: Activating manager daemon x 2024-09-10T21:50:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: mgrmap e2: x(active, starting, since 0.00449811s) 2024-09-10T21:50:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T21:50:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T21:50:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T21:50:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:50:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T21:50:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: Manager daemon x is now available 2024-09-10T21:50:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T21:50:39.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T21:50:39.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' 2024-09-10T21:50:39.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' 2024-09-10T21:50:39.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:38 smithi090 ceph-mon[30350]: from='mgr.14100 172.21.15.90:0/3328534047' entity='mgr.x' 2024-09-10T21:50:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:40 smithi090 ceph-mon[30350]: mgrmap e3: x(active, since 1.01023s) 2024-09-10T21:50:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:40 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2384890828' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T21:50:41.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-09-10T21:50:41.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:41.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T21:50:41.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:41.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T21:50:41.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T21:50:41.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T21:50:41.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T21:50:41.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T21:50:41.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T21:50:41.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:41.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T21:50:41.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T21:50:41.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T21:50:41.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T21:50:41.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T21:50:41.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T21:50:41.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T21:50:41.231 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T21:50:11:271199+0000", 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T21:50:41.232 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T21:50:11.271935+0000", 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T21:50:41.233 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-09-10T21:50:41.234 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T21:50:41.234 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-10T21:50:41.234 INFO:teuthology.orchestra.run.smithi090.stdout:mgr is available 2024-09-10T21:50:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:41 smithi090 ceph-mon[30350]: mgrmap e4: x(active, since 2s) 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T21:50:42.575 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T21:50:42.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-09-10T21:50:42.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T21:50:42.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T21:50:42.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T21:50:42.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T21:50:42.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T21:50:42.576 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T21:50:42.576 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling cephadm module... 2024-09-10T21:50:42.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:42 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3488877706' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T21:50:44.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:43 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2014273052' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-10T21:50:44.106 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:43 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setuser ceph since I am not root 2024-09-10T21:50:44.106 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:43 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setgroup ceph since I am not root 2024-09-10T21:50:45.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:44 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2014273052' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T21:50:45.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:44 smithi090 ceph-mon[30350]: mgrmap e5: x(active, since 5s) 2024-09-10T21:50:45.968 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:45 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3887393074' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T21:50:45.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-10T21:50:45.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T21:50:45.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T21:50:45.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T21:50:45.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T21:50:45.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-10T21:50:45.977 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-09-10T21:50:45.977 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 5... 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: Active manager daemon x restarted 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: Activating manager daemon x 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: mgrmap e6: x(active, starting, since 0.00354303s) 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T21:50:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:49 smithi090 ceph-mon[30350]: Manager daemon x is now available 2024-09-10T21:50:51.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-10T21:50:51.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T21:50:51.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T21:50:51.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-10T21:50:51.450 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 5 is available 2024-09-10T21:50:51.450 INFO:teuthology.orchestra.run.smithi090.stdout:Setting orchestrator backend to cephadm... 2024-09-10T21:50:52.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:51 smithi090 ceph-mon[30350]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T21:50:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:51 smithi090 ceph-mon[30350]: mgrmap e7: x(active, since 1.00748s) 2024-09-10T21:50:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:51 smithi090 ceph-mon[30350]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T21:50:53.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:50:53.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:50:53.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: Found migration_current of "None". Setting to last migration. 2024-09-10T21:50:53.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:50:53.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:50:53.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:50:53.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:50:53.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T21:50:53.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T21:50:53.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:50:53.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:50:53.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:50:53.290 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T21:50:53.168+0000 7f8989200640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:21:50:53] ENGINE Error in HTTPServer.serve 2024-09-10T21:50:53.290 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: Traceback (most recent call last): 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self._connections.run(self.expiration_interval) 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self._run(expiration_interval) 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: s = self.context.wrap_socket( 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: return self.sslsocket_class._create( 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T21:50:53.291 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self.do_handshake() 2024-09-10T21:50:53.292 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T21:50:53.292 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self._sslobj.do_handshake() 2024-09-10T21:50:53.292 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T21:50:53.292 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:54 smithi090 ceph-mon[30350]: [10/Sep/2024:21:50:53] ENGINE Bus STARTING 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:54 smithi090 ceph-mon[30350]: [10/Sep/2024:21:50:53] ENGINE Serving on https://172.21.15.90:7150 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:54 smithi090 ceph-mon[30350]: [10/Sep/2024:21:50:53] ENGINE Error in HTTPServer.serve 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._run(expiration_interval) 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T21:50:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: s = self.context.wrap_socket( 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.sslsocket_class._create( 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: self.do_handshake() 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._sslobj.do_handshake() 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:54 smithi090 ceph-mon[30350]: [10/Sep/2024:21:50:53] ENGINE Serving on http://172.21.15.90:8765 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:54 smithi090 ceph-mon[30350]: [10/Sep/2024:21:50:53] ENGINE Bus STARTED 2024-09-10T21:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:54 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:50:54.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:54 smithi090 ceph-mon[30350]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:50:54.193 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T21:50:54.193 INFO:teuthology.orchestra.run.smithi090.stdout:Generating ssh key... 2024-09-10T21:50:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-mon[30350]: mgrmap e8: x(active, since 4s) 2024-09-10T21:50:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-mon[30350]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:50:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-mon[30350]: Generating ssh key... 2024-09-10T21:50:55.356 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: Generating public/private rsa key pair. 2024-09-10T21:50:55.356 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: Your identification has been saved in /tmp/tmp1ftc327l/key 2024-09-10T21:50:55.356 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: Your public key has been saved in /tmp/tmp1ftc327l/key.pub 2024-09-10T21:50:55.356 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: The key fingerprint is: 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: SHA256:SN7cinUbcFxaFHCw2rnfVDZpLj0TPeIwQV7hUup5XvI ceph-815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: The key's randomart image is: 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: +---[RSA 3072]----+ 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: | o=*=. | 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: | .o*= | 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: | . . == . | 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: | o + *..+ o| 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: | o S *= + B+| 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: | o o +* Oo+| 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: | . . o +.E | 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: | . o. o| 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: | . . | 2024-09-10T21:50:55.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:50:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: +----[SHA256]-----+ 2024-09-10T21:50:56.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:56 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:50:56.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:56 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:50:57.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:57 smithi090 ceph-mon[30350]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:50:57.419 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZcX+QpukWrktrZ/dSRl6nbL6NsOYxrcbZAJnaSzGs9PA3Y6GA9XiGIRkkQH+6TUrXEq9kQesc0iYHDBXQrn/595iCbNc9IHPBylIgSJiFz8UHy5N2G+KS2wL6D/85BbY+Sg93INyLXkza6cHsckqgMCC03mJg5+k3TZLSz9wLwKQFuuNxxE/cXOOnIyc7ALqcdDlLUrlf25omj1bEHpix+ULD/hV5FGBd6px10wWQp/Vv9rSS5aX+tUVTfzwCgcSrGYiU8lyjogEYs/TMcmb7zt1ZjHRPTLEx20H4N35Q1KuUdIVjRvMKh5kMprPXghYZnOiP1v4UairSSwfv4B22Zkmj7DQYIUVO6cVf1soRS3alq5ezSkCiJx8khx+21gzOAFPLc0HVGsCfgt1hMoi/ephLH9K3OUVbapsH7TZWiPNVgxoIVq34p83tu/aV/lMjV1gLS4gwjcdLw5pyeSS0OkeUtsGTX4Q4zNzxix9vVSYpI7tYZ583oGfNl5GBSzk= ceph-815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:50:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T21:50:57.420 INFO:teuthology.orchestra.run.smithi090.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T21:50:57.425 INFO:teuthology.orchestra.run.smithi090.stdout:Adding host smithi090... 2024-09-10T21:50:59.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:58 smithi090 ceph-mon[30350]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:00.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:50:59 smithi090 ceph-mon[30350]: Deploying cephadm binary to smithi090 2024-09-10T21:51:02.648 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Added host 'smithi090' with addr '172.21.15.90' 2024-09-10T21:51:02.648 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mon service... 2024-09-10T21:51:03.266 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:03 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:51:03.266 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:03 smithi090 ceph-mon[30350]: Added host smithi090 2024-09-10T21:51:03.266 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:03 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:51:04.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:04 smithi090 ceph-mon[30350]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:04.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:04 smithi090 ceph-mon[30350]: Saving service mon spec with placement count:5 2024-09-10T21:51:04.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:04 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:51:05.329 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T21:51:05.329 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mgr service... 2024-09-10T21:51:05.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:05 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:51:05.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:05 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:51:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:07 smithi090 ceph-mon[30350]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:07 smithi090 ceph-mon[30350]: Saving service mgr spec with placement count:2 2024-09-10T21:51:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:07 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:51:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:07 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:51:07.398 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T21:51:09.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:09 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/60918572' entity='client.admin' 2024-09-10T21:51:09.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:09 smithi090 ceph-mon[30350]: from='mgr.14118 172.21.15.90:0/3025367505' entity='mgr.x' 2024-09-10T21:51:10.268 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling the dashboard module... 2024-09-10T21:51:10.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:10 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1909459712' entity='client.admin' 2024-09-10T21:51:10.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:10 smithi090 ceph-mon[30350]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:11.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:11 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1011175649' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-10T21:51:12.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:11 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setuser ceph since I am not root 2024-09-10T21:51:12.102 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:11 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setgroup ceph since I am not root 2024-09-10T21:51:12.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:12 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1011175649' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T21:51:12.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:12 smithi090 ceph-mon[30350]: mgrmap e9: x(active, since 21s) 2024-09-10T21:51:13.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:13 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3993071328' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T21:51:13.902 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-10T21:51:13.903 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T21:51:13.903 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T21:51:13.903 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T21:51:13.903 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T21:51:13.903 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-10T21:51:13.903 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-09-10T21:51:13.903 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 9... 2024-09-10T21:51:18.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: Active manager daemon x restarted 2024-09-10T21:51:18.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: Activating manager daemon x 2024-09-10T21:51:18.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T21:51:18.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: mgrmap e10: x(active, starting, since 0.00500249s) 2024-09-10T21:51:18.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:51:18.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T21:51:18.024 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T21:51:18.024 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T21:51:18.024 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T21:51:18.024 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:17 smithi090 ceph-mon[30350]: Manager daemon x is now available 2024-09-10T21:51:19.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:51:19.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T21:51:19.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T21:51:19.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-mon[30350]: mgrmap e11: x(active, since 1.00811s) 2024-09-10T21:51:19.023 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T21:51:18.722+0000 7f76a4000640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:21:51:18] ENGINE Error in HTTPServer.serve 2024-09-10T21:51:19.023 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: Traceback (most recent call last): 2024-09-10T21:51:19.023 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T21:51:19.023 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self._connections.run(self.expiration_interval) 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self._run(expiration_interval) 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: s = self.context.wrap_socket( 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: return self.sslsocket_class._create( 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self.do_handshake() 2024-09-10T21:51:19.024 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T21:51:19.025 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self._sslobj.do_handshake() 2024-09-10T21:51:19.025 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T21:51:19.025 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:51:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T21:51:20.049 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:19 smithi090 ceph-mon[30350]: [10/Sep/2024:21:51:18] ENGINE Bus STARTING 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:19 smithi090 ceph-mon[30350]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:19 smithi090 ceph-mon[30350]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:19 smithi090 ceph-mon[30350]: [10/Sep/2024:21:51:18] ENGINE Serving on https://172.21.15.90:7150 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:19 smithi090 ceph-mon[30350]: [10/Sep/2024:21:51:18] ENGINE Error in HTTPServer.serve 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._run(expiration_interval) 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T21:51:20.050 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: s = self.context.wrap_socket( 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.sslsocket_class._create( 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: self.do_handshake() 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._sslobj.do_handshake() 2024-09-10T21:51:20.051 INFO:journalctl@ceph.mon.a.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T21:51:20.056 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-09-10T21:51:20.056 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T21:51:20.057 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T21:51:20.057 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-09-10T21:51:20.057 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 9 is available 2024-09-10T21:51:20.057 INFO:teuthology.orchestra.run.smithi090.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-10T21:51:20.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:20 smithi090 ceph-mon[30350]: [10/Sep/2024:21:51:18] ENGINE Serving on http://172.21.15.90:8765 2024-09-10T21:51:20.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:20 smithi090 ceph-mon[30350]: [10/Sep/2024:21:51:18] ENGINE Bus STARTED 2024-09-10T21:51:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:20 smithi090 ceph-mon[30350]: mgrmap e12: x(active, since 2s) 2024-09-10T21:51:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:22.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUV9ALV0YtJLd9eNJ5J2iSXP2QY3YwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMDIxNTEyMVoXDTM0\nMDkxMTIxNTEyMVowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjkwMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAox8gb/CWBLn63t0qI8xO+Tmxscr9Y1BHmj82\nxyyawu4C5JWOatJwaiYXTQVa0JPP/fe/FNdfuyLM+zfxuOhgbLjYQ5tI/RshbceZ\nIrLmqzQWtqfPEpfTyDIgkjsHQvoVc1cltrdU9NDXmM+NRjiGx/QGEOHSeYxqc2K7\no1Ll3LjCKbYg/ftCzeMhE/MxF1SOEVyGqKJbyr7T1mS9U1g6Xad1HYcZJPbNgKeo\nZkwfyt2qvCy5r4P5DqHEzQGJBxv2br8Y38l2XsWQlVXCWYFo9Q6Xn1LN3wtQIIYj\nP8xD7knwpZCNuKpzkwk6lGepymZ/B0azzUBs2NgidYJEYa9yPuFqYY7XXylRYNpj\ngp5mnza/12lFF3bOyBCtuV1ECnjlNqemBhYqi9ldscbfvgOcAmWY+w2rwGISU/Zr\n5SM9Pjm17Qkr9X7tmxb80B/8XJUF1DDmKy7Z3qDHPL7cMfZJhnJViytBvbHbjibb\n1LF5iDlc6Izxs4WeQpIPq9vvMYOxkOL5QHO5EvtfHa+yTX+ibJrmyj+muNdkU1Jz\ny/4NR78r81SvY0sKYhnwPopt2F4Ft8XDpdGqvr9Og2limkYezRql+K2egt8Usf9C\nv+J+7RWbHXMAxd3k7Cntd17b6d6onPKHaLT1M9FA8vMg3XAZRWOQSdYxDaaB9UFy\nKuowZ6sCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDkwghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPWjAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQBU\nkRc9lMjZTYwg5g32e0ZHE/gVPTgZX8ldDtpWwBFcQiGs0x1odjqnct51ax2gwQHL\ndL1Z2C5wiHPrKc0S0KptVGjDMyMOZpjSCh/W7cW7rlKiP7dQHI8c10zSuDKLg/LY\nYO2FIadQBQ4KZI8iqqLyLpS94qrsC6rjnxTa4K3VOQP05uNrgbABZQ0eUdz/pbqd\nZLzT2sS0FAw5wSW7bDRwZ+FNgzo2hsGL+qTIm0tyWdg17RbEJcEADaGhR3x6c7VM\nHBWG0rJBeVyHR9jPoSHIP4qU+Uf8nisnuRIz9dit228mBTGYaEhMRNnogRoEg2xS\nfIUqH8DRHAw0jkj16Jq3RVLUyy3LyNmmlJiCV79S0WdtnksZN5SX2+FLEbSWlQ2b\n++l3S+JWDIZ5ueJhcGZ9ZB+RN4r9D2d/UxkP0kgzYDoG9/lovpNCVwAKo7iP6JpD\ng1CvGgRSXxFRv8doEEdugkZMXBx/0FamOSl/q7Ni22wnGY7C+47PezRYFqfSqTdH\nspgKVZ9WouUoXr/HCBAnKgudYLX5YaQ90oEkbgTHGqXpiKXxrNGVp+nNhu5Zdwye\nQUK9Hc9G6rdTvNmB2OR5w/HYTPR825nhLxNt4ynN39Uec/0KPhKAsO2hMIyl1/1v\npm4JgG1YMlj+pkwksuzJLFvNeeyPRg8p9leD/9ISKg==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAox8gb/CWBLn63t0qI8xO+Tmxscr9Y1BHmj82xyyawu4C5JWO\natJwaiYXTQVa0JPP/fe/FNdfuyLM+zfxuOhgbLjYQ5tI/RshbceZIrLmqzQWtqfP\nEpfTyDIgkjsHQvoVc1cltrdU9NDXmM+NRjiGx/QGEOHSeYxqc2K7o1Ll3LjCKbYg\n/ftCzeMhE/MxF1SOEVyGqKJbyr7T1mS9U1g6Xad1HYcZJPbNgKeoZkwfyt2qvCy5\nr4P5DqHEzQGJBxv2br8Y38l2XsWQlVXCWYFo9Q6Xn1LN3wtQIIYjP8xD7knwpZCN\nuKpzkwk6lGepymZ/B0azzUBs2NgidYJEYa9yPuFqYY7XXylRYNpjgp5mnza/12lF\nF3bOyBCtuV1ECnjlNqemBhYqi9ldscbfvgOcAmWY+w2rwGISU/Zr5SM9Pjm17Qkr\n9X7tmxb80B/8XJUF1DDmKy7Z3qDHPL7cMfZJhnJViytBvbHbjibb1LF5iDlc6Izx\ns4WeQpIPq9vvMYOxkOL5QHO5EvtfHa+yTX+ibJrmyj+muNdkU1Jzy/4NR78r81Sv\nY0sKYhnwPopt2F4Ft8XDpdGqvr9Og2limkYezRql+K2egt8Usf9Cv+J+7RWbHXMA\nxd3k7Cntd17b6d6onPKHaLT1M9FA8vMg3XAZRWOQSdYxDaaB9UFyKuowZ6sCAwEA\nAQKCAgBGIPoxlfXxJ0HnMZ9Nz6b+N+MgxFvmqBt4gYfb3l3Uwz3kJzYYz8s7n4+x\n8cTm2j13+tygKa4DBsWolZcq/2CBLYDwo+CfzFu1VjQMemx4G0dOlNuOK08GCSL+\nqzkGfvzc3xEcOjpgnC6siEK8901expcbB/MUhZ04aTCxWVDwcVAsXZxWhQDmAil0\n6b9nv9bqOOS9uDuNcTgiuqG4pLxHCljr7mqNfTvBfZMFE+IxNdaS2wmjnzHirAKv\nfmL6dveeCjM39uWYfgiYY0vQbjYA8RqXiSJwK21x5ileYsOrvQckLv727vUMOFIC\nfoS9Ow6OwsNHUVTabyV1bqFWe/a5yOFnBIacBo4QrI333nKeaFbhJcjXnpQKTF7g\nrnP3G8IeQLZp/S4yxwYY0w3ijkR6naWQ9GFyH4bu8T9WggOdQ/dCltjktICLN5e2\nj30pLSoq/sNc+bdFf6/5aSP3nLeUT0Zn7S8fZCE4E2NeNtnVio25wM01FzSSg9Iz\npphhkpGoedZMcxE4bXo+fbDwHodogz80+l4yLYrlfEE2JzsHjevOPyt+j2KzLb9j\nWl7aia3flm1RaZ8BiAN6x/gt8Sr1ayKOHecoY9QjbjY2VizkRFv1WqzDWz9y91rq\nzd+peEwjOZeFoqzfETPYx3k6cyis/ftvJSKukazWKGXyi2Y2CQKCAQEA2aPr46sS\n3FVkoV0/dQKD7tlFl1jDJajnrLLXJ7Po7Bf3Dp/Q6TAthRie74EmoWWAhsmOa72P\nmfE8FHkGnipT3E4n3LrhSIaPt6jQTN9SRJIiPzqnAISwN3+Rgwnsl6Ay/9U5ndjm\nB2giU8EfZHzWuTF/EONxwtZi2qPL2VTwm5tAxSoMstMCB6uX3L45qnsUSqZK071s\nUIUfcIKKx9JKxCuXdrMc3QtvgY8+kUmuN/7gOic+nPdF8EHU7tdKZaL2imGeObsX\nud9xGyy7FydWuttbIgxQ6cVXH2pFZGBPaFppHfi7UfTCLA9hH61djCvNzrnc9LP0\n/I+jB0xiFAjf+QKCAQEAv99IhFIPAXT2R4wHw2kyLFYA/A8klJGIMvEX1M0iHtG+\n8DO3mmE1LCuZHIi4wTp+FuetojQztSlz3eAsMlPRcQWHQI20IUaIdhyWZ1DzfM05\n5tFypPu+AVwxbCMFCjTH4UUnYkTqhExSmeZj/39ar5KoL6qROPNaPa1l+FgiRPTo\ngDPmD6Y145ApIn37Bhu6b3JILIpCig0gi+D/D/oii+6W/5ni4rO3Ondi/ArWqGfg\nhifdPKW94TLTePDds0FHV24rP0rFyoDOFAfy4ejSUI14nVt0VMR0AH8PSAEMrUmy\nFgO3IStzKqAhtR9Rv6MpQ5QVWeDmDVaSdArDd9l1wwKCAQBfM/Fv+dmaQFBwtdWY\nXa1QwgJLHizfQG93gBPV7kEpZaJQ51//YW/IFOkRY7F+UGRUkb6XKqiycnARlI9s\niJ9+c0HEsIrzqskbUeEsf1QBYttQRVvgQirDXC5O3/x6pz0n5CH8BnE5NDgv4v7U\nejeDcz9eJ+APlrBPa+8z9o/5wYQnuykq4Fq2+DzJ9vxKsuRBc1SwIuBKqLEEClai\n6j6CaX8sEVXC68mvJvX7n/txyzBSv/AHwbL6UhHPGKYWp56pEeKg5lKOoEtd08dT\nbYXm3yWSU0jhuc1OlLIeInMZ9tKcpb1rMGHv3BpkPPX65v50O6GH+WBlSn/KJYOR\nuAAhAoIBAQCO5pMxyZPXd+78tnQJu81ca+lr0TNjW0he+GTRXr4syfY1G6+A0Sky\n7n2oXMONVi0WjhIpnktsf6UF+LGyAe8ZCA8FqwQfTFLR9NSnugi+SHL5T8Dc04ld\npfZBHYvhYBIF2uVMMRug0N8sNAUCAy3dWZLdfWGPUt9A1GX/olRTmzuWUBDmUtoL\n1BBu2fkdQYWG/0x1r7uC6ef1hesxURBtu5LGZywtv7+VUdls8f2Ri1wTTpldU+up\nT5tkla/ok5O4sBqRlvTcrzXMgWcJjcclq1fLipOHlpD21arh67YvUZXLA5VepOev\nNXglV2AhJDnVVPLZ78Am6vVEa8fM+S7VAoIBACIKDNbpufvzz6v/heXbB0IUCFUT\nNwl5f+6ZZIcEiKkacD2zdaFYG2kkpwZC0abJSQn4em3/HEKb8PgfiypdPAv1FhDd\nOpa9LACReMdYbUADQQbW1eD6d1swCMvvuNpCsTdncPf4a6ORLNcLq1J6lPBUzUx0\n8cjvL0rCsOuT2YidI9oH7pEXpA/4ifZsG7TLxdlyDMmO9eYL/Bwc7MjCq7AsHZ0K\n61AumnMHqOKNQAOa7Ovdb23Cwrs911lu/tIuS603OvHvM1JL/1XqjmP3IMZjxxQS\nvS7achjUPqxvqVmV/Su34Mu7SvfxoDBKGPmp8hgVck9n9/GqoSU6qlGeq7k=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-10T21:51:22.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:22 smithi090 ceph-mon[30350]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:22.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:22 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:23.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-10T21:51:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:24 smithi090 ceph-mon[30350]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:25.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-10T21:51:25.896 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial admin user... 2024-09-10T21:51:26.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:26 smithi090 ceph-mon[30350]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:26.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:26 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:27.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:27 smithi090 ceph-mon[30350]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:27.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:27 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:27.555 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$YzJlchtIVP/RVSwuV3dWP.nqaLM5586LyOJ5fa8VAoY5Bqn2xo6c2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726005086, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T21:51:27.555 INFO:teuthology.orchestra.run.smithi090.stdout:Fetching dashboard port number... 2024-09-10T21:51:28.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:28 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4082718072' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-10T21:51:28.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T21:51:28.966 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T21:51:28.966 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-09-10T21:51:28.980 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T21:51:28.980 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-09-10T21:51:28.980 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-09-10T21:51:28.980 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T21:51:28.983 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph Dashboard is now available at: 2024-09-10T21:51:28.983 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:28.983 INFO:teuthology.orchestra.run.smithi090.stdout: URL: https://smithi090.front.sepia.ceph.com:8443/ 2024-09-10T21:51:28.983 INFO:teuthology.orchestra.run.smithi090.stdout: User: admin 2024-09-10T21:51:28.983 INFO:teuthology.orchestra.run.smithi090.stdout: Password: xnjowy97g6 2024-09-10T21:51:28.983 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:28.983 INFO:teuthology.orchestra.run.smithi090.stdout:Saving cluster configuration to /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config directory 2024-09-10T21:51:30.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T21:51:30.481 INFO:teuthology.orchestra.run.smithi090.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout: ceph telemetry on 2024-09-10T21:51:30.482 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:30.483 INFO:teuthology.orchestra.run.smithi090.stdout:For more information see: 2024-09-10T21:51:30.483 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:30.483 INFO:teuthology.orchestra.run.smithi090.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T21:51:30.483 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:30.483 INFO:teuthology.orchestra.run.smithi090.stdout:Bootstrap complete. 2024-09-10T21:51:30.541 INFO:tasks.cephadm:Fetching config... 2024-09-10T21:51:30.541 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:51:30.541 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T21:51:30.577 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T21:51:30.577 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:51:30.577 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T21:51:30.653 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T21:51:30.653 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:51:30.653 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T21:51:30.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:30 smithi090 ceph-mon[30350]: mgrmap e13: x(active, since 12s) 2024-09-10T21:51:30.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:30 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/956792695' entity='client.admin' 2024-09-10T21:51:30.750 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T21:51:30.750 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:51:30.750 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T21:51:30.826 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T21:51:30.827 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZcX+QpukWrktrZ/dSRl6nbL6NsOYxrcbZAJnaSzGs9PA3Y6GA9XiGIRkkQH+6TUrXEq9kQesc0iYHDBXQrn/595iCbNc9IHPBylIgSJiFz8UHy5N2G+KS2wL6D/85BbY+Sg93INyLXkza6cHsckqgMCC03mJg5+k3TZLSz9wLwKQFuuNxxE/cXOOnIyc7ALqcdDlLUrlf25omj1bEHpix+ULD/hV5FGBd6px10wWQp/Vv9rSS5aX+tUVTfzwCgcSrGYiU8lyjogEYs/TMcmb7zt1ZjHRPTLEx20H4N35Q1KuUdIVjRvMKh5kMprPXghYZnOiP1v4UairSSwfv4B22Zkmj7DQYIUVO6cVf1soRS3alq5ezSkCiJx8khx+21gzOAFPLc0HVGsCfgt1hMoi/ephLH9K3OUVbapsH7TZWiPNVgxoIVq34p83tu/aV/lMjV1gLS4gwjcdLw5pyeSS0OkeUtsGTX4Q4zNzxix9vVSYpI7tYZ583oGfNl5GBSzk= ceph-815bde3e-6fbe-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T21:51:30.955 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZcX+QpukWrktrZ/dSRl6nbL6NsOYxrcbZAJnaSzGs9PA3Y6GA9XiGIRkkQH+6TUrXEq9kQesc0iYHDBXQrn/595iCbNc9IHPBylIgSJiFz8UHy5N2G+KS2wL6D/85BbY+Sg93INyLXkza6cHsckqgMCC03mJg5+k3TZLSz9wLwKQFuuNxxE/cXOOnIyc7ALqcdDlLUrlf25omj1bEHpix+ULD/hV5FGBd6px10wWQp/Vv9rSS5aX+tUVTfzwCgcSrGYiU8lyjogEYs/TMcmb7zt1ZjHRPTLEx20H4N35Q1KuUdIVjRvMKh5kMprPXghYZnOiP1v4UairSSwfv4B22Zkmj7DQYIUVO6cVf1soRS3alq5ezSkCiJx8khx+21gzOAFPLc0HVGsCfgt1hMoi/ephLH9K3OUVbapsH7TZWiPNVgxoIVq34p83tu/aV/lMjV1gLS4gwjcdLw5pyeSS0OkeUtsGTX4Q4zNzxix9vVSYpI7tYZ583oGfNl5GBSzk= ceph-815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:51:30.981 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZcX+QpukWrktrZ/dSRl6nbL6NsOYxrcbZAJnaSzGs9PA3Y6GA9XiGIRkkQH+6TUrXEq9kQesc0iYHDBXQrn/595iCbNc9IHPBylIgSJiFz8UHy5N2G+KS2wL6D/85BbY+Sg93INyLXkza6cHsckqgMCC03mJg5+k3TZLSz9wLwKQFuuNxxE/cXOOnIyc7ALqcdDlLUrlf25omj1bEHpix+ULD/hV5FGBd6px10wWQp/Vv9rSS5aX+tUVTfzwCgcSrGYiU8lyjogEYs/TMcmb7zt1ZjHRPTLEx20H4N35Q1KuUdIVjRvMKh5kMprPXghYZnOiP1v4UairSSwfv4B22Zkmj7DQYIUVO6cVf1soRS3alq5ezSkCiJx8khx+21gzOAFPLc0HVGsCfgt1hMoi/ephLH9K3OUVbapsH7TZWiPNVgxoIVq34p83tu/aV/lMjV1gLS4gwjcdLw5pyeSS0OkeUtsGTX4Q4zNzxix9vVSYpI7tYZ583oGfNl5GBSzk= ceph-815bde3e-6fbe-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T21:51:31.073 INFO:teuthology.orchestra.run.smithi154.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZcX+QpukWrktrZ/dSRl6nbL6NsOYxrcbZAJnaSzGs9PA3Y6GA9XiGIRkkQH+6TUrXEq9kQesc0iYHDBXQrn/595iCbNc9IHPBylIgSJiFz8UHy5N2G+KS2wL6D/85BbY+Sg93INyLXkza6cHsckqgMCC03mJg5+k3TZLSz9wLwKQFuuNxxE/cXOOnIyc7ALqcdDlLUrlf25omj1bEHpix+ULD/hV5FGBd6px10wWQp/Vv9rSS5aX+tUVTfzwCgcSrGYiU8lyjogEYs/TMcmb7zt1ZjHRPTLEx20H4N35Q1KuUdIVjRvMKh5kMprPXghYZnOiP1v4UairSSwfv4B22Zkmj7DQYIUVO6cVf1soRS3alq5ezSkCiJx8khx+21gzOAFPLc0HVGsCfgt1hMoi/ephLH9K3OUVbapsH7TZWiPNVgxoIVq34p83tu/aV/lMjV1gLS4gwjcdLw5pyeSS0OkeUtsGTX4Q4zNzxix9vVSYpI7tYZ583oGfNl5GBSzk= ceph-815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:51:31.098 DEBUG:teuthology.orchestra.run.smithi171:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZcX+QpukWrktrZ/dSRl6nbL6NsOYxrcbZAJnaSzGs9PA3Y6GA9XiGIRkkQH+6TUrXEq9kQesc0iYHDBXQrn/595iCbNc9IHPBylIgSJiFz8UHy5N2G+KS2wL6D/85BbY+Sg93INyLXkza6cHsckqgMCC03mJg5+k3TZLSz9wLwKQFuuNxxE/cXOOnIyc7ALqcdDlLUrlf25omj1bEHpix+ULD/hV5FGBd6px10wWQp/Vv9rSS5aX+tUVTfzwCgcSrGYiU8lyjogEYs/TMcmb7zt1ZjHRPTLEx20H4N35Q1KuUdIVjRvMKh5kMprPXghYZnOiP1v4UairSSwfv4B22Zkmj7DQYIUVO6cVf1soRS3alq5ezSkCiJx8khx+21gzOAFPLc0HVGsCfgt1hMoi/ephLH9K3OUVbapsH7TZWiPNVgxoIVq34p83tu/aV/lMjV1gLS4gwjcdLw5pyeSS0OkeUtsGTX4Q4zNzxix9vVSYpI7tYZ583oGfNl5GBSzk= ceph-815bde3e-6fbe-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T21:51:31.193 INFO:teuthology.orchestra.run.smithi171.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZcX+QpukWrktrZ/dSRl6nbL6NsOYxrcbZAJnaSzGs9PA3Y6GA9XiGIRkkQH+6TUrXEq9kQesc0iYHDBXQrn/595iCbNc9IHPBylIgSJiFz8UHy5N2G+KS2wL6D/85BbY+Sg93INyLXkza6cHsckqgMCC03mJg5+k3TZLSz9wLwKQFuuNxxE/cXOOnIyc7ALqcdDlLUrlf25omj1bEHpix+ULD/hV5FGBd6px10wWQp/Vv9rSS5aX+tUVTfzwCgcSrGYiU8lyjogEYs/TMcmb7zt1ZjHRPTLEx20H4N35Q1KuUdIVjRvMKh5kMprPXghYZnOiP1v4UairSSwfv4B22Zkmj7DQYIUVO6cVf1soRS3alq5ezSkCiJx8khx+21gzOAFPLc0HVGsCfgt1hMoi/ephLH9K3OUVbapsH7TZWiPNVgxoIVq34p83tu/aV/lMjV1gLS4gwjcdLw5pyeSS0OkeUtsGTX4Q4zNzxix9vVSYpI7tYZ583oGfNl5GBSzk= ceph-815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:51:31.218 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T21:51:31.593 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:51:33.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:33 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4279347453' entity='client.admin' 2024-09-10T21:51:33.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:33 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:33.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:33 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:33.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:33 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"} : dispatch 2024-09-10T21:51:33.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:33 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:33.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:33 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:51:33.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:33 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:33.604 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T21:51:33.604 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T21:51:34.120 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:51:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:35 smithi090 ceph-mon[30350]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:35 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:35 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:51:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:35 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:51:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:35 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:51:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:35 smithi090 ceph-mon[30350]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-10T21:51:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:35 smithi090 ceph-mon[30350]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:51:36.195 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi154 2024-09-10T21:51:36.195 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:51:36.195 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.conf 2024-09-10T21:51:36.235 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:51:36.235 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:51:36.312 INFO:tasks.cephadm:Adding host smithi154 to orchestrator... 2024-09-10T21:51:36.312 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch host add smithi154 2024-09-10T21:51:36.823 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:51:36.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:36 smithi090 ceph-mon[30350]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:51:36.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:36 smithi090 ceph-mon[30350]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:51:36.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:36 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:36.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:36 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:36.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:36 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:38.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:37 smithi090 ceph-mon[30350]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi154", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:37 smithi090 ceph-mon[30350]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:41.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:40 smithi090 ceph-mon[30350]: Deploying cephadm binary to smithi154 2024-09-10T21:51:41.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:40 smithi090 ceph-mon[30350]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:41.808 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi154' with addr '172.21.15.154' 2024-09-10T21:51:42.607 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T21:51:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:42 smithi090 ceph-mon[30350]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:42 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:42 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:51:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:42 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:42.963 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:51:43.699 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:43.699 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.154", "hostname": "smithi154", "labels": [], "status": ""}] 2024-09-10T21:51:44.058 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:43 smithi090 ceph-mon[30350]: Added host smithi154 2024-09-10T21:51:44.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:43 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:44.366 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi171 2024-09-10T21:51:44.367 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:51:44.367 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.conf 2024-09-10T21:51:44.407 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:51:44.407 DEBUG:teuthology.orchestra.run.smithi171:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:51:44.484 INFO:tasks.cephadm:Adding host smithi171 to orchestrator... 2024-09-10T21:51:44.484 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch host add smithi171 2024-09-10T21:51:44.841 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:51:44.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:44 smithi090 ceph-mon[30350]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:44.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:44 smithi090 ceph-mon[30350]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:51:44.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:44 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:46 smithi090 ceph-mon[30350]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:46 smithi090 ceph-mon[30350]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:48.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:47 smithi090 ceph-mon[30350]: Deploying cephadm binary to smithi171 2024-09-10T21:51:48.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:47 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:47 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:47 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:47 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:47 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"} : dispatch 2024-09-10T21:51:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:47 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:51:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:47 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:51:49.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:48 smithi090 ceph-mon[30350]: Updating smithi154:/etc/ceph/ceph.conf 2024-09-10T21:51:49.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:48 smithi090 ceph-mon[30350]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:49.387 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi171' with addr '172.21.15.171' 2024-09-10T21:51:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:49 smithi090 ceph-mon[30350]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:51:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:49 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:50.106 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T21:51:50.464 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:51:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:50 smithi090 ceph-mon[30350]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:51:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:50 smithi090 ceph-mon[30350]: Added host smithi171 2024-09-10T21:51:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:50 smithi090 ceph-mon[30350]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:51:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:50 smithi090 ceph-mon[30350]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:50 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:50 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:50 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:50 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:51:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:50 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:51.172 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:51:51.172 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.154", "hostname": "smithi154", "labels": [], "status": ""}, {"addr": "172.21.15.171", "hostname": "smithi171", "labels": [], "status": ""}] 2024-09-10T21:51:51.868 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T21:51:51.868 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T21:51:52.227 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:51:52.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:52 smithi090 ceph-mon[30350]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:51:52.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:52 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:52.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:52 smithi090 ceph-mon[30350]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:53.375 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-09-10T21:51:53.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:53 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:53.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4000982437' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T21:51:54.101 INFO:tasks.cephadm:Adding mon.a on smithi090 2024-09-10T21:51:54.101 INFO:tasks.cephadm:Adding mon.b on smithi154 2024-09-10T21:51:54.102 INFO:tasks.cephadm:Adding mon.c on smithi171 2024-09-10T21:51:54.102 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi090:172.21.15.90=a;smithi154:172.21.15.154=b;smithi171:172.21.15.171=c' 2024-09-10T21:51:54.628 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T21:51:54.716 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T21:51:54.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:54 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4000982437' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T21:51:54.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:54 smithi090 ceph-mon[30350]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T21:51:54.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:54 smithi090 ceph-mon[30350]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:55.532 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mon update... 2024-09-10T21:51:56.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi154:172.21.15.154=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:51:56.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi154:172.21.15.154=b;smithi171:172.21.15.171=c;count:3 2024-09-10T21:51:56.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:56.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:56.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:56.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:56.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:56.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:56.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"} : dispatch 2024-09-10T21:51:56.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:51:56.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:56 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:51:57.285 DEBUG:teuthology.orchestra.run.smithi154:mon.b> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T21:51:57.290 DEBUG:teuthology.orchestra.run.smithi171:mon.c> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T21:51:57.294 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T21:51:57.294 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T21:51:57.711 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:51:57.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:57 smithi090 ceph-mon[30350]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-10T21:51:58.584 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:51:58.585 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":1,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","modified":"2024-09-10T21:50:07.568893Z","created":"2024-09-10T21:50:07.568893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T21:51:58.585 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 1 2024-09-10T21:51:58.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:58 smithi090 ceph-mon[30350]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:51:58.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:58 smithi090 ceph-mon[30350]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:51:58.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:58 smithi090 ceph-mon[30350]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:51:58.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:58 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:58.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:58 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:58.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:58 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:51:58.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:58 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:51:58.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:58 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:51:59.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:59 smithi090 ceph-mon[30350]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:51:59.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:59 smithi090 ceph-mon[30350]: Deploying daemon mon.c on smithi171 2024-09-10T21:51:59.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:51:59 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/3115186521' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:00.543 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T21:52:00.543 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T21:52:00.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:00 smithi090 ceph-mon[30350]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:00.986 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:02.021 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:52:02.021 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":1,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","modified":"2024-09-10T21:50:07.568893Z","created":"2024-09-10T21:50:07.568893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T21:52:02.021 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 1 2024-09-10T21:52:03.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:02 smithi090 ceph-mon[30350]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:02 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/1923675060' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:04.275 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T21:52:04.275 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T21:52:04.973 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:04 smithi171 podman[34092]: 2024-09-10 21:52:04.753701803 +0000 UTC m=+0.195717238 container create 6336244ad0d5abdcdd9238a440382ab7a77b11c385549d9cabc4bee403362aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-c, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T21:52:04.973 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:04 smithi171 podman[34092]: 2024-09-10 21:52:04.972425317 +0000 UTC m=+0.414440768 container init 6336244ad0d5abdcdd9238a440382ab7a77b11c385549d9cabc4bee403362aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2) 2024-09-10T21:52:05.092 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:52:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:04 smithi090 ceph-mon[30350]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:04 smithi171 podman[34092]: 2024-09-10 21:52:04.979052285 +0000 UTC m=+0.421067720 container start 6336244ad0d5abdcdd9238a440382ab7a77b11c385549d9cabc4bee403362aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:52:05.233 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pidfile_write: ignore empty --pid-file 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: load: jerasure load: lrc 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: RocksDB version: 7.9.2 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Git sha 0 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: DB SUMMARY 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: DB Session ID: 74OQE7DANH3JANK6D8B6 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: CURRENT file: CURRENT 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T21:52:05.234 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.error_if_exists: 0 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.create_if_missing: 0 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.paranoid_checks: 1 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.env: 0x563da2445180 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.info_log: 0x563ddb5936e0 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.statistics: (nil) 2024-09-10T21:52:05.235 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.use_fsync: 0 2024-09-10T21:52:05.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_log_file_size: 0 2024-09-10T21:52:05.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T21:52:05.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T21:52:05.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T21:52:05.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T21:52:05.236 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.allow_fallocate: 1 2024-09-10T21:52:05.237 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.use_direct_reads: 0 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.db_log_dir: 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.wal_dir: 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T21:52:05.238 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.write_buffer_manager: 0x563ddb597b80 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.unordered_write: 0 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T21:52:05.239 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.row_cache: None 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.wal_filter: None 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.two_write_queues: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.wal_compression: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.atomic_flush: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.log_readahead_size: 0 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T21:52:05.240 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_background_jobs: 2 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_background_compactions: -1 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_subcompactions: 1 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T21:52:05.241 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_open_files: -1 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_background_flushes: -1 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Compression algorithms supported: 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: kZSTD supported: 0 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: kXpressCompression supported: 0 2024-09-10T21:52:05.242 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: kBZip2Compression supported: 0 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: kLZ4Compression supported: 1 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: kZlibCompression supported: 1 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: kSnappyCompression supported: 1 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.merge_operator: 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_filter: None 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T21:52:05.243 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563ddb593300) 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks: 1 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: pin_top_level_index_and_filter: 1 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_type: 0 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_index_type: 0 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_shortening: 1 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: checksum: 4 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: no_block_cache: 0 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache: 0x563ddb587610 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_name: BinnedLRUCache 2024-09-10T21:52:05.244 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_options: 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: capacity : 536870912 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_shard_bits : 4 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: strict_capacity_limit : 0 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: high_pri_pool_ratio: 0.000 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_cache_compressed: (nil) 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: persistent_cache: (nil) 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size: 4096 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_size_deviation: 10 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_restart_interval: 16 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: index_block_restart_interval: 1 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: metadata_block_size: 4096 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: partition_filters: 0 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: use_delta_encoding: 1 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: filter_policy: bloomfilter 2024-09-10T21:52:05.245 INFO:journalctl@ceph.mon.c.smithi171.stdout: whole_key_filtering: 1 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout: verify_compression: 0 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout: read_amp_bytes_per_bit: 0 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout: format_version: 5 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_index_compression: 1 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout: block_align: 0 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout: max_auto_readahead_size: 262144 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout: prepopulate_block_cache: 0 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout: initial_auto_readahead_size: 8192 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression: NoCompression 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T21:52:05.246 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.num_levels: 7 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T21:52:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T21:52:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T21:52:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.inplace_update_support: 0 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.bloom_locality: 0 2024-09-10T21:52:05.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.max_successive_merges: 0 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.ttl: 2592000 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.enable_blob_files: false 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.min_blob_size: 0 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T21:52:05.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c9ff6612-79b2-4758-a7a8-901c2c6df2dd 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726005125022433, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726005125023190, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726005125, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c9ff6612-79b2-4758-a7a8-901c2c6df2dd", "db_session_id": "74OQE7DANH3JANK6D8B6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726005125023471, "job": 1, "event": "recovery_finished"} 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T21:52:05.252 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563ddb5b8e00 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: DB pointer 0x563ddb6b8000 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** DB Stats ** 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-09-10T21:52:05.253 INFO:journalctl@ceph.mon.c.smithi171.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** Compaction Stats [default] ** 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T21:52:05.254 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: Cumulative compaction: 0.00 GB write, 0.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: Interval compaction: 0.00 GB write, 0.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache BinnedLRUCache@0x563ddb587610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: using public_addr v2:172.21.15.171:0/0 -> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: starting mon.c rank -1 at public addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] at bind addrs [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:05.255 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(???) e0 preinit fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout: e1 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout: btime 2024-09-10T21:50:11:271199+0000 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout: legacy client fscid: -1 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout: 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout: No filesystems configured 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T21:52:05.256 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Deploying cephadm binary to smithi171 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.257 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Updating smithi154:/etc/ceph/ceph.conf 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Added host smithi171 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:05.258 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4000982437' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4000982437' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi154:172.21.15.154=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi154:172.21.15.154=b;smithi171:172.21.15.171=c;count:3 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.259 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.260 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: Deploying daemon mon.c on smithi171 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/3115186521' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/1923675060' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 ceph-mon[34138]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 bash[34092]: 6336244ad0d5abdcdd9238a440382ab7a77b11c385549d9cabc4bee403362aa9 2024-09-10T21:52:05.261 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:05 smithi171 systemd[1]: Started Ceph mon.c for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:52:10.371 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:52:10.371 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","modified":"2024-09-10T21:52:05.063495Z","created":"2024-09-10T21:50:07.568893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T21:52:10.371 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-09-10T21:52:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:52:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: mon.a calling monitor election 2024-09-10T21:52:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: mon.c calling monitor election 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: monmap epoch 2 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: last_changed 2024-09-10T21:52:05.063495+0000 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: created 2024-09-10T21:50:07.568893+0000 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: min_mon_release 19 (squid) 2024-09-10T21:52:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: election_strategy: 1 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: fsmap 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: mgrmap e13: x(active, since 52s) 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: overall HEALTH_OK 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:10.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:10 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:10.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:52:10.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: mon.a calling monitor election 2024-09-10T21:52:10.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:10.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: mon.c calling monitor election 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: monmap epoch 2 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: last_changed 2024-09-10T21:52:05.063495+0000 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: created 2024-09-10T21:50:07.568893+0000 2024-09-10T21:52:10.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: min_mon_release 19 (squid) 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: election_strategy: 1 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: fsmap 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: mgrmap e13: x(active, since 52s) 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:10 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:11.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:11 smithi171 ceph-mon[34138]: Deploying daemon mon.b on smithi154 2024-09-10T21:52:11.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:11 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/3424933785' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:11.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:11 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:11.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:11 smithi090 ceph-mon[30350]: Deploying daemon mon.b on smithi154 2024-09-10T21:52:11.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:11 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/3424933785' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:11.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:11 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:12.109 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T21:52:12.110 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T21:52:12.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:12 smithi090 ceph-mon[30350]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:12.356 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:12 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T21:52:12.063+0000 7f76e4600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T21:52:12.473 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:52:12.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:12 smithi171 ceph-mon[34138]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:13.256 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:52:13.256 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":2,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","modified":"2024-09-10T21:52:05.063495Z","created":"2024-09-10T21:50:07.568893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T21:52:13.256 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 2 2024-09-10T21:52:13.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:13 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/270807461' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:13.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:13 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/270807461' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:14 smithi090 ceph-mon[30350]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:14.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:14 smithi171 ceph-mon[34138]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:14.932 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T21:52:14.933 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T21:52:15.083 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 podman[34475]: 2024-09-10 21:52:15.047922082 +0000 UTC m=+0.363677382 container start 670bf7b33c22dbffd812e48187972e765bd1a7d0c6c8b75c8ca0b0d07d6acaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T21:52:15.298 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pidfile_write: ignore empty --pid-file 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: load: jerasure load: lrc 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: RocksDB version: 7.9.2 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Git sha 0 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: DB SUMMARY 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: DB Session ID: ATG04FTZ4Y3GIJ15REFM 2024-09-10T21:52:15.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: CURRENT file: CURRENT 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.error_if_exists: 0 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.create_if_missing: 0 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.paranoid_checks: 1 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.env: 0x55b60eacd180 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T21:52:15.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.info_log: 0x55b6285a56e0 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.statistics: (nil) 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.use_fsync: 0 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_log_file_size: 0 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.allow_fallocate: 1 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.use_direct_reads: 0 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T21:52:15.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T21:52:15.421 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.db_log_dir: 2024-09-10T21:52:15.421 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.wal_dir: 2024-09-10T21:52:15.422 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T21:52:15.424 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T21:52:15.424 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T21:52:15.424 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T21:52:15.424 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T21:52:15.424 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.write_buffer_manager: 0x55b6285a9b80 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T21:52:15.425 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.unordered_write: 0 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.row_cache: None 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.wal_filter: None 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.two_write_queues: 0 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T21:52:15.426 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.wal_compression: 0 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.atomic_flush: 0 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.log_readahead_size: 0 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T21:52:15.427 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_background_jobs: 2 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_background_compactions: -1 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_subcompactions: 1 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T21:52:15.428 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_open_files: -1 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_background_flushes: -1 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Compression algorithms supported: 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: kZSTD supported: 0 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: kXpressCompression supported: 0 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: kBZip2Compression supported: 0 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: kLZ4Compression supported: 1 2024-09-10T21:52:15.429 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: kZlibCompression supported: 1 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: kSnappyCompression supported: 1 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.merge_operator: 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_filter: None 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b6285a5300) 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout: cache_index_and_filter_blocks: 1 2024-09-10T21:52:15.430 INFO:journalctl@ceph.mon.b.smithi154.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: pin_top_level_index_and_filter: 1 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: index_type: 0 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: data_block_index_type: 0 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: index_shortening: 1 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: checksum: 4 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: no_block_cache: 0 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: block_cache: 0x55b628599610 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: block_cache_name: BinnedLRUCache 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: block_cache_options: 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: capacity : 536870912 2024-09-10T21:52:15.431 INFO:journalctl@ceph.mon.b.smithi154.stdout: num_shard_bits : 4 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: strict_capacity_limit : 0 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: high_pri_pool_ratio: 0.000 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: block_cache_compressed: (nil) 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: persistent_cache: (nil) 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: block_size: 4096 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: block_size_deviation: 10 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: block_restart_interval: 16 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: index_block_restart_interval: 1 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: metadata_block_size: 4096 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: partition_filters: 0 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: use_delta_encoding: 1 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: filter_policy: bloomfilter 2024-09-10T21:52:15.432 INFO:journalctl@ceph.mon.b.smithi154.stdout: whole_key_filtering: 1 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout: verify_compression: 0 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout: read_amp_bytes_per_bit: 0 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout: format_version: 5 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout: enable_index_compression: 1 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout: block_align: 0 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout: max_auto_readahead_size: 262144 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout: prepopulate_block_cache: 0 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout: initial_auto_readahead_size: 8192 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression: NoCompression 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T21:52:15.433 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.num_levels: 7 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T21:52:15.434 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T21:52:15.435 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T21:52:15.436 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T21:52:15.437 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.inplace_update_support: 0 2024-09-10T21:52:15.438 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.bloom_locality: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.max_successive_merges: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.ttl: 2592000 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T21:52:15.439 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.enable_blob_files: false 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.min_blob_size: 0 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ef178bb8-89d0-468d-8311-c41973e7dd38 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726005135093185, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T21:52:15.440 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726005135094255, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726005135, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ef178bb8-89d0-468d-8311-c41973e7dd38", "db_session_id": "ATG04FTZ4Y3GIJ15REFM", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726005135094648, "job": 1, "event": "recovery_finished"} 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b6285cae00 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: DB pointer 0x55b6286ca000 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout: ** DB Stats ** 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T21:52:15.441 INFO:journalctl@ceph.mon.b.smithi154.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: ** Compaction Stats [default] ** 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: ** Compaction Stats [default] ** 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T21:52:15.442 INFO:journalctl@ceph.mon.b.smithi154.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: Cumulative compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: Interval compaction: 0.00 GB write, 0.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T21:52:15.443 INFO:journalctl@ceph.mon.b.smithi154.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout: Block cache BinnedLRUCache@0x55b628599610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout: 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: using public_addr v2:172.21.15.154:0/0 -> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: starting mon.b rank -1 at public addrs [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] at bind addrs [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(???) e0 preinit fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout: e1 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout: btime 2024-09-10T21:50:11:271199+0000 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T21:52:15.444 INFO:journalctl@ceph.mon.b.smithi154.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout: legacy client fscid: -1 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout: 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout: No filesystems configured 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T21:52:15.445 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T21:52:15.446 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T21:52:15.446 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T21:52:15.446 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T21:52:15.446 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.446 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi171", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Deploying cephadm binary to smithi171 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Updating smithi154:/etc/ceph/ceph.conf 2024-09-10T21:52:15.447 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Added host smithi171 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.448 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4000982437' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4000982437' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi154:172.21.15.154=b;smithi171:172.21.15.171=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi154:172.21.15.154=b;smithi171:172.21.15.171=c;count:3 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.449 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:15.450 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Deploying daemon mon.c on smithi171 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/3115186521' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/1923675060' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.a calling monitor election 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:15.451 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.c calling monitor election 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: monmap epoch 2 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: last_changed 2024-09-10T21:52:05.063495+0000 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: created 2024-09-10T21:50:07.568893+0000 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: min_mon_release 19 (squid) 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: election_strategy: 1 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-09-10T21:52:15.452 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: fsmap 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mgrmap e13: x(active, since 52s) 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: overall HEALTH_OK 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: Deploying daemon mon.b on smithi154 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/3424933785' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:15.453 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.454 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/270807461' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:15.454 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:15.454 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 ceph-mon[34495]: mon.b@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-09-10T21:52:15.454 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 bash[34475]: 670bf7b33c22dbffd812e48187972e765bd1a7d0c6c8b75c8ca0b0d07d6acaa2 2024-09-10T21:52:15.454 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:15 smithi154 systemd[1]: Started Ceph mon.b for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:52:20.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: mon.a calling monitor election 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: mon.c calling monitor election 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: mon.b calling monitor election 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: monmap epoch 3 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: last_changed 2024-09-10T21:52:15.136564+0000 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: created 2024-09-10T21:50:07.568893+0000 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: min_mon_release 19 (squid) 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: election_strategy: 1 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: 2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: fsmap 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: mgrmap e13: x(active, since 62s) 2024-09-10T21:52:20.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: overall HEALTH_OK 2024-09-10T21:52:20.500 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.500 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.500 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.500 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:20 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.525 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:52:20.525 INFO:teuthology.orchestra.run.smithi171.stdout:{"epoch":3,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","modified":"2024-09-10T21:52:15.136564Z","created":"2024-09-10T21:50:07.568893Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:3300","nonce":0},{"type":"v1","addr":"172.21.15.171:6789","nonce":0}]},"addr":"172.21.15.171:6789/0","public_addr":"172.21.15.171:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T21:52:20.525 INFO:teuthology.orchestra.run.smithi171.stderr:dumped monmap epoch 3 2024-09-10T21:52:20.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:52:20.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: mon.a calling monitor election 2024-09-10T21:52:20.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:20.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: mon.c calling monitor election 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: mon.b calling monitor election 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: monmap epoch 3 2024-09-10T21:52:20.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: last_changed 2024-09-10T21:52:15.136564+0000 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: created 2024-09-10T21:50:07.568893+0000 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: min_mon_release 19 (squid) 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: election_strategy: 1 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: 2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: fsmap 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: mgrmap e13: x(active, since 62s) 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:20 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:52:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: mon.a calling monitor election 2024-09-10T21:52:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: mon.c calling monitor election 2024-09-10T21:52:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: mon.b calling monitor election 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: monmap epoch 3 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: last_changed 2024-09-10T21:52:15.136564+0000 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: created 2024-09-10T21:50:07.568893+0000 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: min_mon_release 19 (squid) 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: election_strategy: 1 2024-09-10T21:52:20.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: 1: [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] mon.c 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: 2: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.b 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: fsmap 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: mgrmap e13: x(active, since 62s) 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: overall HEALTH_OK 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:20.670 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:20 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:21.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:21 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:21.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:21 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/988590910' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:21.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:21 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:21.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:21 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:21.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:21 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/988590910' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:21.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:21 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:21.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:21 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:21.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:21 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/988590910' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T21:52:21.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:21 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:21.868 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T21:52:21.868 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T21:52:22.238 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:22 smithi171 ceph-mon[34138]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:22 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:22 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:23.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:22 smithi090 ceph-mon[30350]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:23.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:22 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:23.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:22 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:23.026 INFO:teuthology.orchestra.run.smithi090.stdout:# minimal ceph.conf for 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:23.026 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-09-10T21:52:23.026 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T21:52:23.026 INFO:teuthology.orchestra.run.smithi090.stdout: mon_host = [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] 2024-09-10T21:52:23.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:22 smithi154 ceph-mon[34495]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:23.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:22 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:23.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:22 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:23.903 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T21:52:23.903 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:52:23.903 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T21:52:23.961 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:52:23.961 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:24.000 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:24.000 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: Updating smithi154:/etc/ceph/ceph.conf 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/651262752' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:24.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:23 smithi171 ceph-mon[34138]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:24.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:24.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:24.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:24.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:24.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-10T21:52:24.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: Updating smithi154:/etc/ceph/ceph.conf 2024-09-10T21:52:24.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-10T21:52:24.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:24.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/651262752' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:24.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:24.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:24.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:24.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:23 smithi090 ceph-mon[30350]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:24.060 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:52:24.060 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T21:52:24.109 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:24.109 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:24.109 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:24.109 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:24.109 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-10T21:52:24.110 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: Updating smithi154:/etc/ceph/ceph.conf 2024-09-10T21:52:24.110 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-10T21:52:24.110 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:24.110 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/651262752' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:24.110 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:24.110 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:24.110 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:24.110 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:23 smithi154 ceph-mon[34495]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:24.118 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:52:24.118 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:24.214 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:52:24.214 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T21:52:24.272 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:52:24.272 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:24.367 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-10T21:52:24.367 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-10T21:52:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T21:52:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:24 smithi090 ceph-mon[30350]: Reconfiguring daemon mon.a on smithi090 2024-09-10T21:52:25.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T21:52:25.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:25.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:25.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:25.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:24 smithi154 ceph-mon[34495]: Reconfiguring daemon mon.a on smithi090 2024-09-10T21:52:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:25.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:25.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:24 smithi171 ceph-mon[34138]: Reconfiguring daemon mon.a on smithi090 2024-09-10T21:52:25.489 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:26.789 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:26 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:26.789 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:26 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:26.789 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:26 smithi154 ceph-mon[34495]: Reconfiguring mon.b (monmap changed)... 2024-09-10T21:52:26.789 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:26 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:26.789 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:26 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:26.790 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:26 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:26.790 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:26 smithi154 ceph-mon[34495]: Reconfiguring daemon mon.b on smithi154 2024-09-10T21:52:26.790 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:26 smithi154 ceph-mon[34495]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:26 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:26 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:26 smithi090 ceph-mon[30350]: Reconfiguring mon.b (monmap changed)... 2024-09-10T21:52:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:26 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:26 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:26 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:26 smithi090 ceph-mon[30350]: Reconfiguring daemon mon.b on smithi154 2024-09-10T21:52:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:26 smithi090 ceph-mon[30350]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:26 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:26 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:26 smithi171 ceph-mon[34138]: Reconfiguring mon.b (monmap changed)... 2024-09-10T21:52:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:26 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:26 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:26 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:26 smithi171 ceph-mon[34138]: Reconfiguring daemon mon.b on smithi154 2024-09-10T21:52:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:26 smithi171 ceph-mon[34138]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:27.848 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:27 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2070008600' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-10T21:52:27.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:27 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:27.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:27 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:27.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:27 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:27.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:27 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:27.849 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:27 smithi171 ceph-mon[34138]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:27.855 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:27 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setuser ceph since I am not root 2024-09-10T21:52:27.856 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:27 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setgroup ceph since I am not root 2024-09-10T21:52:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:27 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2070008600' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-10T21:52:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:27 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:27 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:27 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:27 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:27 smithi090 ceph-mon[30350]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:27.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:27 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2070008600' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-10T21:52:27.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:27 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:27.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:27 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' 2024-09-10T21:52:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:27 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:27 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:27 smithi154 ceph-mon[34495]: from='mgr.14150 172.21.15.90:0/1892679431' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:28.273 INFO:tasks.cephadm:Adding mgr.x on smithi090 2024-09-10T21:52:28.273 INFO:tasks.cephadm:Adding mgr.y on smithi154 2024-09-10T21:52:28.273 INFO:tasks.cephadm:Adding mgr.z on smithi171 2024-09-10T21:52:28.273 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi090=x;smithi154=y;smithi171=z' 2024-09-10T21:52:28.835 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:28 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2070008600' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T21:52:28.835 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:28 smithi171 ceph-mon[34138]: mgrmap e14: x(active, since 69s) 2024-09-10T21:52:28.845 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:52:28.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:28 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2070008600' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T21:52:28.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:28 smithi090 ceph-mon[30350]: mgrmap e14: x(active, since 69s) 2024-09-10T21:52:28.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:28 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2070008600' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-10T21:52:28.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:28 smithi154 ceph-mon[34495]: mgrmap e14: x(active, since 69s) 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: Active manager daemon x restarted 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: Activating manager daemon x 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: mgrmap e15: x(active, starting, since 0.00513376s) 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T21:52:33.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:33 smithi090 ceph-mon[30350]: Manager daemon x is now available 2024-09-10T21:52:33.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: Active manager daemon x restarted 2024-09-10T21:52:33.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: Activating manager daemon x 2024-09-10T21:52:33.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T21:52:33.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: mgrmap e15: x(active, starting, since 0.00513376s) 2024-09-10T21:52:33.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:52:33.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:33.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:33.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T21:52:33.896 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T21:52:33.897 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T21:52:33.897 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T21:52:33.897 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:33 smithi171 ceph-mon[34138]: Manager daemon x is now available 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: Active manager daemon x restarted 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: Activating manager daemon x 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: mgrmap e15: x(active, starting, since 0.00513376s) 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T21:52:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T21:52:33.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T21:52:33.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:33 smithi154 ceph-mon[34495]: Manager daemon x is now available 2024-09-10T21:52:34.500 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mgr update... 2024-09-10T21:52:34.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:34.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T21:52:34.836 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T21:52:34.836 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-mon[30350]: mgrmap e16: x(active, since 1.01297s) 2024-09-10T21:52:34.836 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:34.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:34.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T21:52:34.850 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T21:52:34.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:34 smithi171 ceph-mon[34138]: mgrmap e16: x(active, since 1.01297s) 2024-09-10T21:52:34.851 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:34.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:34.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T21:52:34.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T21:52:34.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:34 smithi154 ceph-mon[34495]: mgrmap e16: x(active, since 1.01297s) 2024-09-10T21:52:34.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:35.106 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T21:52:34.834+0000 7fb7a4400640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:21:52:34] ENGINE Error in HTTPServer.serve 2024-09-10T21:52:35.106 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: Traceback (most recent call last): 2024-09-10T21:52:35.106 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T21:52:35.106 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self._connections.run(self.expiration_interval) 2024-09-10T21:52:35.106 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self._run(expiration_interval) 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: s = self.context.wrap_socket( 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: return self.sslsocket_class._create( 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self.do_handshake() 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self._sslobj.do_handshake() 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T21:52:35.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 21:52:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T21:52:35.198 DEBUG:teuthology.orchestra.run.smithi154:mgr.y> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T21:52:35.202 DEBUG:teuthology.orchestra.run.smithi171:mgr.z> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T21:52:35.206 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-10T21:52:35.206 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-10T21:52:35.210 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-10T21:52:35.210 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set client debug_client 20 2024-09-10T21:52:35.255 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-10T21:52:35.255 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set client debug_ms 1 2024-09-10T21:52:35.299 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-10T21:52:35.299 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-10T21:52:35.346 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-10T21:52:35.346 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-10T21:52:35.393 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-10T21:52:35.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-10T21:52:35.440 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-10T21:52:35.440 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-10T21:52:35.488 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-10T21:52:35.489 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-10T21:52:35.543 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-10T21:52:35.543 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-10T21:52:35.591 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-10T21:52:35.591 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-09-10T21:52:35.648 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-10T21:52:35.649 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-09-10T21:52:35.705 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-10T21:52:35.705 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-09-10T21:52:35.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:35 smithi171 ceph-mon[34138]: Saving service mgr spec with placement smithi090=x;smithi154=y;smithi171=z;count:3 2024-09-10T21:52:35.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:35.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:35.753 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-10T21:52:35.753 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-10T21:52:35.804 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-10T21:52:35.804 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-10T21:52:35.810 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:35 smithi090 ceph-mon[30350]: Saving service mgr spec with placement smithi090=x;smithi154=y;smithi171=z;count:3 2024-09-10T21:52:35.810 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:35.810 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:35.823 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-10T21:52:35.823 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-10T21:52:35.863 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:35.880 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-10T21:52:35.880 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-10T21:52:35.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:35 smithi154 ceph-mon[34495]: Saving service mgr spec with placement smithi090=x;smithi154=y;smithi171=z;count:3 2024-09-10T21:52:35.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:35.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:35.939 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-10T21:52:35.940 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-10T21:52:35.998 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T21:52:35.999 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-10T21:52:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:36.027 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T21:52:36.027 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-10T21:52:36.096 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-10T21:52:36.096 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-09-10T21:52:36.156 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-10T21:52:36.156 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-10T21:52:36.205 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-10T21:52:36.206 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-10T21:52:36.272 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-10T21:52:36.272 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-10T21:52:36.323 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-10T21:52:36.323 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-10T21:52:36.392 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T21:52:36.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T21:52:36.454 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-10T21:52:36.455 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-10T21:52:36.527 DEBUG:tasks.cephadm:set 26 configs 2024-09-10T21:52:36.527 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi090') 2024-09-10T21:52:36.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: [10/Sep/2024:21:52:34] ENGINE Bus STARTING 2024-09-10T21:52:36.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: [10/Sep/2024:21:52:34] ENGINE Serving on https://172.21.15.90:7150 2024-09-10T21:52:36.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: [10/Sep/2024:21:52:34] ENGINE Error in HTTPServer.serve 2024-09-10T21:52:36.567 INFO:journalctl@ceph.mon.a.smithi090.stdout: Traceback (most recent call last): 2024-09-10T21:52:36.567 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._connections.run(self.expiration_interval) 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._run(expiration_interval) 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: s = self.context.wrap_socket( 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: return self.sslsocket_class._create( 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: self.do_handshake() 2024-09-10T21:52:36.568 INFO:journalctl@ceph.mon.a.smithi090.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T21:52:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout: self._sslobj.do_handshake() 2024-09-10T21:52:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T21:52:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: [10/Sep/2024:21:52:34] ENGINE Serving on http://172.21.15.90:8765 2024-09-10T21:52:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: [10/Sep/2024:21:52:34] ENGINE Bus STARTED 2024-09-10T21:52:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: mgrmap e17: x(active, since 2s) 2024-09-10T21:52:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:36 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: [10/Sep/2024:21:52:34] ENGINE Bus STARTING 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: [10/Sep/2024:21:52:34] ENGINE Serving on https://172.21.15.90:7150 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: [10/Sep/2024:21:52:34] ENGINE Error in HTTPServer.serve 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout: Traceback (most recent call last): 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout: self._connections.run(self.expiration_interval) 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout: self._run(expiration_interval) 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T21:52:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: s = self.context.wrap_socket( 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: return self.sslsocket_class._create( 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: self.do_handshake() 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: self._sslobj.do_handshake() 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: [10/Sep/2024:21:52:34] ENGINE Serving on http://172.21.15.90:8765 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: [10/Sep/2024:21:52:34] ENGINE Bus STARTED 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: mgrmap e17: x(active, since 2s) 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:36 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: [10/Sep/2024:21:52:34] ENGINE Bus STARTING 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: [10/Sep/2024:21:52:34] ENGINE Serving on https://172.21.15.90:7150 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: [10/Sep/2024:21:52:34] ENGINE Error in HTTPServer.serve 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout: Traceback (most recent call last): 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout: self._connections.run(self.expiration_interval) 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout: self._run(expiration_interval) 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T21:52:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: s = self.context.wrap_socket( 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: return self.sslsocket_class._create( 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: self.do_handshake() 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: self._sslobj.do_handshake() 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: [10/Sep/2024:21:52:34] ENGINE Serving on http://172.21.15.90:8765 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: [10/Sep/2024:21:52:34] ENGINE Bus STARTED 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: mgrmap e17: x(active, since 2s) 2024-09-10T21:52:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:36.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:36 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:37.715 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:37.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:37.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:37.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:37.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:37.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:37.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:37.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:37 smithi090 ceph-mon[30350]: from='client.? ' entity='client.admin' 2024-09-10T21:52:37.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:37 smithi090 ceph-mon[30350]: from='client.? ' entity='client.admin' 2024-09-10T21:52:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:37 smithi171 ceph-mon[34138]: from='client.? ' entity='client.admin' 2024-09-10T21:52:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:37 smithi171 ceph-mon[34138]: from='client.? ' entity='client.admin' 2024-09-10T21:52:38.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:38.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:38.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi154", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:38.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:38.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:38.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi171", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:38.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:37 smithi154 ceph-mon[34495]: from='client.? ' entity='client.admin' 2024-09-10T21:52:38.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:37 smithi154 ceph-mon[34495]: from='client.? ' entity='client.admin' 2024-09-10T21:52:39.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:39.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:39.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:39.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:39.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:39.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:39 smithi090 ceph-mon[30350]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-10T21:52:39.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:39 smithi090 ceph-mon[30350]: Updating smithi154:/etc/ceph/ceph.conf 2024-09-10T21:52:39.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:39 smithi090 ceph-mon[30350]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-10T21:52:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:39 smithi171 ceph-mon[34138]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-10T21:52:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:39 smithi171 ceph-mon[34138]: Updating smithi154:/etc/ceph/ceph.conf 2024-09-10T21:52:39.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:39 smithi171 ceph-mon[34138]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-10T21:52:39.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 -- 172.21.15.90:0/2776313480 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f62b803f610 msgr2=0x7f62b8041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:39.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 --2- 172.21.15.90:0/2776313480 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f62b803f610 0x7f62b8041ad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009df0 tx=0x7f62c4005f10 comp rx=0 tx=0).stop 2024-09-10T21:52:39.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 -- 172.21.15.90:0/2776313480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d41039f0 msgr2=0x7f62d419b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:39.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 --2- 172.21.15.90:0/2776313480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d41039f0 0x7f62d419b0e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f62c0004f50 tx=0x7f62c0005080 comp rx=0 tx=0).stop 2024-09-10T21:52:39.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 -- 172.21.15.90:0/2776313480 shutdown_connections 2024-09-10T21:52:39.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 --2- 172.21.15.90:0/2776313480 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f62b803f610 0x7f62b8041ad0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:39.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 --2- 172.21.15.90:0/2776313480 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62d41043b0 0x7f62d419bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 --2- 172.21.15.90:0/2776313480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f62d41039f0 0x7f62d419b0e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 --2- 172.21.15.90:0/2776313480 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62d4102890 0x7f62d419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 -- 172.21.15.90:0/2776313480 >> 172.21.15.90:0/2776313480 conn(0x7f62d40fe180 msgr2=0x7f62d4075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 -- 172.21.15.90:0/2776313480 shutdown_connections 2024-09-10T21:52:39.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:39.785+0000 7f62d3e00640 1 -- 172.21.15.90:0/2776313480 wait complete. 2024-09-10T21:52:39.868 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi090') 2024-09-10T21:52:39.868 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi090') 2024-09-10T21:52:39.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:39.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:39.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"} : dispatch 2024-09-10T21:52:39.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:39 smithi154 ceph-mon[34495]: Updating smithi090:/etc/ceph/ceph.conf 2024-09-10T21:52:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:39 smithi154 ceph-mon[34495]: Updating smithi154:/etc/ceph/ceph.conf 2024-09-10T21:52:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:39 smithi154 ceph-mon[34495]: Updating smithi171:/etc/ceph/ceph.conf 2024-09-10T21:52:40.575 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:40.575 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:40.575 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:40.575 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:40.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:40.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:40.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2776313480' entity='client.admin' 2024-09-10T21:52:40.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.612 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi090') 2024-09-10T21:52:40.632 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.632 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.632 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.657 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.657 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.657 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.658 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.664 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.671 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.671 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.679 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.704 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.704 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.704 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.714 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.719 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.736 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2776313480' entity='client.admin' 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T21:52:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: Updating smithi171:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: Updating smithi154:/etc/ceph/ceph.client.admin.keyring 2024-09-10T21:52:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2776313480' entity='client.admin' 2024-09-10T21:52:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:40.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:41.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:41.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:41.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:41.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:41.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:41.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T21:52:41.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T21:52:41.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T21:52:41.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:41.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:41 smithi090 ceph-mon[30350]: Deploying daemon mgr.z on smithi171 2024-09-10T21:52:41.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:41.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:41.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:41.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:41.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:41.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T21:52:41.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T21:52:41.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T21:52:41.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:41.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:41 smithi171 ceph-mon[34138]: Deploying daemon mgr.z on smithi171 2024-09-10T21:52:41.793 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: Updating smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:41.793 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: Updating smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:41.793 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: Updating smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T21:52:41.793 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:41.793 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:41.793 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T21:52:41.793 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T21:52:41.794 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T21:52:41.794 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:41.794 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:41 smithi154 ceph-mon[34495]: Deploying daemon mgr.z on smithi171 2024-09-10T21:52:42.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.873+0000 7f5cfa000640 1 -- 172.21.15.90:0/3494254811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 msgr2=0x7f5cf41132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:42.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.873+0000 7f5cfa000640 1 --2- 172.21.15.90:0/3494254811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 0x7f5cf41132a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0009900 tx=0x7f5ce002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.873+0000 7f5cfa000640 1 -- 172.21.15.90:0/3494254811 shutdown_connections 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.873+0000 7f5cfa000640 1 --2- 172.21.15.90:0/3494254811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf41137e0 0x7f5cf4115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.873+0000 7f5cfa000640 1 --2- 172.21.15.90:0/3494254811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 0x7f5cf41132a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.873+0000 7f5cfa000640 1 --2- 172.21.15.90:0/3494254811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cf40776d0 0x7f5cf4077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.873+0000 7f5cfa000640 1 -- 172.21.15.90:0/3494254811 >> 172.21.15.90:0/3494254811 conn(0x7f5cf4100840 msgr2=0x7f5cf4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.874+0000 7f5cfa000640 1 -- 172.21.15.90:0/3494254811 shutdown_connections 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.874+0000 7f5cfa000640 1 -- 172.21.15.90:0/3494254811 wait complete. 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cfa000640 1 Processor -- start 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cfa000640 1 -- start start 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cfa000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf40776d0 0x7f5cf419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cfa000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 0x7f5cf419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cfa000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cf41137e0 0x7f5cf41a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:42.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cfa000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf4118e60 con 0x7f5cf4077ff0 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cfa000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf4118ce0 con 0x7f5cf40776d0 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cfa000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf4118fe0 con 0x7f5cf41137e0 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cf3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 0x7f5cf419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cf8c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf40776d0 0x7f5cf419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cf3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 0x7f5cf419f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60376/0 (socket says 172.21.15.90:60376) 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cf3e00640 1 -- 172.21.15.90:0/4263917678 learned_addr learned my addr 172.21.15.90:0/4263917678 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.875+0000 7f5cf9600640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cf41137e0 0x7f5cf41a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cf3e00640 1 -- 172.21.15.90:0/4263917678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cf41137e0 msgr2=0x7f5cf41a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cf3e00640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cf41137e0 0x7f5cf41a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cf3e00640 1 -- 172.21.15.90:0/4263917678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf40776d0 msgr2=0x7f5cf419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cf3e00640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf40776d0 0x7f5cf419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cf3e00640 1 -- 172.21.15.90:0/4263917678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf41a5440 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cf3e00640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 0x7f5cf419f8d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005ab0 tx=0x7f5ce0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cf1600640 1 -- 172.21.15.90:0/4263917678 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce003f070 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cf1600640 1 -- 172.21.15.90:0/4263917678 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f5ce003bcc0 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf41a56d0 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.876+0000 7f5cf1600640 1 -- 172.21.15.90:0/4263917678 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0035030 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.877+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf41abdf0 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.878+0000 7f5cf1600640 1 -- 172.21.15.90:0/4263917678 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f5ce003b870 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.878+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf4077ab0 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.878+0000 7f5cf1600640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cdc03f610 0x7f5cdc041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.878+0000 7f5cf1600640 1 -- 172.21.15.90:0/4263917678 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5ce0074ab0 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.878+0000 7f5cf8c00640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cdc03f610 0x7f5cdc041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.879+0000 7f5cf8c00640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cdc03f610 0x7f5cdc041ad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005fd0 tx=0x7f5ce800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:41.883+0000 7f5cf1600640 1 -- 172.21.15.90:0/4263917678 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5ce0031490 con 0x7f5cf4077ff0 2024-09-10T21:52:42.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.090+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f5cf41ac020 con 0x7f5cf4077ff0 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.093+0000 7f5cf1600640 1 -- 172.21.15.90:0/4263917678 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v13)=0 v13) ==== 108+0+0 (secure 0 0 0) 0x7f5ce00303a0 con 0x7f5cf4077ff0 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cdc03f610 msgr2=0x7f5cdc041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cdc03f610 0x7f5cdc041ad0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005fd0 tx=0x7f5ce800b390 comp rx=0 tx=0).stop 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 msgr2=0x7f5cf419f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 0x7f5cf419f8d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ce0005ab0 tx=0x7f5ce0003cd0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 shutdown_connections 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cdc03f610 0x7f5cdc041ad0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cf41137e0 0x7f5cf41a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf4077ff0 0x7f5cf419f8d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 --2- 172.21.15.90:0/4263917678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cf40776d0 0x7f5cf419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 >> 172.21.15.90:0/4263917678 conn(0x7f5cf4100840 msgr2=0x7f5cf4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 shutdown_connections 2024-09-10T21:52:42.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.096+0000 7f5cfa000640 1 -- 172.21.15.90:0/4263917678 wait complete. 2024-09-10T21:52:43.096 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 21:52:42 smithi171 systemd[1]: Starting Ceph mgr.z for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T21:52:43.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.242+0000 7f7d4d000640 1 -- 172.21.15.90:0/510722475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d48104df0 msgr2=0x7f7d481051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.242+0000 7f7d4d000640 1 --2- 172.21.15.90:0/510722475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d48104df0 0x7f7d481051d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009900 tx=0x7f7d3802a3f0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.242+0000 7f7d4d000640 1 -- 172.21.15.90:0/510722475 shutdown_connections 2024-09-10T21:52:43.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.242+0000 7f7d4d000640 1 --2- 172.21.15.90:0/510722475 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d4810b180 0x7f7d48111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.242+0000 7f7d4d000640 1 --2- 172.21.15.90:0/510722475 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d4810a800 0x7f7d4810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.242+0000 7f7d4d000640 1 --2- 172.21.15.90:0/510722475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d48104df0 0x7f7d481051d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.242+0000 7f7d4d000640 1 -- 172.21.15.90:0/510722475 >> 172.21.15.90:0/510722475 conn(0x7f7d48100840 msgr2=0x7f7d48102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:43.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.242+0000 7f7d4d000640 1 -- 172.21.15.90:0/510722475 shutdown_connections 2024-09-10T21:52:43.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.242+0000 7f7d4d000640 1 -- 172.21.15.90:0/510722475 wait complete. 2024-09-10T21:52:43.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.243+0000 7f7d4d000640 1 Processor -- start 2024-09-10T21:52:43.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.243+0000 7f7d4d000640 1 -- start start 2024-09-10T21:52:43.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d4d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d48104df0 0x7f7d4819aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:43.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d4d000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d4810a800 0x7f7d4819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:43.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d4d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d4810b180 0x7f7d481a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:43.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d4d000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7d48114a70 con 0x7f7d4810b180 2024-09-10T21:52:43.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d4d000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7d481148f0 con 0x7f7d4810a800 2024-09-10T21:52:43.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d4d000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7d48114bf0 con 0x7f7d48104df0 2024-09-10T21:52:43.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d4810b180 0x7f7d481a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:43.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d48104df0 0x7f7d4819aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:43.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d4810b180 0x7f7d481a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60412/0 (socket says 172.21.15.90:60412) 2024-09-10T21:52:43.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d48104df0 0x7f7d4819aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:54404/0 (socket says 172.21.15.90:54404) 2024-09-10T21:52:43.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47e00640 1 -- 172.21.15.90:0/1639408693 learned_addr learned my addr 172.21.15.90:0/1639408693 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:43.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d3fe00640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d4810a800 0x7f7d4819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47e00640 1 -- 172.21.15.90:0/1639408693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d48104df0 msgr2=0x7f7d4819aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47e00640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d48104df0 0x7f7d4819aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47e00640 1 -- 172.21.15.90:0/1639408693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d4810a800 msgr2=0x7f7d4819b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47e00640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d4810a800 0x7f7d4819b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.244+0000 7f7d47e00640 1 -- 172.21.15.90:0/1639408693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d481a0f90 con 0x7f7d4810b180 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.245+0000 7f7d47e00640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d4810b180 0x7f7d481a07c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000cce0 tx=0x7f7d4000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.245+0000 7f7d44c00640 1 -- 172.21.15.90:0/1639408693 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000f450 con 0x7f7d4810b180 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.245+0000 7f7d44c00640 1 -- 172.21.15.90:0/1639408693 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f7d4000fa40 con 0x7f7d4810b180 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.245+0000 7f7d44c00640 1 -- 172.21.15.90:0/1639408693 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40018910 con 0x7f7d4810b180 2024-09-10T21:52:43.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.245+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d481a1280 con 0x7f7d4810b180 2024-09-10T21:52:43.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.245+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d481a7940 con 0x7f7d4810b180 2024-09-10T21:52:43.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.246+0000 7f7d44c00640 1 -- 172.21.15.90:0/1639408693 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f7d4000f5f0 con 0x7f7d4810b180 2024-09-10T21:52:43.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.246+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d481051d0 con 0x7f7d4810b180 2024-09-10T21:52:43.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.247+0000 7f7d44c00640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d2c03f610 0x7f7d2c041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:43.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.247+0000 7f7d44c00640 1 -- 172.21.15.90:0/1639408693 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7d40054250 con 0x7f7d4810b180 2024-09-10T21:52:43.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.247+0000 7f7d47400640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d2c03f610 0x7f7d2c041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:43.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.248+0000 7f7d47400640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d2c03f610 0x7f7d2c041ad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000c00 tx=0x7f7d38005b50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:43.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.252+0000 7f7d44c00640 1 -- 172.21.15.90:0/1639408693 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7d40014030 con 0x7f7d4810b180 2024-09-10T21:52:43.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.220+0000 7f1b27200640 1 -- 172.21.15.90:0/2696350595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20108610 msgr2=0x7f1b20108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.220+0000 7f1b27200640 1 --2- 172.21.15.90:0/2696350595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20108610 0x7f1b20108a50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1b10009900 tx=0x7f1b1002a410 comp rx=0 tx=0).stop 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.220+0000 7f1b27200640 1 -- 172.21.15.90:0/2696350595 shutdown_connections 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.220+0000 7f1b27200640 1 --2- 172.21.15.90:0/2696350595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b20108f90 0x7f1b2010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.220+0000 7f1b27200640 1 --2- 172.21.15.90:0/2696350595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20108610 0x7f1b20108a50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.220+0000 7f1b27200640 1 --2- 172.21.15.90:0/2696350595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20102c00 0x7f1b20102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.220+0000 7f1b27200640 1 -- 172.21.15.90:0/2696350595 >> 172.21.15.90:0/2696350595 conn(0x7f1b200fe6d0 msgr2=0x7f1b20100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.220+0000 7f1b27200640 1 -- 172.21.15.90:0/2696350595 shutdown_connections 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.220+0000 7f1b27200640 1 -- 172.21.15.90:0/2696350595 wait complete. 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.221+0000 7f1b27200640 1 Processor -- start 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.221+0000 7f1b27200640 1 -- start start 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.221+0000 7f1b27200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c00 0x7f1b200754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.221+0000 7f1b27200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b20108610 0x7f1b200759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.221+0000 7f1b27200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20108f90 0x7f1b20075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.221+0000 7f1b27200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20112910 con 0x7f1b20102c00 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.221+0000 7f1b27200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20112790 con 0x7f1b20108610 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.221+0000 7f1b27200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20112a90 con 0x7f1b20108f90 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c00 0x7f1b200754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c00 0x7f1b200754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60400/0 (socket says 172.21.15.90:60400) 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25e00640 1 -- 172.21.15.90:0/3236837768 learned_addr learned my addr 172.21.15.90:0/3236837768 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25400640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b20108610 0x7f1b200759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b26800640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20108f90 0x7f1b20075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25e00640 1 -- 172.21.15.90:0/3236837768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20108f90 msgr2=0x7f1b20075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25e00640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20108f90 0x7f1b20075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25e00640 1 -- 172.21.15.90:0/3236837768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b20108610 msgr2=0x7f1b200759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25e00640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b20108610 0x7f1b200759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25e00640 1 -- 172.21.15.90:0/3236837768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b201a71e0 con 0x7f1b20102c00 2024-09-10T21:52:43.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.222+0000 7f1b25e00640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c00 0x7f1b200754a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800db80 tx=0x7f1b1800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.223+0000 7f1b16a00640 1 -- 172.21.15.90:0/3236837768 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b18017070 con 0x7f1b20102c00 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.223+0000 7f1b16a00640 1 -- 172.21.15.90:0/3236837768 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f1b18004020 con 0x7f1b20102c00 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.223+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b201a74d0 con 0x7f1b20102c00 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.223+0000 7f1b16a00640 1 -- 172.21.15.90:0/3236837768 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1800be60 con 0x7f1b20102c00 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.223+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b201a7a30 con 0x7f1b20102c00 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.224+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b20102fe0 con 0x7f1b20102c00 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.224+0000 7f1b16a00640 1 -- 172.21.15.90:0/3236837768 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f1b18003b70 con 0x7f1b20102c00 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.225+0000 7f1b16a00640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1b0c03f610 0x7f1b0c041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.225+0000 7f1b16a00640 1 -- 172.21.15.90:0/3236837768 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1b18053c60 con 0x7f1b20102c00 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.225+0000 7f1b25400640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1b0c03f610 0x7f1b0c041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.226+0000 7f1b25400640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1b0c03f610 0x7f1b0c041ad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005ab0 tx=0x7f1b10005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.230+0000 7f1b16a00640 1 -- 172.21.15.90:0/3236837768 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1b18020070 con 0x7f1b20102c00 2024-09-10T21:52:43.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.420+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f1b2010ad70 con 0x7f1b20102c00 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.424+0000 7f1b16a00640 1 -- 172.21.15.90:0/3236837768 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v14)=0 v14) ==== 133+0+0 (secure 0 0 0) 0x7f1b180071c0 con 0x7f1b20102c00 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1b0c03f610 msgr2=0x7f1b0c041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1b0c03f610 0x7f1b0c041ad0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1b10005ab0 tx=0x7f1b10005a20 comp rx=0 tx=0).stop 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c00 msgr2=0x7f1b200754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c00 0x7f1b200754a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1b1800db80 tx=0x7f1b1800b630 comp rx=0 tx=0).stop 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 shutdown_connections 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1b0c03f610 0x7f1b0c041ad0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1b20108f90 0x7f1b20075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1b20108610 0x7f1b200759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 --2- 172.21.15.90:0/3236837768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b20102c00 0x7f1b200754a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 >> 172.21.15.90:0/3236837768 conn(0x7f1b200fe6d0 msgr2=0x7f1b20100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 shutdown_connections 2024-09-10T21:52:43.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:42.427+0000 7f1b27200640 1 -- 172.21.15.90:0/3236837768 wait complete. 2024-09-10T21:52:43.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:43 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4263917678' entity='client.admin' 2024-09-10T21:52:43.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:43 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3236837768' entity='client.admin' 2024-09-10T21:52:43.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:43 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4263917678' entity='client.admin' 2024-09-10T21:52:43.365 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:43 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3236837768' entity='client.admin' 2024-09-10T21:52:43.365 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 21:52:43 smithi171 podman[36839]: 2024-09-10 21:52:43.037854833 +0000 UTC m=+0.026886579 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:52:43.365 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 21:52:43 smithi171 podman[36839]: 2024-09-10 21:52:43.146013147 +0000 UTC m=+0.135044892 container create 105c9e4c7c7653b0b0530e7382ad2f620cde6c672e9facb53c4e17c6f8b1f24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T21:52:43.365 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 21:52:43 smithi171 podman[36839]: 2024-09-10 21:52:43.365118248 +0000 UTC m=+0.354149994 container init 105c9e4c7c7653b0b0530e7382ad2f620cde6c672e9facb53c4e17c6f8b1f24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T21:52:43.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:43 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4263917678' entity='client.admin' 2024-09-10T21:52:43.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:43 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3236837768' entity='client.admin' 2024-09-10T21:52:43.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.442+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f7d4810cf60 con 0x7f7d4810b180 2024-09-10T21:52:43.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.446+0000 7f7d44c00640 1 -- 172.21.15.90:0/1639408693 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v15)=0 v15) ==== 115+0+0 (secure 0 0 0) 0x7f7d40021c90 con 0x7f7d4810b180 2024-09-10T21:52:43.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.448+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d2c03f610 msgr2=0x7f7d2c041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.448+0000 7f7d4d000640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d2c03f610 0x7f7d2c041ad0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d38000c00 tx=0x7f7d38005b50 comp rx=0 tx=0).stop 2024-09-10T21:52:43.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d4810b180 msgr2=0x7f7d481a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:43.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d4810b180 0x7f7d481a07c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000cce0 tx=0x7f7d4000a620 comp rx=0 tx=0).stop 2024-09-10T21:52:43.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 shutdown_connections 2024-09-10T21:52:43.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d2c03f610 0x7f7d2c041ad0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d4810b180 0x7f7d481a07c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d4810a800 0x7f7d4819b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 --2- 172.21.15.90:0/1639408693 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d48104df0 0x7f7d4819aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:43.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 >> 172.21.15.90:0/1639408693 conn(0x7f7d48100840 msgr2=0x7f7d48102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:43.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 shutdown_connections 2024-09-10T21:52:43.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:43.449+0000 7f7d4d000640 1 -- 172.21.15.90:0/1639408693 wait complete. 2024-09-10T21:52:43.747 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 21:52:43 smithi171 podman[36839]: 2024-09-10 21:52:43.373036746 +0000 UTC m=+0.362068492 container start 105c9e4c7c7653b0b0530e7382ad2f620cde6c672e9facb53c4e17c6f8b1f24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T21:52:43.747 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 21:52:43 smithi171 ceph-mgr[36858]: -- 172.21.15.171:0/4092157138 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570ff2190e0 con 0x5570ff1f6800 2024-09-10T21:52:43.747 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 21:52:43 smithi171 bash[36839]: 105c9e4c7c7653b0b0530e7382ad2f620cde6c672e9facb53c4e17c6f8b1f24b 2024-09-10T21:52:43.747 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 21:52:43 smithi171 systemd[1]: Started Ceph mgr.z for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:52:44.484 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1639408693' entity='client.admin' 2024-09-10T21:52:44.484 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:44.484 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:44.484 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:44.484 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T21:52:44.484 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T21:52:44.484 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T21:52:44.484 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:44.484 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: Deploying daemon mgr.y on smithi154 2024-09-10T21:52:44.485 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1196059322' entity='client.admin' 2024-09-10T21:52:44.485 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:44 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/305528758' entity='client.admin' 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1639408693' entity='client.admin' 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: Deploying daemon mgr.y on smithi154 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1196059322' entity='client.admin' 2024-09-10T21:52:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:44 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/305528758' entity='client.admin' 2024-09-10T21:52:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1639408693' entity='client.admin' 2024-09-10T21:52:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T21:52:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T21:52:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T21:52:44.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:44.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: Deploying daemon mgr.y on smithi154 2024-09-10T21:52:44.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1196059322' entity='client.admin' 2024-09-10T21:52:44.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:44 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/305528758' entity='client.admin' 2024-09-10T21:52:46.067 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 21:52:45 smithi154 systemd[1]: Starting Ceph mgr.y for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T21:52:46.323 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 21:52:46 smithi154 podman[36812]: 2024-09-10 21:52:45.97019297 +0000 UTC m=+0.027554433 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:52:46.323 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 21:52:46 smithi154 podman[36812]: 2024-09-10 21:52:46.097425942 +0000 UTC m=+0.154787384 container create bd31d1cd99cb0a81edc9761ab4c9053d13297e196197bdcb50cdfcd31622d83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T21:52:46.323 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 21:52:46 smithi154 podman[36812]: 2024-09-10 21:52:46.31649291 +0000 UTC m=+0.373854353 container init bd31d1cd99cb0a81edc9761ab4c9053d13297e196197bdcb50cdfcd31622d83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T21:52:46.323 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 21:52:46 smithi154 podman[36812]: 2024-09-10 21:52:46.322670655 +0000 UTC m=+0.380032098 container start bd31d1cd99cb0a81edc9761ab4c9053d13297e196197bdcb50cdfcd31622d83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T21:52:46.667 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 21:52:46 smithi154 ceph-mgr[36831]: -- 172.21.15.154:0/2418397060 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d49c9d30e0 con 0x55d49c9b0800 2024-09-10T21:52:46.667 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 21:52:46 smithi154 bash[36812]: bd31d1cd99cb0a81edc9761ab4c9053d13297e196197bdcb50cdfcd31622d83c 2024-09-10T21:52:46.667 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 21:52:46 smithi154 systemd[1]: Started Ceph mgr.y for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:52:47.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:46 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/760419298' entity='client.admin' 2024-09-10T21:52:47.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:46 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3062260927' entity='client.admin' 2024-09-10T21:52:47.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:47.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:46 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4220734422' entity='client.admin' 2024-09-10T21:52:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:46 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/760419298' entity='client.admin' 2024-09-10T21:52:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:46 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3062260927' entity='client.admin' 2024-09-10T21:52:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:46 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4220734422' entity='client.admin' 2024-09-10T21:52:47.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:46 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/760419298' entity='client.admin' 2024-09-10T21:52:47.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:46 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3062260927' entity='client.admin' 2024-09-10T21:52:47.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:47.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:52:47.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:46 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4220734422' entity='client.admin' 2024-09-10T21:52:47.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.453+0000 7f5b1d600640 1 -- 172.21.15.90:0/2128090034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b1810b350 msgr2=0x7f5b18111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.453+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2128090034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b1810b350 0x7f5b18111b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009ca0 tx=0x7f5b1002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.453+0000 7f5b1d600640 1 -- 172.21.15.90:0/2128090034 shutdown_connections 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.453+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2128090034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b1810b350 0x7f5b18111b50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.453+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2128090034 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b1810a9d0 0x7f5b1810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.453+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2128090034 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b18104fc0 0x7f5b181053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.453+0000 7f5b1d600640 1 -- 172.21.15.90:0/2128090034 >> 172.21.15.90:0/2128090034 conn(0x7f5b181009f0 msgr2=0x7f5b18102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.453+0000 7f5b1d600640 1 -- 172.21.15.90:0/2128090034 shutdown_connections 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.453+0000 7f5b1d600640 1 -- 172.21.15.90:0/2128090034 wait complete. 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.454+0000 7f5b1d600640 1 Processor -- start 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.454+0000 7f5b1d600640 1 -- start start 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b1d600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b18104fc0 0x7f5b1819b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b1d600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b1810a9d0 0x7f5b1819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:47.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b1d600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b1810b350 0x7f5b181a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b1d600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18114d60 con 0x7f5b18104fc0 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b1d600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18114be0 con 0x7f5b1810a9d0 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b1d600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5b18114ee0 con 0x7f5b1810b350 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b16a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b18104fc0 0x7f5b1819b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b0fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b1810a9d0 0x7f5b1819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b16a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b18104fc0 0x7f5b1819b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60532/0 (socket says 172.21.15.90:60532) 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b16a00640 1 -- 172.21.15.90:0/2819677810 learned_addr learned my addr 172.21.15.90:0/2819677810 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b17400640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b1810b350 0x7f5b181a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b16a00640 1 -- 172.21.15.90:0/2819677810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b1810b350 msgr2=0x7f5b181a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b16a00640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b1810b350 0x7f5b181a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b16a00640 1 -- 172.21.15.90:0/2819677810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b1810a9d0 msgr2=0x7f5b1819b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b16a00640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b1810a9d0 0x7f5b1819b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b16a00640 1 -- 172.21.15.90:0/2819677810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b181a13a0 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.455+0000 7f5b0fe00640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b1810a9d0 0x7f5b1819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.456+0000 7f5b16a00640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b18104fc0 0x7f5b1819b2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800d950 tx=0x7f5b0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.456+0000 7f5b0f400640 1 -- 172.21.15.90:0/2819677810 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b08017070 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.456+0000 7f5b0f400640 1 -- 172.21.15.90:0/2819677810 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f5b08009c30 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.456+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b181a1690 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.456+0000 7f5b0f400640 1 -- 172.21.15.90:0/2819677810 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800f620 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.456+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b181a7d50 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.457+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b181053a0 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.458+0000 7f5b0f400640 1 -- 172.21.15.90:0/2819677810 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f5b08010460 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.458+0000 7f5b0f400640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5afc03f560 0x7f5afc041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.458+0000 7f5b0f400640 1 -- 172.21.15.90:0/2819677810 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5b08053b70 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.458+0000 7f5b0fe00640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5afc03f560 0x7f5afc041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.459+0000 7f5b0fe00640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5afc03f560 0x7f5afc041a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b18109520 tx=0x7f5b0000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.463+0000 7f5b0f400640 1 -- 172.21.15.90:0/2819677810 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5b08009de0 con 0x7f5b18104fc0 2024-09-10T21:52:47.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.657+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f5b1810d130 con 0x7f5b18104fc0 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.660+0000 7f5b0f400640 1 -- 172.21.15.90:0/2819677810 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v21)=0 v21) ==== 127+0+0 (secure 0 0 0) 0x7f5b08010e10 con 0x7f5b18104fc0 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5afc03f560 msgr2=0x7f5afc041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5afc03f560 0x7f5afc041a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b18109520 tx=0x7f5b0000b390 comp rx=0 tx=0).stop 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b18104fc0 msgr2=0x7f5b1819b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b18104fc0 0x7f5b1819b2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800d950 tx=0x7f5b0800de10 comp rx=0 tx=0).stop 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 shutdown_connections 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5afc03f560 0x7f5afc041a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5b1810b350 0x7f5b181a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5b1810a9d0 0x7f5b1819b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 --2- 172.21.15.90:0/2819677810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b18104fc0 0x7f5b1819b2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 >> 172.21.15.90:0/2819677810 conn(0x7f5b181009f0 msgr2=0x7f5b18102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 shutdown_connections 2024-09-10T21:52:47.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:47.663+0000 7f5b1d600640 1 -- 172.21.15.90:0/2819677810 wait complete. 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.210+0000 7f9b7ec00640 1 -- 172.21.15.90:0/759611232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 msgr2=0x7f9b7810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.210+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/759611232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 0x7f9b7810c920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009900 tx=0x7f9b6402a410 comp rx=0 tx=0).stop 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.210+0000 7f9b7ec00640 1 -- 172.21.15.90:0/759611232 shutdown_connections 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.210+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/759611232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b7810cf70 0x7f9b7810f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.210+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/759611232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 0x7f9b7810c920 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.210+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/759611232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b78103b10 0x7f9b78103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.210+0000 7f9b7ec00640 1 -- 172.21.15.90:0/759611232 >> 172.21.15.90:0/759611232 conn(0x7f9b780fd5b0 msgr2=0x7f9b780ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.211+0000 7f9b7ec00640 1 -- 172.21.15.90:0/759611232 shutdown_connections 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.211+0000 7f9b7ec00640 1 -- 172.21.15.90:0/759611232 wait complete. 2024-09-10T21:52:48.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.211+0000 7f9b7ec00640 1 Processor -- start 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ec00640 1 -- start start 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ec00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b78103b10 0x7f9b7819b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ec00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 0x7f9b7819b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ec00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b7810cf70 0x7f9b781a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ec00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9b78112850 con 0x7f9b78104430 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ec00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781126d0 con 0x7f9b78103b10 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ec00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b781129d0 con 0x7f9b7810cf70 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ce00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 0x7f9b7819b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ce00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 0x7f9b7819b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60470/0 (socket says 172.21.15.90:60470) 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7ce00640 1 -- 172.21.15.90:0/305528758 learned_addr learned my addr 172.21.15.90:0/305528758 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7e200640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b7810cf70 0x7f9b781a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.212+0000 7f9b7d800640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b78103b10 0x7f9b7819b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7ce00640 1 -- 172.21.15.90:0/305528758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b7810cf70 msgr2=0x7f9b781a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:48.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7ce00640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b7810cf70 0x7f9b781a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7ce00640 1 -- 172.21.15.90:0/305528758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b78103b10 msgr2=0x7f9b7819b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7ce00640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b78103b10 0x7f9b7819b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7ce00640 1 -- 172.21.15.90:0/305528758 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b781a1110 con 0x7f9b78104430 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7e200640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b7810cf70 0x7f9b781a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7d800640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b78103b10 0x7f9b7819b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7ce00640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 0x7f9b7819b5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9b64004650 tx=0x7f9b640328c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b76000640 1 -- 172.21.15.90:0/305528758 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6403f030 con 0x7f9b78104430 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b76000640 1 -- 172.21.15.90:0/305528758 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f9b64032db0 con 0x7f9b78104430 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b781a13a0 con 0x7f9b78104430 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b76000640 1 -- 172.21.15.90:0/305528758 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b64035070 con 0x7f9b78104430 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.213+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b781a7ac0 con 0x7f9b78104430 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.215+0000 7f9b76000640 1 -- 172.21.15.90:0/305528758 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f9b640035f0 con 0x7f9b78104430 2024-09-10T21:52:48.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.215+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b78103ef0 con 0x7f9b78104430 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.215+0000 7f9b76000640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b6003f610 0x7f9b60041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.215+0000 7f9b76000640 1 -- 172.21.15.90:0/305528758 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9b640744a0 con 0x7f9b78104430 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.215+0000 7f9b7d800640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b6003f610 0x7f9b60041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.216+0000 7f9b7d800640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b6003f610 0x7f9b60041ad0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009fc0 tx=0x7f9b6c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.220+0000 7f9b76000640 1 -- 172.21.15.90:0/305528758 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9b64003910 con 0x7f9b78104430 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.417+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f9b781a7c90 con 0x7f9b78104430 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.421+0000 7f9b76000640 1 -- 172.21.15.90:0/305528758 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v17)=0 v17) ==== 115+0+0 (secure 0 0 0) 0x7f9b64003c60 con 0x7f9b78104430 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b6003f610 msgr2=0x7f9b60041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b6003f610 0x7f9b60041ad0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009fc0 tx=0x7f9b6c005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 msgr2=0x7f9b7819b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 0x7f9b7819b5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9b64004650 tx=0x7f9b640328c0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 shutdown_connections 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b6003f610 0x7f9b60041ad0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b7810cf70 0x7f9b781a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b78104430 0x7f9b7819b5a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 --2- 172.21.15.90:0/305528758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b78103b10 0x7f9b7819b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 >> 172.21.15.90:0/305528758 conn(0x7f9b780fd5b0 msgr2=0x7f9b780ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 shutdown_connections 2024-09-10T21:52:48.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.424+0000 7f9b7ec00640 1 -- 172.21.15.90:0/305528758 wait complete. 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2819677810' entity='client.admin' 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T21:52:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T21:52:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:48.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2819677810' entity='client.admin' 2024-09-10T21:52:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T21:52:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T21:52:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2819677810' entity='client.admin' 2024-09-10T21:52:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:52:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T21:52:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T21:52:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:49.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.113+0000 7f941a200640 1 -- 172.21.15.90:0/1626988880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9414104fa0 msgr2=0x7f9414105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.113+0000 7f941a200640 1 --2- 172.21.15.90:0/1626988880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9414104fa0 0x7f9414105380 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9404009900 tx=0x7f940402a410 comp rx=0 tx=0).stop 2024-09-10T21:52:49.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.113+0000 7f941a200640 1 -- 172.21.15.90:0/1626988880 shutdown_connections 2024-09-10T21:52:49.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.113+0000 7f941a200640 1 --2- 172.21.15.90:0/1626988880 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f941410b330 0x7f9414111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.113+0000 7f941a200640 1 --2- 172.21.15.90:0/1626988880 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f941410a9b0 0x7f941410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.113+0000 7f941a200640 1 --2- 172.21.15.90:0/1626988880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9414104fa0 0x7f9414105380 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.113+0000 7f941a200640 1 -- 172.21.15.90:0/1626988880 >> 172.21.15.90:0/1626988880 conn(0x7f9414100a50 msgr2=0x7f9414102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:49.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.113+0000 7f941a200640 1 -- 172.21.15.90:0/1626988880 shutdown_connections 2024-09-10T21:52:49.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.113+0000 7f941a200640 1 -- 172.21.15.90:0/1626988880 wait complete. 2024-09-10T21:52:49.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.114+0000 7f941a200640 1 Processor -- start 2024-09-10T21:52:49.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.114+0000 7f941a200640 1 -- start start 2024-09-10T21:52:49.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.114+0000 7f941a200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9414104fa0 0x7f941419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:49.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.114+0000 7f941a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f941410a9b0 0x7f941419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:49.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.114+0000 7f941a200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f941410b330 0x7f94141a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:49.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.114+0000 7f941a200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9414114d00 con 0x7f941410a9b0 2024-09-10T21:52:49.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f941a200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9414114b80 con 0x7f941410b330 2024-09-10T21:52:49.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f941a200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9414114e80 con 0x7f9414104fa0 2024-09-10T21:52:49.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9412a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f941410a9b0 0x7f941419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:49.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9413400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9414104fa0 0x7f941419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:49.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9412a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f941410a9b0 0x7f941419b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56330/0 (socket says 172.21.15.90:56330) 2024-09-10T21:52:49.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9412a00640 1 -- 172.21.15.90:0/2227970665 learned_addr learned my addr 172.21.15.90:0/2227970665 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:49.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9413e00640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f941410b330 0x7f94141a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:49.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9412a00640 1 -- 172.21.15.90:0/2227970665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9414104fa0 msgr2=0x7f941419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9412a00640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9414104fa0 0x7f941419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9412a00640 1 -- 172.21.15.90:0/2227970665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f941410b330 msgr2=0x7f94141a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9412a00640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f941410b330 0x7f94141a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9412a00640 1 -- 172.21.15.90:0/2227970665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94141a1380 con 0x7f941410a9b0 2024-09-10T21:52:49.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.115+0000 7f9412a00640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f941410a9b0 0x7f941419b810 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f940000e9f0 tx=0x7f940000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:49.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.116+0000 7f940be00640 1 -- 172.21.15.90:0/2227970665 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940000cd80 con 0x7f941410a9b0 2024-09-10T21:52:49.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.116+0000 7f940be00640 1 -- 172.21.15.90:0/2227970665 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f94000108a0 con 0x7f941410a9b0 2024-09-10T21:52:49.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.116+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94141a1670 con 0x7f941410a9b0 2024-09-10T21:52:49.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.116+0000 7f940be00640 1 -- 172.21.15.90:0/2227970665 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9400018750 con 0x7f941410a9b0 2024-09-10T21:52:49.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.116+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94141a7d30 con 0x7f941410a9b0 2024-09-10T21:52:49.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.117+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9414105380 con 0x7f941410a9b0 2024-09-10T21:52:49.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.117+0000 7f940be00640 1 -- 172.21.15.90:0/2227970665 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f94000188f0 con 0x7f941410a9b0 2024-09-10T21:52:49.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.118+0000 7f940be00640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93f803f560 0x7f93f8041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:49.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.118+0000 7f940be00640 1 -- 172.21.15.90:0/2227970665 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9400054520 con 0x7f941410a9b0 2024-09-10T21:52:49.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.118+0000 7f9413400640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93f803f560 0x7f93f8041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:49.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.119+0000 7f9413400640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93f803f560 0x7f93f8041a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9404005ab0 tx=0x7f9404005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:49.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.122+0000 7f940be00640 1 -- 172.21.15.90:0/2227970665 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9400020360 con 0x7f941410a9b0 2024-09-10T21:52:49.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.325+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f941410d110 con 0x7f941410a9b0 2024-09-10T21:52:49.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.329+0000 7f940be00640 1 -- 172.21.15.90:0/2227970665 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v22)=0 v22) ==== 118+0+0 (secure 0 0 0) 0x7f940001fe00 con 0x7f941410a9b0 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93f803f560 msgr2=0x7f93f8041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93f803f560 0x7f93f8041a20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9404005ab0 tx=0x7f9404005a20 comp rx=0 tx=0).stop 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f941410a9b0 msgr2=0x7f941419b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f941410a9b0 0x7f941419b810 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f940000e9f0 tx=0x7f940000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 shutdown_connections 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93f803f560 0x7f93f8041a20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f941410b330 0x7f94141a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f941410a9b0 0x7f941419b810 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 --2- 172.21.15.90:0/2227970665 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9414104fa0 0x7f941419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.331+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 >> 172.21.15.90:0/2227970665 conn(0x7f9414100a50 msgr2=0x7f9414102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:49.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.332+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 shutdown_connections 2024-09-10T21:52:49.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.332+0000 7f941a200640 1 -- 172.21.15.90:0/2227970665 wait complete. 2024-09-10T21:52:49.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.542+0000 7f7a52a00640 1 -- 172.21.15.90:0/2184335138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 msgr2=0x7f7a4c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.542+0000 7f7a52a00640 1 --2- 172.21.15.90:0/2184335138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 0x7f7a4c102fe0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009ca0 tx=0x7f7a4002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.542+0000 7f7a52a00640 1 -- 172.21.15.90:0/2184335138 shutdown_connections 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.542+0000 7f7a52a00640 1 --2- 172.21.15.90:0/2184335138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4c108f90 0x7f7a4c10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.542+0000 7f7a52a00640 1 --2- 172.21.15.90:0/2184335138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c108610 0x7f7a4c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.542+0000 7f7a52a00640 1 --2- 172.21.15.90:0/2184335138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 0x7f7a4c102fe0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.542+0000 7f7a52a00640 1 -- 172.21.15.90:0/2184335138 >> 172.21.15.90:0/2184335138 conn(0x7f7a4c0fe6d0 msgr2=0x7f7a4c100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.542+0000 7f7a52a00640 1 -- 172.21.15.90:0/2184335138 shutdown_connections 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.542+0000 7f7a52a00640 1 -- 172.21.15.90:0/2184335138 wait complete. 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.543+0000 7f7a52a00640 1 Processor -- start 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.543+0000 7f7a52a00640 1 -- start start 2024-09-10T21:52:49.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.543+0000 7f7a52a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 0x7f7a4c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.543+0000 7f7a52a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c108610 0x7f7a4c19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.543+0000 7f7a52a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4c108f90 0x7f7a4c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.543+0000 7f7a52a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c1129e0 con 0x7f7a4c102c00 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.543+0000 7f7a52a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c112860 con 0x7f7a4c108f90 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.543+0000 7f7a52a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c112b60 con 0x7f7a4c108610 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.543+0000 7f7a51600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 0x7f7a4c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a51600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 0x7f7a4c19b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56348/0 (socket says 172.21.15.90:56348) 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a51600640 1 -- 172.21.15.90:0/374454889 learned_addr learned my addr 172.21.15.90:0/374454889 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a50c00640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c108610 0x7f7a4c19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a52000640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4c108f90 0x7f7a4c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a51600640 1 -- 172.21.15.90:0/374454889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c108610 msgr2=0x7f7a4c19b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a51600640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c108610 0x7f7a4c19b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a51600640 1 -- 172.21.15.90:0/374454889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4c108f90 msgr2=0x7f7a4c1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a51600640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4c108f90 0x7f7a4c1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a51600640 1 -- 172.21.15.90:0/374454889 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c1a1150 con 0x7f7a4c102c00 2024-09-10T21:52:49.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a51600640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 0x7f7a4c19b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a40006010 tx=0x7f7a40009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:49.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.544+0000 7f7a4a000640 1 -- 172.21.15.90:0/374454889 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40003db0 con 0x7f7a4c102c00 2024-09-10T21:52:49.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.545+0000 7f7a4a000640 1 -- 172.21.15.90:0/374454889 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f7a400044b0 con 0x7f7a4c102c00 2024-09-10T21:52:49.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.545+0000 7f7a4a000640 1 -- 172.21.15.90:0/374454889 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a40030ba0 con 0x7f7a4c102c00 2024-09-10T21:52:49.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.545+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c1a13e0 con 0x7f7a4c102c00 2024-09-10T21:52:49.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.545+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c1a7b00 con 0x7f7a4c102c00 2024-09-10T21:52:49.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.546+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4c102fe0 con 0x7f7a4c102c00 2024-09-10T21:52:49.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.546+0000 7f7a4a000640 1 -- 172.21.15.90:0/374454889 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f7a40030d40 con 0x7f7a4c102c00 2024-09-10T21:52:49.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.547+0000 7f7a4a000640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7a3403f610 0x7f7a34041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:49.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.547+0000 7f7a4a000640 1 -- 172.21.15.90:0/374454889 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7a40075470 con 0x7f7a4c102c00 2024-09-10T21:52:49.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.547+0000 7f7a50c00640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7a3403f610 0x7f7a34041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:49.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.548+0000 7f7a50c00640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7a3403f610 0x7f7a34041ad0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c1080c0 tx=0x7f7a38005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.552+0000 7f7a4a000640 1 -- 172.21.15.90:0/374454889 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7a40004660 con 0x7f7a4c102c00 2024-09-10T21:52:49.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.745+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f7a4c10f180 con 0x7f7a4c102c00 2024-09-10T21:52:49.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.750+0000 7f7a4a000640 1 -- 172.21.15.90:0/374454889 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v23)=0 v23) ==== 123+0+0 (secure 0 0 0) 0x7f7a4004ece0 con 0x7f7a4c102c00 2024-09-10T21:52:49.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.751+0000 7f7a4a000640 1 -- 172.21.15.90:0/374454889 <== mon.0 v2:172.21.15.90:3300/0 8 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a40036030 con 0x7f7a4c102c00 2024-09-10T21:52:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7a3403f610 msgr2=0x7f7a34041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7a3403f610 0x7f7a34041ad0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c1080c0 tx=0x7f7a38005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 msgr2=0x7f7a4c19b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 0x7f7a4c19b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7a40006010 tx=0x7f7a40009360 comp rx=0 tx=0).stop 2024-09-10T21:52:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 shutdown_connections 2024-09-10T21:52:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7a3403f610 0x7f7a34041ad0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7a4c108f90 0x7f7a4c1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7a4c108610 0x7f7a4c19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 --2- 172.21.15.90:0/374454889 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a4c102c00 0x7f7a4c19b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 >> 172.21.15.90:0/374454889 conn(0x7f7a4c0fe6d0 msgr2=0x7f7a4c104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 shutdown_connections 2024-09-10T21:52:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.752+0000 7f7a52a00640 1 -- 172.21.15.90:0/374454889 wait complete. 2024-09-10T21:52:49.810 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:49 smithi090 ceph-mon[30350]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T21:52:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:49 smithi090 ceph-mon[30350]: Reconfiguring daemon mgr.x on smithi090 2024-09-10T21:52:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:49 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2227970665' entity='client.admin' 2024-09-10T21:52:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:49 smithi090 ceph-mon[30350]: Standby manager daemon z started 2024-09-10T21:52:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:49 smithi090 ceph-mon[30350]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T21:52:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:49 smithi090 ceph-mon[30350]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T21:52:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:49 smithi090 ceph-mon[30350]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T21:52:49.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:49 smithi090 ceph-mon[30350]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T21:52:49.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:49 smithi154 ceph-mon[34495]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T21:52:49.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:49 smithi154 ceph-mon[34495]: Reconfiguring daemon mgr.x on smithi090 2024-09-10T21:52:49.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:49.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:49 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2227970665' entity='client.admin' 2024-09-10T21:52:49.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:49 smithi154 ceph-mon[34495]: Standby manager daemon z started 2024-09-10T21:52:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:49 smithi154 ceph-mon[34495]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T21:52:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:49 smithi154 ceph-mon[34495]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T21:52:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:49 smithi154 ceph-mon[34495]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T21:52:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:49 smithi154 ceph-mon[34495]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T21:52:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:49 smithi171 ceph-mon[34138]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T21:52:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:49 smithi171 ceph-mon[34138]: Reconfiguring daemon mgr.x on smithi090 2024-09-10T21:52:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:49 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2227970665' entity='client.admin' 2024-09-10T21:52:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:49 smithi171 ceph-mon[34138]: Standby manager daemon z started 2024-09-10T21:52:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:49 smithi171 ceph-mon[34138]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T21:52:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:49 smithi171 ceph-mon[34138]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T21:52:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:49 smithi171 ceph-mon[34138]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T21:52:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:49 smithi171 ceph-mon[34138]: from='mgr.? 172.21.15.171:0/852798442' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T21:52:50.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.720+0000 7fc729e00640 1 -- 172.21.15.90:0/1015666983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc724103b10 msgr2=0x7fc724103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.720+0000 7fc729e00640 1 --2- 172.21.15.90:0/1015666983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc724103b10 0x7fc724103ef0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc714009900 tx=0x7fc71402a410 comp rx=0 tx=0).stop 2024-09-10T21:52:50.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.720+0000 7fc729e00640 1 -- 172.21.15.90:0/1015666983 shutdown_connections 2024-09-10T21:52:50.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.720+0000 7fc729e00640 1 --2- 172.21.15.90:0/1015666983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc72410cf70 0x7fc72410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.720+0000 7fc729e00640 1 --2- 172.21.15.90:0/1015666983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724104430 0x7fc72410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.720+0000 7fc729e00640 1 --2- 172.21.15.90:0/1015666983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc724103b10 0x7fc724103ef0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.720+0000 7fc729e00640 1 -- 172.21.15.90:0/1015666983 >> 172.21.15.90:0/1015666983 conn(0x7fc7240fd5b0 msgr2=0x7fc7240ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.720+0000 7fc729e00640 1 -- 172.21.15.90:0/1015666983 shutdown_connections 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.720+0000 7fc729e00640 1 -- 172.21.15.90:0/1015666983 wait complete. 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729e00640 1 Processor -- start 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729e00640 1 -- start start 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc724103b10 0x7fc72419afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724104430 0x7fc72419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc72410cf70 0x7fc7241a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc7241128a0 con 0x7fc72410cf70 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc724112720 con 0x7fc724103b10 2024-09-10T21:52:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc724112a20 con 0x7fc724104430 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc72410cf70 0x7fc7241a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc72410cf70 0x7fc7241a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60482/0 (socket says 172.21.15.90:60482) 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc729400640 1 -- 172.21.15.90:0/760419298 learned_addr learned my addr 172.21.15.90:0/760419298 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc728a00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc724103b10 0x7fc72419afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.721+0000 7fc723e00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724104430 0x7fc72419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc729400640 1 -- 172.21.15.90:0/760419298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724104430 msgr2=0x7fc72419b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc729400640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724104430 0x7fc72419b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc729400640 1 -- 172.21.15.90:0/760419298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc724103b10 msgr2=0x7fc72419afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc729400640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc724103b10 0x7fc72419afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc729400640 1 -- 172.21.15.90:0/760419298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7241a1080 con 0x7fc72410cf70 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc729400640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc72410cf70 0x7fc7241a08b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009fd0 tx=0x7fc71c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc721600640 1 -- 172.21.15.90:0/760419298 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c017070 con 0x7fc72410cf70 2024-09-10T21:52:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc721600640 1 -- 172.21.15.90:0/760419298 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fc71c003dc0 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7241a1370 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc721600640 1 -- 172.21.15.90:0/760419298 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0047a0 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.722+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7241a7a30 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.724+0000 7fc721600640 1 -- 172.21.15.90:0/760419298 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7fc71c003970 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.724+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc724103ef0 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.724+0000 7fc721600640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc70803f610 0x7fc708041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.724+0000 7fc721600640 1 -- 172.21.15.90:0/760419298 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc71c053c00 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.724+0000 7fc728a00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc70803f610 0x7fc708041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.725+0000 7fc728a00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc70803f610 0x7fc708041ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc714005ab0 tx=0x7fc714005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.729+0000 7fc721600640 1 -- 172.21.15.90:0/760419298 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc71c00e350 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.941+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fc7241a7c90 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.945+0000 7fc721600640 1 -- 172.21.15.90:0/760419298 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v18)=0 v18) ==== 123+0+0 (secure 0 0 0) 0x7fc71c022df0 con 0x7fc72410cf70 2024-09-10T21:52:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.945+0000 7fc721600640 1 -- 172.21.15.90:0/760419298 <== mon.0 v2:172.21.15.90:3300/0 8 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fc71c059080 con 0x7fc72410cf70 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc70803f610 msgr2=0x7fc708041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc70803f610 0x7fc708041ad0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc714005ab0 tx=0x7fc714005a20 comp rx=0 tx=0).stop 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc72410cf70 msgr2=0x7fc7241a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc72410cf70 0x7fc7241a08b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc71c009fd0 tx=0x7fc71c00b260 comp rx=0 tx=0).stop 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 shutdown_connections 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc70803f610 0x7fc708041ad0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc72410cf70 0x7fc7241a08b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc724104430 0x7fc72419b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 --2- 172.21.15.90:0/760419298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc724103b10 0x7fc72419afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 >> 172.21.15.90:0/760419298 conn(0x7fc7240fd5b0 msgr2=0x7fc7240ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 shutdown_connections 2024-09-10T21:52:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:45.947+0000 7fc729e00640 1 -- 172.21.15.90:0/760419298 wait complete. 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.125+0000 7f275a200640 1 -- 172.21.15.90:0/1490193306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 msgr2=0x7f2754111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.125+0000 7f275a200640 1 --2- 172.21.15.90:0/1490193306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 0x7f2754111b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f274c009ca0 tx=0x7f274c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.125+0000 7f275a200640 1 -- 172.21.15.90:0/1490193306 shutdown_connections 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.125+0000 7f275a200640 1 --2- 172.21.15.90:0/1490193306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 0x7f2754111b30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.125+0000 7f275a200640 1 --2- 172.21.15.90:0/1490193306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f275410a9b0 0x7f275410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.125+0000 7f275a200640 1 --2- 172.21.15.90:0/1490193306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2754104fa0 0x7f2754105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.125+0000 7f275a200640 1 -- 172.21.15.90:0/1490193306 >> 172.21.15.90:0/1490193306 conn(0x7f2754100a50 msgr2=0x7f2754102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.125+0000 7f275a200640 1 -- 172.21.15.90:0/1490193306 shutdown_connections 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.125+0000 7f275a200640 1 -- 172.21.15.90:0/1490193306 wait complete. 2024-09-10T21:52:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.126+0000 7f275a200640 1 Processor -- start 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f275a200640 1 -- start start 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f275a200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2754104fa0 0x7f275419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f275a200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f275410a9b0 0x7f275419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f275a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 0x7f27541a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f275a200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2754114c20 con 0x7f275410b330 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f275a200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2754114aa0 con 0x7f2754104fa0 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f275a200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2754114da0 con 0x7f275410a9b0 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f2753e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 0x7f27541a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f2753e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 0x7f27541a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60438/0 (socket says 172.21.15.90:60438) 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f2753e00640 1 -- 172.21.15.90:0/1196059322 learned_addr learned my addr 172.21.15.90:0/1196059322 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:50.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f2753400640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2754104fa0 0x7f275419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.127+0000 7f2752a00640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f275410a9b0 0x7f275419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f2753e00640 1 -- 172.21.15.90:0/1196059322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f275410a9b0 msgr2=0x7f275419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f2753e00640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f275410a9b0 0x7f275419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f2753e00640 1 -- 172.21.15.90:0/1196059322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2754104fa0 msgr2=0x7f275419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f2753e00640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2754104fa0 0x7f275419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f2753e00640 1 -- 172.21.15.90:0/1196059322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27541a12c0 con 0x7f275410b330 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f2753e00640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 0x7f27541a0af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f274c031040 tx=0x7f274c004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f274be00640 1 -- 172.21.15.90:0/1196059322 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c0305c0 con 0x7f275410b330 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f274be00640 1 -- 172.21.15.90:0/1196059322 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f274c030bb0 con 0x7f275410b330 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27541a1550 con 0x7f275410b330 2024-09-10T21:52:50.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.128+0000 7f274be00640 1 -- 172.21.15.90:0/1196059322 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c031840 con 0x7f275410b330 2024-09-10T21:52:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.129+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27541a7c70 con 0x7f275410b330 2024-09-10T21:52:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.129+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2754105380 con 0x7f275410b330 2024-09-10T21:52:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.130+0000 7f274be00640 1 -- 172.21.15.90:0/1196059322 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f274c04d030 con 0x7f275410b330 2024-09-10T21:52:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.130+0000 7f274be00640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f273803f560 0x7f2738041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.130+0000 7f274be00640 1 -- 172.21.15.90:0/1196059322 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f274c074c90 con 0x7f275410b330 2024-09-10T21:52:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.130+0000 7f2753400640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f273803f560 0x7f2738041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.131+0000 7f2753400640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f273803f560 0x7f2738041a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2744009bd0 tx=0x7f2744005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.135+0000 7f274be00640 1 -- 172.21.15.90:0/1196059322 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f274c030940 con 0x7f275410b330 2024-09-10T21:52:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.331+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f275410d110 con 0x7f275410b330 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.334+0000 7f274be00640 1 -- 172.21.15.90:0/1196059322 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f274c04ecd0 con 0x7f275410b330 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f273803f560 msgr2=0x7f2738041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f273803f560 0x7f2738041a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2744009bd0 tx=0x7f2744005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 msgr2=0x7f27541a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 0x7f27541a0af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f274c031040 tx=0x7f274c004530 comp rx=0 tx=0).stop 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 shutdown_connections 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f273803f560 0x7f2738041a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f275410b330 0x7f27541a0af0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f275410a9b0 0x7f275419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 --2- 172.21.15.90:0/1196059322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2754104fa0 0x7f275419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 >> 172.21.15.90:0/1196059322 conn(0x7f2754100a50 msgr2=0x7f2754102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 shutdown_connections 2024-09-10T21:52:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:44.337+0000 7f275a200640 1 -- 172.21.15.90:0/1196059322 wait complete. 2024-09-10T21:52:51.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:50 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/374454889' entity='client.admin' 2024-09-10T21:52:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:50 smithi090 ceph-mon[30350]: from='client.? ' entity='client.admin' 2024-09-10T21:52:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:50 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2713131362' entity='client.admin' 2024-09-10T21:52:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:50 smithi090 ceph-mon[30350]: mgrmap e18: x(active, since 16s), standbys: z 2024-09-10T21:52:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T21:52:51.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:50 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/374454889' entity='client.admin' 2024-09-10T21:52:51.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:50 smithi154 ceph-mon[34495]: from='client.? ' entity='client.admin' 2024-09-10T21:52:51.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:50 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2713131362' entity='client.admin' 2024-09-10T21:52:51.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:50 smithi154 ceph-mon[34495]: mgrmap e18: x(active, since 16s), standbys: z 2024-09-10T21:52:51.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T21:52:51.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:50 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/374454889' entity='client.admin' 2024-09-10T21:52:51.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:50 smithi171 ceph-mon[34138]: from='client.? ' entity='client.admin' 2024-09-10T21:52:51.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:50 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2713131362' entity='client.admin' 2024-09-10T21:52:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:50 smithi171 ceph-mon[34138]: mgrmap e18: x(active, since 16s), standbys: z 2024-09-10T21:52:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T21:52:51.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.956+0000 7f9943400640 1 -- 172.21.15.90:0/1388977698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c10d120 msgr2=0x7f993c10f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.956+0000 7f9943400640 1 --2- 172.21.15.90:0/1388977698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c10d120 0x7f993c10f580 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9934009ca0 tx=0x7f993402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.956+0000 7f9943400640 1 -- 172.21.15.90:0/1388977698 shutdown_connections 2024-09-10T21:52:51.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.956+0000 7f9943400640 1 --2- 172.21.15.90:0/1388977698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c10d120 0x7f993c10f580 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.956+0000 7f9943400640 1 --2- 172.21.15.90:0/1388977698 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f993c1045e0 0x7f993c10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.956+0000 7f9943400640 1 --2- 172.21.15.90:0/1388977698 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c103cc0 0x7f993c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.956+0000 7f9943400640 1 -- 172.21.15.90:0/1388977698 >> 172.21.15.90:0/1388977698 conn(0x7f993c0fd700 msgr2=0x7f993c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:51.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.956+0000 7f9943400640 1 -- 172.21.15.90:0/1388977698 shutdown_connections 2024-09-10T21:52:51.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.956+0000 7f9943400640 1 -- 172.21.15.90:0/1388977698 wait complete. 2024-09-10T21:52:51.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.957+0000 7f9943400640 1 Processor -- start 2024-09-10T21:52:51.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.957+0000 7f9943400640 1 -- start start 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9943400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c103cc0 0x7f993c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9943400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c1045e0 0x7f993c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9943400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f993c10d120 0x7f993c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9943400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f993c112c90 con 0x7f993c103cc0 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9943400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f993c112b10 con 0x7f993c1045e0 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9943400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f993c112e10 con 0x7f993c10d120 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9940a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c103cc0 0x7f993c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9940a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c103cc0 0x7f993c19b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56388/0 (socket says 172.21.15.90:56388) 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9940a00640 1 -- 172.21.15.90:0/2713131362 learned_addr learned my addr 172.21.15.90:0/2713131362 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f993be00640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c1045e0 0x7f993c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.958+0000 7f9941400640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f993c10d120 0x7f993c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9940a00640 1 -- 172.21.15.90:0/2713131362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f993c10d120 msgr2=0x7f993c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9940a00640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f993c10d120 0x7f993c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9940a00640 1 -- 172.21.15.90:0/2713131362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c1045e0 msgr2=0x7f993c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9940a00640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c1045e0 0x7f993c19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9940a00640 1 -- 172.21.15.90:0/2713131362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c1a1470 con 0x7f993c103cc0 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f993be00640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c1045e0 0x7f993c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9940a00640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c103cc0 0x7f993c19b3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f993000e9f0 tx=0x7f993000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9939600640 1 -- 172.21.15.90:0/2713131362 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993000cd80 con 0x7f993c103cc0 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9939600640 1 -- 172.21.15.90:0/2713131362 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99300108c0 con 0x7f993c103cc0 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c1a1760 con 0x7f993c103cc0 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9939600640 1 -- 172.21.15.90:0/2713131362 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99300187c0 con 0x7f993c103cc0 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.959+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c1a7e20 con 0x7f993c103cc0 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.960+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f993c1040a0 con 0x7f993c103cc0 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.960+0000 7f9939600640 1 -- 172.21.15.90:0/2713131362 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7f99300189a0 con 0x7f993c103cc0 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.961+0000 7f9939600640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f992403f560 0x7f9924041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:51.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.961+0000 7f9939600640 1 -- 172.21.15.90:0/2713131362 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9930054550 con 0x7f993c103cc0 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.962+0000 7f993be00640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f992403f560 0x7f9924041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.962+0000 7f993be00640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f992403f560 0x7f9924041a20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f993c10e750 tx=0x7f9928005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.965+0000 7f9939600640 1 -- 172.21.15.90:0/2713131362 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9930016e00 con 0x7f993c103cc0 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.162+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f993c1a8160 con 0x7f993c103cc0 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.166+0000 7f9939600640 1 -- 172.21.15.90:0/2713131362 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v25)=0 v25) ==== 106+0+0 (secure 0 0 0) 0x7f9930010d50 con 0x7f993c103cc0 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.168+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f992403f560 msgr2=0x7f9924041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.168+0000 7f9943400640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f992403f560 0x7f9924041a20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f993c10e750 tx=0x7f9928005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.168+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c103cc0 msgr2=0x7f993c19b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.168+0000 7f9943400640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c103cc0 0x7f993c19b3c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f993000e9f0 tx=0x7f993000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.169+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 shutdown_connections 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.169+0000 7f9943400640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f992403f560 0x7f9924041a20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.169+0000 7f9943400640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f993c10d120 0x7f993c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.169+0000 7f9943400640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f993c1045e0 0x7f993c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.169+0000 7f9943400640 1 --2- 172.21.15.90:0/2713131362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c103cc0 0x7f993c19b3c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.169+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 >> 172.21.15.90:0/2713131362 conn(0x7f993c0fd700 msgr2=0x7f993c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:51.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.169+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 shutdown_connections 2024-09-10T21:52:51.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.169+0000 7f9943400640 1 -- 172.21.15.90:0/2713131362 wait complete. 2024-09-10T21:52:51.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.394+0000 7f9f9ee00640 1 -- 172.21.15.90:0/1373220580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810b330 msgr2=0x7f9f98111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.394+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/1373220580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810b330 0x7f9f98111b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009900 tx=0x7f9f9002a410 comp rx=0 tx=0).stop 2024-09-10T21:52:51.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.394+0000 7f9f9ee00640 1 -- 172.21.15.90:0/1373220580 shutdown_connections 2024-09-10T21:52:51.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.394+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/1373220580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810b330 0x7f9f98111b30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.394+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/1373220580 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f9810a9b0 0x7f9f9810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.394+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/1373220580 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f98104fa0 0x7f9f98105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.394+0000 7f9f9ee00640 1 -- 172.21.15.90:0/1373220580 >> 172.21.15.90:0/1373220580 conn(0x7f9f98100a50 msgr2=0x7f9f98102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.394+0000 7f9f9ee00640 1 -- 172.21.15.90:0/1373220580 shutdown_connections 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.394+0000 7f9f9ee00640 1 -- 172.21.15.90:0/1373220580 wait complete. 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.395+0000 7f9f9ee00640 1 Processor -- start 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.395+0000 7f9f9ee00640 1 -- start start 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.395+0000 7f9f9ee00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f98104fa0 0x7f9f9819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.395+0000 7f9f9ee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810a9b0 0x7f9f9819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f9ee00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f9810b330 0x7f9f981a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f9ee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98114e10 con 0x7f9f9810a9b0 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f9ee00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98114c90 con 0x7f9f9810b330 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f9ee00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98114f90 con 0x7f9f98104fa0 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810a9b0 0x7f9f9819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:51.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810a9b0 0x7f9f9819b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56420/0 (socket says 172.21.15.90:56420) 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97400640 1 -- 172.21.15.90:0/38061563 learned_addr learned my addr 172.21.15.90:0/38061563 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f9ce00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f9810b330 0x7f9f981a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97e00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f98104fa0 0x7f9f9819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97400640 1 -- 172.21.15.90:0/38061563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f98104fa0 msgr2=0x7f9f9819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97400640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f98104fa0 0x7f9f9819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97400640 1 -- 172.21.15.90:0/38061563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f9810b330 msgr2=0x7f9f981a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97400640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f9810b330 0x7f9f981a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97400640 1 -- 172.21.15.90:0/38061563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f981a1430 con 0x7f9f9810a9b0 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.396+0000 7f9f97400640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810a9b0 0x7f9f9819b8c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c920 tx=0x7f9f8400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.397+0000 7f9f94c00640 1 -- 172.21.15.90:0/38061563 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84007c70 con 0x7f9f9810a9b0 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.397+0000 7f9f94c00640 1 -- 172.21.15.90:0/38061563 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f84007e10 con 0x7f9f9810a9b0 2024-09-10T21:52:51.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.397+0000 7f9f94c00640 1 -- 172.21.15.90:0/38061563 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84018710 con 0x7f9f9810a9b0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.397+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f981a1720 con 0x7f9f9810a9b0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.397+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981a7fb0 con 0x7f9f9810a9b0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.398+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f98105380 con 0x7f9f9810a9b0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.400+0000 7f9f94c00640 1 -- 172.21.15.90:0/38061563 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 102354+0+0 (secure 0 0 0) 0x7f9f8400f450 con 0x7f9f9810a9b0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.400+0000 7f9f94c00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9f7c07a400 0x7f9f7c07c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.401+0000 7f9f94c00640 1 -- 172.21.15.90:0/38061563 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9f8409bf60 con 0x7f9f9810a9b0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.401+0000 7f9f97e00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9f7c07a400 0x7f9f7c07c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.401+0000 7f9f97e00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9f7c07a400 0x7f9f7c07c8c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9f88005fd0 tx=0x7f9f88009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.405+0000 7f9f94c00640 1 -- 172.21.15.90:0/38061563 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9f84014030 con 0x7f9f9810a9b0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.598+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f9f9819c4b0 con 0x7f9f9810a9b0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.602+0000 7f9f94c00640 1 -- 172.21.15.90:0/38061563 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v26)=0 v26) ==== 120+0+0 (secure 0 0 0) 0x7f9f840644d0 con 0x7f9f9810a9b0 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9f7c07a400 msgr2=0x7f9f7c07c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9f7c07a400 0x7f9f7c07c8c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9f88005fd0 tx=0x7f9f88009500 comp rx=0 tx=0).stop 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810a9b0 msgr2=0x7f9f9819b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810a9b0 0x7f9f9819b8c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400c920 tx=0x7f9f8400cde0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 shutdown_connections 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9f7c07a400 0x7f9f7c07c8c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f9810b330 0x7f9f981a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f9810a9b0 0x7f9f9819b8c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 --2- 172.21.15.90:0/38061563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f98104fa0 0x7f9f9819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.604+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 >> 172.21.15.90:0/38061563 conn(0x7f9f98100a50 msgr2=0x7f9f98102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.605+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 shutdown_connections 2024-09-10T21:52:51.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:51.605+0000 7f9f9ee00640 1 -- 172.21.15.90:0/38061563 wait complete. 2024-09-10T21:52:52.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.233+0000 7fde3d400640 1 -- 172.21.15.90:0/1258773264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 msgr2=0x7fde38111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:52.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.233+0000 7fde3d400640 1 --2- 172.21.15.90:0/1258773264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 0x7fde38111980 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fde30009900 tx=0x7fde3002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.233+0000 7fde3d400640 1 -- 172.21.15.90:0/1258773264 shutdown_connections 2024-09-10T21:52:52.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.233+0000 7fde3d400640 1 --2- 172.21.15.90:0/1258773264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 0x7fde38111980 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.233+0000 7fde3d400640 1 --2- 172.21.15.90:0/1258773264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde3810a800 0x7fde3810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.233+0000 7fde3d400640 1 --2- 172.21.15.90:0/1258773264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde38104df0 0x7fde381051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.233+0000 7fde3d400640 1 -- 172.21.15.90:0/1258773264 >> 172.21.15.90:0/1258773264 conn(0x7fde38100840 msgr2=0x7fde38102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.233+0000 7fde3d400640 1 -- 172.21.15.90:0/1258773264 shutdown_connections 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.233+0000 7fde3d400640 1 -- 172.21.15.90:0/1258773264 wait complete. 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.234+0000 7fde3d400640 1 Processor -- start 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.234+0000 7fde3d400640 1 -- start start 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.234+0000 7fde3d400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde38104df0 0x7fde3819afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.234+0000 7fde3d400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde3810a800 0x7fde3819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.234+0000 7fde3d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 0x7fde381a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.234+0000 7fde3d400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde38114bd0 con 0x7fde3810b180 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.234+0000 7fde3d400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde38114a50 con 0x7fde38104df0 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.234+0000 7fde3d400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde38114d50 con 0x7fde3810a800 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.234+0000 7fde3ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 0x7fde381a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde3ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 0x7fde381a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60500/0 (socket says 172.21.15.90:60500) 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde2f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde3810a800 0x7fde3819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:52.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde3ca00640 1 -- 172.21.15.90:0/3062260927 learned_addr learned my addr 172.21.15.90:0/3062260927 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde37e00640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde38104df0 0x7fde3819afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde3ca00640 1 -- 172.21.15.90:0/3062260927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde3810a800 msgr2=0x7fde3819b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde3ca00640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde3810a800 0x7fde3819b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde3ca00640 1 -- 172.21.15.90:0/3062260927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde38104df0 msgr2=0x7fde3819afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde3ca00640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde38104df0 0x7fde3819afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde3ca00640 1 -- 172.21.15.90:0/3062260927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde381a1090 con 0x7fde3810b180 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde37e00640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde38104df0 0x7fde3819afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde2f400640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde3810a800 0x7fde3819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.235+0000 7fde3ca00640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 0x7fde381a08c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde30005ab0 tx=0x7fde30003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.236+0000 7fde35600640 1 -- 172.21.15.90:0/3062260927 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3003f070 con 0x7fde3810b180 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.236+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde381a1320 con 0x7fde3810b180 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.236+0000 7fde35600640 1 -- 172.21.15.90:0/3062260927 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fde3003bae0 con 0x7fde3810b180 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.236+0000 7fde35600640 1 -- 172.21.15.90:0/3062260927 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde30035030 con 0x7fde3810b180 2024-09-10T21:52:52.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.236+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde381a7a40 con 0x7fde3810b180 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.237+0000 7fde35600640 1 -- 172.21.15.90:0/3062260927 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7fde3003b670 con 0x7fde3810b180 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.237+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde381051d0 con 0x7fde3810b180 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.237+0000 7fde35600640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde1c03f610 0x7fde1c041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.237+0000 7fde35600640 1 -- 172.21.15.90:0/3062260927 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fde30074850 con 0x7fde3810b180 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.238+0000 7fde37e00640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde1c03f610 0x7fde1c041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.238+0000 7fde37e00640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde1c03f610 0x7fde1c041ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde28005fd0 tx=0x7fde28009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.242+0000 7fde35600640 1 -- 172.21.15.90:0/3062260927 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fde3003bc90 con 0x7fde3810b180 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.432+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fde381a7c10 con 0x7fde3810b180 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.436+0000 7fde35600640 1 -- 172.21.15.90:0/3062260927 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v19)=0 v19) ==== 117+0+0 (secure 0 0 0) 0x7fde30076cc0 con 0x7fde3810b180 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.438+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde1c03f610 msgr2=0x7fde1c041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.438+0000 7fde3d400640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde1c03f610 0x7fde1c041ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde28005fd0 tx=0x7fde28009500 comp rx=0 tx=0).stop 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.438+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 msgr2=0x7fde381a08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.438+0000 7fde3d400640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 0x7fde381a08c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde30005ab0 tx=0x7fde30003c10 comp rx=0 tx=0).stop 2024-09-10T21:52:52.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.439+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 shutdown_connections 2024-09-10T21:52:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.439+0000 7fde3d400640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde1c03f610 0x7fde1c041ad0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.439+0000 7fde3d400640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde3810b180 0x7fde381a08c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.439+0000 7fde3d400640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde3810a800 0x7fde3819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.439+0000 7fde3d400640 1 --2- 172.21.15.90:0/3062260927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde38104df0 0x7fde3819afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.439+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 >> 172.21.15.90:0/3062260927 conn(0x7fde38100840 msgr2=0x7fde38102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.439+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 shutdown_connections 2024-09-10T21:52:52.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.439+0000 7fde3d400640 1 -- 172.21.15.90:0/3062260927 wait complete. 2024-09-10T21:52:52.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.847+0000 7fe10a800640 1 -- 172.21.15.90:0/700892690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410a800 msgr2=0x7fe10410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:52.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.847+0000 7fe10a800640 1 --2- 172.21.15.90:0/700892690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410a800 0x7fe10410ac40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009900 tx=0x7fe0f002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.847+0000 7fe10a800640 1 -- 172.21.15.90:0/700892690 shutdown_connections 2024-09-10T21:52:52.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.847+0000 7fe10a800640 1 --2- 172.21.15.90:0/700892690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10410b180 0x7fe104111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.847+0000 7fe10a800640 1 --2- 172.21.15.90:0/700892690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410a800 0x7fe10410ac40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.847+0000 7fe10a800640 1 --2- 172.21.15.90:0/700892690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe104104df0 0x7fe1041051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.847+0000 7fe10a800640 1 -- 172.21.15.90:0/700892690 >> 172.21.15.90:0/700892690 conn(0x7fe104100840 msgr2=0x7fe104102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:52.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.847+0000 7fe10a800640 1 -- 172.21.15.90:0/700892690 shutdown_connections 2024-09-10T21:52:52.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.847+0000 7fe10a800640 1 -- 172.21.15.90:0/700892690 wait complete. 2024-09-10T21:52:52.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.848+0000 7fe10a800640 1 Processor -- start 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.848+0000 7fe10a800640 1 -- start start 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe10a800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe104104df0 0x7fe10419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe10a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10410a800 0x7fe10419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe10a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410b180 0x7fe1041a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe10a800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe104114d40 con 0x7fe10410b180 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe10a800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe104114bc0 con 0x7fe104104df0 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe10a800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe104114ec0 con 0x7fe10410a800 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe109e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410b180 0x7fe1041a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe109400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe104104df0 0x7fe10419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe109e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410b180 0x7fe1041a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56430/0 (socket says 172.21.15.90:56430) 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe109e00640 1 -- 172.21.15.90:0/634668008 learned_addr learned my addr 172.21.15.90:0/634668008 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:52.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.849+0000 7fe108a00640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10410a800 0x7fe10419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:52.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe109e00640 1 -- 172.21.15.90:0/634668008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10410a800 msgr2=0x7fe10419b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:52.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe109e00640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10410a800 0x7fe10419b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe109e00640 1 -- 172.21.15.90:0/634668008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe104104df0 msgr2=0x7fe10419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:52.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe109e00640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe104104df0 0x7fe10419b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:52.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe109e00640 1 -- 172.21.15.90:0/634668008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1041a1160 con 0x7fe10410b180 2024-09-10T21:52:52.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe109e00640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410b180 0x7fe1041a0990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00cce0 tx=0x7fe0fc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:52.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe102000640 1 -- 172.21.15.90:0/634668008 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc00f450 con 0x7fe10410b180 2024-09-10T21:52:52.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe102000640 1 -- 172.21.15.90:0/634668008 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0fc00fa80 con 0x7fe10410b180 2024-09-10T21:52:52.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1041a1450 con 0x7fe10410b180 2024-09-10T21:52:52.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe102000640 1 -- 172.21.15.90:0/634668008 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc018ad0 con 0x7fe10410b180 2024-09-10T21:52:52.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.850+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1041a7b10 con 0x7fe10410b180 2024-09-10T21:52:52.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.852+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1041051d0 con 0x7fe10410b180 2024-09-10T21:52:52.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:52 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/38061563' entity='client.admin' 2024-09-10T21:52:52.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:52 smithi090 ceph-mon[30350]: Standby manager daemon y started 2024-09-10T21:52:52.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:52 smithi090 ceph-mon[30350]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T21:52:52.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:52 smithi090 ceph-mon[30350]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T21:52:52.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:52 smithi090 ceph-mon[30350]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T21:52:52.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:52 smithi090 ceph-mon[30350]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T21:52:52.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.854+0000 7fe102000640 1 -- 172.21.15.90:0/634668008 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fe0fc00f5f0 con 0x7fe10410b180 2024-09-10T21:52:52.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.855+0000 7fe102000640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b53c0 0x7fe0ec0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:52.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.856+0000 7fe102000640 1 -- 172.21.15.90:0/634668008 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe0fc0e46d0 con 0x7fe10410b180 2024-09-10T21:52:52.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.856+0000 7fe109400640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b53c0 0x7fe0ec0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:52.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.856+0000 7fe109400640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b53c0 0x7fe0ec0b7880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005d00 tx=0x7fe0f800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:52.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:52.859+0000 7fe102000640 1 -- 172.21.15.90:0/634668008 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe0fc014030 con 0x7fe10410b180 2024-09-10T21:52:52.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:52 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/38061563' entity='client.admin' 2024-09-10T21:52:52.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:52 smithi154 ceph-mon[34495]: Standby manager daemon y started 2024-09-10T21:52:52.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:52 smithi154 ceph-mon[34495]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T21:52:52.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:52 smithi154 ceph-mon[34495]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T21:52:52.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:52 smithi154 ceph-mon[34495]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T21:52:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:52 smithi154 ceph-mon[34495]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T21:52:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:52 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/38061563' entity='client.admin' 2024-09-10T21:52:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:52 smithi171 ceph-mon[34138]: Standby manager daemon y started 2024-09-10T21:52:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:52 smithi171 ceph-mon[34138]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T21:52:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:52 smithi171 ceph-mon[34138]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T21:52:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:52 smithi171 ceph-mon[34138]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T21:52:52.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:52 smithi171 ceph-mon[34138]: from='mgr.? 172.21.15.154:0/3109239244' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T21:52:53.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.052+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fe1041a7e30 con 0x7fe10410b180 2024-09-10T21:52:53.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.057+0000 7fe102000640 1 -- 172.21.15.90:0/634668008 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7fe0fc0ad0f0 con 0x7fe10410b180 2024-09-10T21:52:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.059+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b53c0 msgr2=0x7fe0ec0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.059+0000 7fe10a800640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b53c0 0x7fe0ec0b7880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8005d00 tx=0x7fe0f800b410 comp rx=0 tx=0).stop 2024-09-10T21:52:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.059+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410b180 msgr2=0x7fe1041a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.059+0000 7fe10a800640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410b180 0x7fe1041a0990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00cce0 tx=0x7fe0fc00a620 comp rx=0 tx=0).stop 2024-09-10T21:52:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.060+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 shutdown_connections 2024-09-10T21:52:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.060+0000 7fe10a800640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b53c0 0x7fe0ec0b7880 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.060+0000 7fe10a800640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe10410b180 0x7fe1041a0990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.060+0000 7fe10a800640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10410a800 0x7fe10419b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.060+0000 7fe10a800640 1 --2- 172.21.15.90:0/634668008 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe104104df0 0x7fe10419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.060+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 >> 172.21.15.90:0/634668008 conn(0x7fe104100840 msgr2=0x7fe104103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.060+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 shutdown_connections 2024-09-10T21:52:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.060+0000 7fe10a800640 1 -- 172.21.15.90:0/634668008 wait complete. 2024-09-10T21:52:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:52:53.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:53 smithi090 ceph-mon[30350]: mgrmap e19: x(active, since 19s), standbys: z, y 2024-09-10T21:52:53.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T21:52:53.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/634668008' entity='client.admin' 2024-09-10T21:52:53.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/207096239' entity='client.admin' 2024-09-10T21:52:53.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.734+0000 7fca37200640 1 -- 172.21.15.90:0/282172280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38102020 msgr2=0x7fca3810ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:53.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.734+0000 7fca37200640 1 --2- 172.21.15.90:0/282172280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38102020 0x7fca3810ed70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca24009900 tx=0x7fca2402a480 comp rx=0 tx=0).stop 2024-09-10T21:52:53.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.734+0000 7fca37200640 1 -- 172.21.15.90:0/282172280 shutdown_connections 2024-09-10T21:52:53.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.734+0000 7fca37200640 1 --2- 172.21.15.90:0/282172280 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca3810f2b0 0x7fca381116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.734+0000 7fca37200640 1 --2- 172.21.15.90:0/282172280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38102020 0x7fca3810ed70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.734+0000 7fca37200640 1 --2- 172.21.15.90:0/282172280 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca38101700 0x7fca38101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.734+0000 7fca37200640 1 -- 172.21.15.90:0/282172280 >> 172.21.15.90:0/282172280 conn(0x7fca380fd5b0 msgr2=0x7fca380ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.734+0000 7fca37200640 1 -- 172.21.15.90:0/282172280 shutdown_connections 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.734+0000 7fca37200640 1 -- 172.21.15.90:0/282172280 wait complete. 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.735+0000 7fca37200640 1 Processor -- start 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.735+0000 7fca37200640 1 -- start start 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.735+0000 7fca37200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38101700 0x7fca3819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.735+0000 7fca37200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca38102020 0x7fca3819b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca37200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca3810f2b0 0x7fca381a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca37200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca38114bd0 con 0x7fca38101700 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca37200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca38114a50 con 0x7fca38102020 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca37200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca38114d50 con 0x7fca3810f2b0 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38101700 0x7fca3819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:53.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38101700 0x7fca3819b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60512/0 (socket says 172.21.15.90:60512) 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35e00640 1 -- 172.21.15.90:0/4220734422 learned_addr learned my addr 172.21.15.90:0/4220734422 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35400640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca38102020 0x7fca3819b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca36800640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca3810f2b0 0x7fca381a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35e00640 1 -- 172.21.15.90:0/4220734422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca3810f2b0 msgr2=0x7fca381a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35e00640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca3810f2b0 0x7fca381a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35e00640 1 -- 172.21.15.90:0/4220734422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca38102020 msgr2=0x7fca3819b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35e00640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca38102020 0x7fca3819b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35e00640 1 -- 172.21.15.90:0/4220734422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca381a1170 con 0x7fca38101700 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca36800640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca3810f2b0 0x7fca381a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.736+0000 7fca35e00640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38101700 0x7fca3819b0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fca2800e9f0 tx=0x7fca2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.737+0000 7fca2ea00640 1 -- 172.21.15.90:0/4220734422 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2800cd80 con 0x7fca38101700 2024-09-10T21:52:53.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.737+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca381a1460 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.737+0000 7fca2ea00640 1 -- 172.21.15.90:0/4220734422 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fca28005490 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.737+0000 7fca2ea00640 1 -- 172.21.15.90:0/4220734422 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28010430 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.737+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca381a7b20 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.738+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca38101ae0 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.738+0000 7fca2ea00640 1 -- 172.21.15.90:0/4220734422 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7fca28005020 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.739+0000 7fca2ea00640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca1c03f610 0x7fca1c041ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.739+0000 7fca2ea00640 1 -- 172.21.15.90:0/4220734422 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fca280543a0 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.739+0000 7fca35400640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca1c03f610 0x7fca1c041ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.739+0000 7fca35400640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca1c03f610 0x7fca1c041ad0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fca24005ab0 tx=0x7fca24005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.743+0000 7fca2ea00640 1 -- 172.21.15.90:0/4220734422 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fca28004770 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.937+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7fca3810a440 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.942+0000 7fca2ea00640 1 -- 172.21.15.90:0/4220734422 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v20)=0 v20) ==== 116+0+0 (secure 0 0 0) 0x7fca2800fa00 con 0x7fca38101700 2024-09-10T21:52:53.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.944+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca1c03f610 msgr2=0x7fca1c041ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.944+0000 7fca37200640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca1c03f610 0x7fca1c041ad0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fca24005ab0 tx=0x7fca24005a20 comp rx=0 tx=0).stop 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38101700 msgr2=0x7fca3819b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38101700 0x7fca3819b0c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fca2800e9f0 tx=0x7fca2800edf0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 shutdown_connections 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca1c03f610 0x7fca1c041ad0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca3810f2b0 0x7fca381a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca38102020 0x7fca3819b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 --2- 172.21.15.90:0/4220734422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca38101700 0x7fca3819b0c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 >> 172.21.15.90:0/4220734422 conn(0x7fca380fd5b0 msgr2=0x7fca380ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 shutdown_connections 2024-09-10T21:52:53.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:46.945+0000 7fca37200640 1 -- 172.21.15.90:0/4220734422 wait complete. 2024-09-10T21:52:53.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:53 smithi154 ceph-mon[34495]: mgrmap e19: x(active, since 19s), standbys: z, y 2024-09-10T21:52:53.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T21:52:53.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/634668008' entity='client.admin' 2024-09-10T21:52:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/207096239' entity='client.admin' 2024-09-10T21:52:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:53 smithi171 ceph-mon[34138]: mgrmap e19: x(active, since 19s), standbys: z, y 2024-09-10T21:52:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T21:52:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/634668008' entity='client.admin' 2024-09-10T21:52:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/207096239' entity='client.admin' 2024-09-10T21:52:54.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.778+0000 7f5cf4800640 1 -- 172.21.15.90:0/3928157290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec10aa70 msgr2=0x7f5cec1178c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.778+0000 7f5cf4800640 1 --2- 172.21.15.90:0/3928157290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec10aa70 0x7f5cec1178c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009900 tx=0x7f5cdc02a480 comp rx=0 tx=0).stop 2024-09-10T21:52:54.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.778+0000 7f5cf4800640 1 -- 172.21.15.90:0/3928157290 shutdown_connections 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.778+0000 7f5cf4800640 1 --2- 172.21.15.90:0/3928157290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cec117e00 0x7f5cec11a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.778+0000 7f5cf4800640 1 --2- 172.21.15.90:0/3928157290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec10aa70 0x7f5cec1178c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.778+0000 7f5cf4800640 1 --2- 172.21.15.90:0/3928157290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cec10a150 0x7f5cec10a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.778+0000 7f5cf4800640 1 -- 172.21.15.90:0/3928157290 >> 172.21.15.90:0/3928157290 conn(0x7f5cec06e2e0 msgr2=0x7f5cec06e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.778+0000 7f5cf4800640 1 -- 172.21.15.90:0/3928157290 shutdown_connections 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.778+0000 7f5cf4800640 1 -- 172.21.15.90:0/3928157290 wait complete. 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.779+0000 7f5cf4800640 1 Processor -- start 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.779+0000 7f5cf4800640 1 -- start start 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf4800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cec10a150 0x7f5cec1a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf4800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cec10aa70 0x7f5cec1a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf4800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec117e00 0x7f5cec1a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf4800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec11d660 con 0x7f5cec117e00 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf4800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec11d4e0 con 0x7f5cec10aa70 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf4800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5cec11d7e0 con 0x7f5cec10a150 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec117e00 0x7f5cec1a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec117e00 0x7f5cec1a94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56454/0 (socket says 172.21.15.90:56454) 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf2800640 1 -- 172.21.15.90:0/1499751035 learned_addr learned my addr 172.21.15.90:0/1499751035 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf1e00640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cec10a150 0x7f5cec1a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf1400640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cec10aa70 0x7f5cec1a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf2800640 1 -- 172.21.15.90:0/1499751035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cec10a150 msgr2=0x7f5cec1a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf2800640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cec10a150 0x7f5cec1a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf2800640 1 -- 172.21.15.90:0/1499751035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cec10aa70 msgr2=0x7f5cec1a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf2800640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cec10aa70 0x7f5cec1a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.780+0000 7f5cf2800640 1 -- 172.21.15.90:0/1499751035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cec1a9c80 con 0x7f5cec117e00 2024-09-10T21:52:54.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.781+0000 7f5cf2800640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec117e00 0x7f5cec1a94b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009fd0 tx=0x7f5ce800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.781+0000 7f5ce6a00640 1 -- 172.21.15.90:0/1499751035 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8017070 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.781+0000 7f5ce6a00640 1 -- 172.21.15.90:0/1499751035 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5ce8003e00 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.781+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cec1a9f70 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.781+0000 7f5ce6a00640 1 -- 172.21.15.90:0/1499751035 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8010460 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.781+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cec1b0630 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.782+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cec10a530 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.784+0000 7f5ce6a00640 1 -- 172.21.15.90:0/1499751035 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f5ce8003970 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.786+0000 7f5ce6a00640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cd40b5320 0x7f5cd40b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.786+0000 7f5ce6a00640 1 -- 172.21.15.90:0/1499751035 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5ce80e4180 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.786+0000 7f5cf1e00640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cd40b5320 0x7f5cd40b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.787+0000 7f5cf1e00640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cd40b5320 0x7f5cd40b77e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000bbe0 tx=0x7f5ce000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.787+0000 7f5ce6a00640 1 -- 172.21.15.90:0/1499751035 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5ce80acc20 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.975+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f5cec1b0930 con 0x7f5cec117e00 2024-09-10T21:52:54.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.980+0000 7f5ce6a00640 1 -- 172.21.15.90:0/1499751035 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v29)=0 v29) ==== 120+0+0 (secure 0 0 0) 0x7f5ce80b53d0 con 0x7f5cec117e00 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cd40b5320 msgr2=0x7f5cd40b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cd40b5320 0x7f5cd40b77e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000bbe0 tx=0x7f5ce000b390 comp rx=0 tx=0).stop 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec117e00 msgr2=0x7f5cec1a94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec117e00 0x7f5cec1a94b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009fd0 tx=0x7f5ce800b260 comp rx=0 tx=0).stop 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 shutdown_connections 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cd40b5320 0x7f5cd40b77e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cec117e00 0x7f5cec1a94b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cec10aa70 0x7f5cec1a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 --2- 172.21.15.90:0/1499751035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cec10a150 0x7f5cec1a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 >> 172.21.15.90:0/1499751035 conn(0x7f5cec06e2e0 msgr2=0x7f5cec118410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 shutdown_connections 2024-09-10T21:52:54.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.982+0000 7f5cf4800640 1 -- 172.21.15.90:0/1499751035 wait complete. 2024-09-10T21:52:54.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:54 smithi154 ceph-mon[34495]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:54.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:54 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1499751035' entity='client.admin' 2024-09-10T21:52:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:54 smithi171 ceph-mon[34138]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:54 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1499751035' entity='client.admin' 2024-09-10T21:52:55.011 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:54 smithi090 ceph-mon[30350]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:55.011 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:54 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1499751035' entity='client.admin' 2024-09-10T21:52:56.229 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:56 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2902945696' entity='client.admin' 2024-09-10T21:52:56.229 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:56 smithi090 ceph-mon[30350]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:56.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:56 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2902945696' entity='client.admin' 2024-09-10T21:52:56.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:56 smithi154 ceph-mon[34495]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:56.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:56 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2902945696' entity='client.admin' 2024-09-10T21:52:56.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:56 smithi171 ceph-mon[34138]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:57.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.809+0000 7ff6fa000640 1 -- 172.21.15.90:0/824307637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6f410b330 msgr2=0x7ff6f4111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:57.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.809+0000 7ff6fa000640 1 --2- 172.21.15.90:0/824307637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6f410b330 0x7ff6f4111b30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec009900 tx=0x7ff6ec02a410 comp rx=0 tx=0).stop 2024-09-10T21:52:57.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.809+0000 7ff6fa000640 1 -- 172.21.15.90:0/824307637 shutdown_connections 2024-09-10T21:52:57.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.809+0000 7ff6fa000640 1 --2- 172.21.15.90:0/824307637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6f410b330 0x7ff6f4111b30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:57.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.809+0000 7ff6fa000640 1 --2- 172.21.15.90:0/824307637 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f410a9b0 0x7ff6f410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:57.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.809+0000 7ff6fa000640 1 --2- 172.21.15.90:0/824307637 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f4104fa0 0x7ff6f4105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:57.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.809+0000 7ff6fa000640 1 -- 172.21.15.90:0/824307637 >> 172.21.15.90:0/824307637 conn(0x7ff6f4100a50 msgr2=0x7ff6f4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.809+0000 7ff6fa000640 1 -- 172.21.15.90:0/824307637 shutdown_connections 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.809+0000 7ff6fa000640 1 -- 172.21.15.90:0/824307637 wait complete. 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.810+0000 7ff6fa000640 1 Processor -- start 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.810+0000 7ff6fa000640 1 -- start start 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.810+0000 7ff6fa000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f4104fa0 0x7ff6f419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.810+0000 7ff6fa000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f410a9b0 0x7ff6f419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.810+0000 7ff6fa000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6f410b330 0x7ff6f41a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.810+0000 7ff6fa000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4114e60 con 0x7ff6f410b330 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.810+0000 7ff6fa000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4114ce0 con 0x7ff6f410a9b0 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.810+0000 7ff6fa000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f4114fe0 con 0x7ff6f4104fa0 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f410a9b0 0x7ff6f419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:57.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6f410b330 0x7ff6f41a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:57.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f410a9b0 0x7ff6f419b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:51420/0 (socket says 172.21.15.90:51420) 2024-09-10T21:52:57.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f2a00640 1 -- 172.21.15.90:0/179318934 learned_addr learned my addr 172.21.15.90:0/179318934 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:57.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f3400640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f4104fa0 0x7ff6f419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:57.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f2a00640 1 -- 172.21.15.90:0/179318934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f4104fa0 msgr2=0x7ff6f419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:57.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f2a00640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f4104fa0 0x7ff6f419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:57.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f2a00640 1 -- 172.21.15.90:0/179318934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6f410b330 msgr2=0x7ff6f41a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:57.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f2a00640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6f410b330 0x7ff6f41a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:57.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f2a00640 1 -- 172.21.15.90:0/179318934 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f41a1420 con 0x7ff6f410a9b0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.811+0000 7ff6f2a00640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f410a9b0 0x7ff6f419b8b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000d950 tx=0x7ff6e000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.812+0000 7ff6ebe00640 1 -- 172.21.15.90:0/179318934 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0017070 con 0x7ff6f410a9b0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.812+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f41a1710 con 0x7ff6f410a9b0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.812+0000 7ff6ebe00640 1 -- 172.21.15.90:0/179318934 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff6e0009c30 con 0x7ff6f410a9b0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.812+0000 7ff6ebe00640 1 -- 172.21.15.90:0/179318934 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000f620 con 0x7ff6f410a9b0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.812+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f41a7dd0 con 0x7ff6f410a9b0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.813+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f4105380 con 0x7ff6f410a9b0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.813+0000 7ff6ebe00640 1 -- 172.21.15.90:0/179318934 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 17) ==== 51497+0+0 (secure 0 0 0) 0x7ff6e0010460 con 0x7ff6f410a9b0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.814+0000 7ff6ebe00640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6d803f560 0x7ff6d8041a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.814+0000 7ff6ebe00640 1 -- 172.21.15.90:0/179318934 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff6e0053b20 con 0x7ff6f410a9b0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.814+0000 7ff6f3400640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6d803f560 0x7ff6d8041a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:57.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.814+0000 7ff6f3400640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6d803f560 0x7ff6d8041a20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4005fd0 tx=0x7ff6e4009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:49.819+0000 7ff6ebe00640 1 -- 172.21.15.90:0/179318934 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff6e0015d40 con 0x7ff6f410a9b0 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.019+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7ff6f402e5c0 con 0x7ff6f410a9b0 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.025+0000 7ff6ebe00640 1 -- 172.21.15.90:0/179318934 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7ff6e000fce0 con 0x7ff6f410a9b0 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6d803f560 msgr2=0x7ff6d8041a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6d803f560 0x7ff6d8041a20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4005fd0 tx=0x7ff6e4009500 comp rx=0 tx=0).stop 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f410a9b0 msgr2=0x7ff6f419b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f410a9b0 0x7ff6f419b8b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000d950 tx=0x7ff6e000de10 comp rx=0 tx=0).stop 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 shutdown_connections 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6d803f560 0x7ff6d8041a20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:57.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff6f410b330 0x7ff6f41a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:57.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff6f410a9b0 0x7ff6f419b8b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:57.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 --2- 172.21.15.90:0/179318934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff6f4104fa0 0x7ff6f419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:57.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 >> 172.21.15.90:0/179318934 conn(0x7ff6f4100a50 msgr2=0x7ff6f4102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:57.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 shutdown_connections 2024-09-10T21:52:57.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:50.028+0000 7ff6fa000640 1 -- 172.21.15.90:0/179318934 wait complete. 2024-09-10T21:52:57.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:57 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1637676117' entity='client.admin' 2024-09-10T21:52:57.474 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:57 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1637676117' entity='client.admin' 2024-09-10T21:52:57.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:57 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1637676117' entity='client.admin' 2024-09-10T21:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.808+0000 7f552ea00640 1 -- 172.21.15.90:0/2355578355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 msgr2=0x7f55281053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.808+0000 7f552ea00640 1 --2- 172.21.15.90:0/2355578355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 0x7f55281053a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5518009ca0 tx=0x7f551802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.808+0000 7f552ea00640 1 -- 172.21.15.90:0/2355578355 shutdown_connections 2024-09-10T21:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.808+0000 7f552ea00640 1 --2- 172.21.15.90:0/2355578355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f552810b350 0x7f5528111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.808+0000 7f552ea00640 1 --2- 172.21.15.90:0/2355578355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f552810a9d0 0x7f552810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.808+0000 7f552ea00640 1 --2- 172.21.15.90:0/2355578355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 0x7f55281053a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.808+0000 7f552ea00640 1 -- 172.21.15.90:0/2355578355 >> 172.21.15.90:0/2355578355 conn(0x7f55281009f0 msgr2=0x7f5528102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:58.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.808+0000 7f552ea00640 1 -- 172.21.15.90:0/2355578355 shutdown_connections 2024-09-10T21:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.808+0000 7f552ea00640 1 -- 172.21.15.90:0/2355578355 wait complete. 2024-09-10T21:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f552ea00640 1 Processor -- start 2024-09-10T21:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f552ea00640 1 -- start start 2024-09-10T21:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f552ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 0x7f552819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f552ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f552810a9d0 0x7f552819b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f552ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f552810b350 0x7f55281a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f552ea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5528114f50 con 0x7f5528104fc0 2024-09-10T21:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f552ea00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5528114dd0 con 0x7f552810a9d0 2024-09-10T21:52:58.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f552ea00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f55281150d0 con 0x7f552810b350 2024-09-10T21:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f5527e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 0x7f552819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.809+0000 7f5527e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 0x7f552819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56466/0 (socket says 172.21.15.90:56466) 2024-09-10T21:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f5527e00640 1 -- 172.21.15.90:0/2902945696 learned_addr learned my addr 172.21.15.90:0/2902945696 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f551f400640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f552810a9d0 0x7f552819b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f552ca00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f552810b350 0x7f55281a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f5527e00640 1 -- 172.21.15.90:0/2902945696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f552810b350 msgr2=0x7f55281a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f5527e00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f552810b350 0x7f55281a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f5527e00640 1 -- 172.21.15.90:0/2902945696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f552810a9d0 msgr2=0x7f552819b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:58.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f5527e00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f552810a9d0 0x7f552819b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f5527e00640 1 -- 172.21.15.90:0/2902945696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55281a1510 con 0x7f5528104fc0 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f552ca00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f552810b350 0x7f55281a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f5527e00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 0x7f552819b460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5518006010 tx=0x7f5518032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.810+0000 7f5525600640 1 -- 172.21.15.90:0/2902945696 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518004040 con 0x7f5528104fc0 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.811+0000 7f5525600640 1 -- 172.21.15.90:0/2902945696 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5518004670 con 0x7f5528104fc0 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.811+0000 7f5525600640 1 -- 172.21.15.90:0/2902945696 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518030da0 con 0x7f5528104fc0 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.811+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55281a17a0 con 0x7f5528104fc0 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.811+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55281a7ec0 con 0x7f5528104fc0 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.812+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55281053a0 con 0x7f5528104fc0 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.814+0000 7f5525600640 1 -- 172.21.15.90:0/2902945696 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f55180041e0 con 0x7f5528104fc0 2024-09-10T21:52:58.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.815+0000 7f5525600640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f550c0b5320 0x7f550c0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.816+0000 7f5525600640 1 -- 172.21.15.90:0/2902945696 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5518105660 con 0x7f5528104fc0 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.816+0000 7f551f400640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f550c0b5320 0x7f550c0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.816+0000 7f551f400640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f550c0b5320 0x7f550c0b77e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f552810a100 tx=0x7f5510008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:54.819+0000 7f5525600640 1 -- 172.21.15.90:0/2902945696 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5518004820 con 0x7f5528104fc0 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.009+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f552802e5c0 con 0x7f5528104fc0 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.014+0000 7f5525600640 1 -- 172.21.15.90:0/2902945696 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7f55180ce080 con 0x7f5528104fc0 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f550c0b5320 msgr2=0x7f550c0b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f550c0b5320 0x7f550c0b77e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f552810a100 tx=0x7f5510008040 comp rx=0 tx=0).stop 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 msgr2=0x7f552819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:58.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 0x7f552819b460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5518006010 tx=0x7f5518032bc0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 shutdown_connections 2024-09-10T21:52:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f550c0b5320 0x7f550c0b77e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f552810b350 0x7f55281a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f552810a9d0 0x7f552819b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 --2- 172.21.15.90:0/2902945696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5528104fc0 0x7f552819b460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 >> 172.21.15.90:0/2902945696 conn(0x7f55281009f0 msgr2=0x7f5528102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 shutdown_connections 2024-09-10T21:52:58.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.016+0000 7f552ea00640 1 -- 172.21.15.90:0/2902945696 wait complete. 2024-09-10T21:52:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:58 smithi171 ceph-mon[34138]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:58 smithi171 ceph-mon[34138]: Reconfiguring mon.c (monmap changed)... 2024-09-10T21:52:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:58.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:52:58 smithi171 ceph-mon[34138]: Reconfiguring daemon mon.c on smithi171 2024-09-10T21:52:58.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:58 smithi090 ceph-mon[30350]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:58.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:58.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:58.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:58 smithi090 ceph-mon[30350]: Reconfiguring mon.c (monmap changed)... 2024-09-10T21:52:58.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:58.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:58.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:58.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:52:58 smithi090 ceph-mon[30350]: Reconfiguring daemon mon.c on smithi171 2024-09-10T21:52:58.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:58 smithi154 ceph-mon[34495]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:52:58.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:58.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:52:58.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:58 smithi154 ceph-mon[34495]: Reconfiguring mon.c (monmap changed)... 2024-09-10T21:52:58.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T21:52:58.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T21:52:58.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:52:58.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:52:58 smithi154 ceph-mon[34495]: Reconfiguring daemon mon.c on smithi171 2024-09-10T21:52:59.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.949+0000 7f57c9800640 1 -- 172.21.15.90:0/1544688642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4108620 msgr2=0x7f57c4108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.949+0000 7f57c9800640 1 --2- 172.21.15.90:0/1544688642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4108620 0x7f57c4108a60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009900 tx=0x7f57ac02a410 comp rx=0 tx=0).stop 2024-09-10T21:52:59.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.949+0000 7f57c9800640 1 -- 172.21.15.90:0/1544688642 shutdown_connections 2024-09-10T21:52:59.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.949+0000 7f57c9800640 1 --2- 172.21.15.90:0/1544688642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57c4108fa0 0x7f57c410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.949+0000 7f57c9800640 1 --2- 172.21.15.90:0/1544688642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4108620 0x7f57c4108a60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.949+0000 7f57c9800640 1 --2- 172.21.15.90:0/1544688642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c4102c10 0x7f57c4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.949+0000 7f57c9800640 1 -- 172.21.15.90:0/1544688642 >> 172.21.15.90:0/1544688642 conn(0x7f57c40fe660 msgr2=0x7f57c4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.949+0000 7f57c9800640 1 -- 172.21.15.90:0/1544688642 shutdown_connections 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.949+0000 7f57c9800640 1 -- 172.21.15.90:0/1544688642 wait complete. 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.950+0000 7f57c9800640 1 Processor -- start 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.950+0000 7f57c9800640 1 -- start start 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4102c10 0x7f57c40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c9800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57c4108620 0x7f57c40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c9800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c4108fa0 0x7f57c4075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c9800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f57c4112af0 con 0x7f57c4102c10 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c9800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f57c4112970 con 0x7f57c4108fa0 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c9800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f57c4112c70 con 0x7f57c4108620 2024-09-10T21:52:59.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4102c10 0x7f57c40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4102c10 0x7f57c40754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56494/0 (socket says 172.21.15.90:56494) 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c3e00640 1 -- 172.21.15.90:0/1637676117 learned_addr learned my addr 172.21.15.90:0/1637676117 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57bbe00640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57c4108620 0x7f57c40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c8e00640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c4108fa0 0x7f57c4075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.951+0000 7f57c3e00640 1 -- 172.21.15.90:0/1637676117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57c4108620 msgr2=0x7f57c40759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c3e00640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57c4108620 0x7f57c40759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c3e00640 1 -- 172.21.15.90:0/1637676117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c4108fa0 msgr2=0x7f57c4075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c3e00640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c4108fa0 0x7f57c4075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c3e00640 1 -- 172.21.15.90:0/1637676117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c41a7360 con 0x7f57c4102c10 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c3e00640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4102c10 0x7f57c40754a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57b400d950 tx=0x7f57b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c1600640 1 -- 172.21.15.90:0/1637676117 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b4017030 con 0x7f57c4102c10 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c1600640 1 -- 172.21.15.90:0/1637676117 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f57b4004020 con 0x7f57c4102c10 2024-09-10T21:52:59.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c1600640 1 -- 172.21.15.90:0/1637676117 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b40055d0 con 0x7f57c4102c10 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c41a7650 con 0x7f57c4102c10 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.952+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c41a7b60 con 0x7f57c4102c10 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.953+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c4102ff0 con 0x7f57c4102c10 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.957+0000 7f57c1600640 1 -- 172.21.15.90:0/1637676117 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f57b4009a30 con 0x7f57c4102c10 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.959+0000 7f57c1600640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57a80b5340 0x7f57a80b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.959+0000 7f57c1600640 1 -- 172.21.15.90:0/1637676117 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f57b40e40b0 con 0x7f57c4102c10 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.959+0000 7f57bbe00640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57a80b5340 0x7f57a80b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.959+0000 7f57c1600640 1 -- 172.21.15.90:0/1637676117 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f57b40e44e0 con 0x7f57c4102c10 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:55.960+0000 7f57bbe00640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57a80b5340 0x7f57a80b7800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f57ac006010 tx=0x7f57ac005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.158+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f57c40767a0 con 0x7f57c4102c10 2024-09-10T21:52:59.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.162+0000 7f57c1600640 1 -- 172.21.15.90:0/1637676117 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v31)=0 v31) ==== 116+0+0 (secure 0 0 0) 0x7f57b40acad0 con 0x7f57c4102c10 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.164+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57a80b5340 msgr2=0x7f57a80b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.164+0000 7f57c9800640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57a80b5340 0x7f57a80b7800 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f57ac006010 tx=0x7f57ac005e80 comp rx=0 tx=0).stop 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.164+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4102c10 msgr2=0x7f57c40754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.164+0000 7f57c9800640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4102c10 0x7f57c40754a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57b400d950 tx=0x7f57b400de10 comp rx=0 tx=0).stop 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.165+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 shutdown_connections 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.165+0000 7f57c9800640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57a80b5340 0x7f57a80b7800 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.165+0000 7f57c9800640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f57c4108fa0 0x7f57c4075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.165+0000 7f57c9800640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f57c4108620 0x7f57c40759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.165+0000 7f57c9800640 1 --2- 172.21.15.90:0/1637676117 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f57c4102c10 0x7f57c40754a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.165+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 >> 172.21.15.90:0/1637676117 conn(0x7f57c40fe660 msgr2=0x7f57c4100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.165+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 shutdown_connections 2024-09-10T21:52:59.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:56.165+0000 7f57c9800640 1 -- 172.21.15.90:0/1637676117 wait complete. 2024-09-10T21:52:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.213+0000 7f34b6200640 1 -- 172.21.15.90:0/4158546307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 msgr2=0x7f34b010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.213+0000 7f34b6200640 1 --2- 172.21.15.90:0/4158546307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 0x7f34b010f950 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f34a8009ca0 tx=0x7f34a802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.213+0000 7f34b6200640 1 -- 172.21.15.90:0/4158546307 shutdown_connections 2024-09-10T21:52:59.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.213+0000 7f34b6200640 1 --2- 172.21.15.90:0/4158546307 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 0x7f34b010f950 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.214+0000 7f34b6200640 1 --2- 172.21.15.90:0/4158546307 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34b01087d0 0x7f34b0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.214+0000 7f34b6200640 1 --2- 172.21.15.90:0/4158546307 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34b0102dc0 0x7f34b01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.214+0000 7f34b6200640 1 -- 172.21.15.90:0/4158546307 >> 172.21.15.90:0/4158546307 conn(0x7f34b00fe830 msgr2=0x7f34b0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.214+0000 7f34b6200640 1 -- 172.21.15.90:0/4158546307 shutdown_connections 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.214+0000 7f34b6200640 1 -- 172.21.15.90:0/4158546307 wait complete. 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.214+0000 7f34b6200640 1 Processor -- start 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34b6200640 1 -- start start 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34b6200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34b0102dc0 0x7f34b0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34b6200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34b01087d0 0x7f34b01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34b6200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 0x7f34b019ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34b6200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0112cd0 con 0x7f34b0109150 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34b6200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0112b50 con 0x7f34b01087d0 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34b6200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0112e50 con 0x7f34b0102dc0 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 0x7f34b019ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 0x7f34b019ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56446/0 (socket says 172.21.15.90:56446) 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34afe00640 1 -- 172.21.15.90:0/207096239 learned_addr learned my addr 172.21.15.90:0/207096239 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.215+0000 7f34af400640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34b0102dc0 0x7f34b0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34a7e00640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34b01087d0 0x7f34b01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34afe00640 1 -- 172.21.15.90:0/207096239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34b0102dc0 msgr2=0x7f34b0199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34afe00640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34b0102dc0 0x7f34b0199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34afe00640 1 -- 172.21.15.90:0/207096239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34b01087d0 msgr2=0x7f34b01996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34afe00640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34b01087d0 0x7f34b01996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34afe00640 1 -- 172.21.15.90:0/207096239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b019f210 con 0x7f34b0109150 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34af400640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34b0102dc0 0x7f34b0199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34afe00640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 0x7f34b019ea40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005e30 tx=0x7f34a80093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34acc00640 1 -- 172.21.15.90:0/207096239 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8032d10 con 0x7f34b0109150 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b019f4a0 con 0x7f34b0109150 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34acc00640 1 -- 172.21.15.90:0/207096239 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f34a8032eb0 con 0x7f34b0109150 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34acc00640 1 -- 172.21.15.90:0/207096239 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a8030b20 con 0x7f34b0109150 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.216+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b01a5bc0 con 0x7f34b0109150 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.218+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b01031a0 con 0x7f34b0109150 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.220+0000 7f34acc00640 1 -- 172.21.15.90:0/207096239 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f34a8030cc0 con 0x7f34b0109150 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.221+0000 7f34acc00640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f34940b52a0 0x7f34940b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.221+0000 7f34acc00640 1 -- 172.21.15.90:0/207096239 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f34a8105450 con 0x7f34b0109150 2024-09-10T21:52:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.222+0000 7f34af400640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f34940b52a0 0x7f34940b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.222+0000 7f34af400640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f34940b52a0 0x7f34940b7760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009c30 tx=0x7f34a0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.223+0000 7f34acc00640 1 -- 172.21.15.90:0/207096239 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f34a80cdf00 con 0x7f34b0109150 2024-09-10T21:52:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.418+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f34b019a290 con 0x7f34b0109150 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.423+0000 7f34acc00640 1 -- 172.21.15.90:0/207096239 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f34b019a290 con 0x7f34b0109150 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f34940b52a0 msgr2=0x7f34940b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f34940b52a0 0x7f34940b7760 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009c30 tx=0x7f34a0005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 msgr2=0x7f34b019ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 0x7f34b019ea40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005e30 tx=0x7f34a80093d0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 shutdown_connections 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f34940b52a0 0x7f34940b7760 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f34b0109150 0x7f34b019ea40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f34b01087d0 0x7f34b01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 --2- 172.21.15.90:0/207096239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f34b0102dc0 0x7f34b0199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 >> 172.21.15.90:0/207096239 conn(0x7f34b00fe830 msgr2=0x7f34b00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 shutdown_connections 2024-09-10T21:52:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:52:53.425+0000 7f34b6200640 1 -- 172.21.15.90:0/207096239 wait complete. 2024-09-10T21:53:00.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:00.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:00.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:00.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:00.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:00 smithi090 ceph-mon[30350]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:00.386 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi090') 2024-09-10T21:53:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:00 smithi171 ceph-mon[34138]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:00.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:00.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:00.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:00.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:00.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:00.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:00.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:00 smithi154 ceph-mon[34495]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:01.931 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi090') 2024-09-10T21:53:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:02 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2985193194' entity='client.admin' 2024-09-10T21:53:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:02 smithi090 ceph-mon[30350]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:02.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:02 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2985193194' entity='client.admin' 2024-09-10T21:53:02.417 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:02 smithi154 ceph-mon[34495]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:02.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:02 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2985193194' entity='client.admin' 2024-09-10T21:53:02.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:02 smithi171 ceph-mon[34138]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:03.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.873+0000 7f65bce00640 1 -- 172.21.15.90:0/1559066083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65b8113990 msgr2=0x7f65b8115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:03.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.873+0000 7f65bce00640 1 --2- 172.21.15.90:0/1559066083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65b8113990 0x7f65b8115d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f65b0009900 tx=0x7f65b002a410 comp rx=0 tx=0).stop 2024-09-10T21:53:03.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.873+0000 7f65bce00640 1 -- 172.21.15.90:0/1559066083 shutdown_connections 2024-09-10T21:53:03.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.873+0000 7f65bce00640 1 --2- 172.21.15.90:0/1559066083 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65b8113990 0x7f65b8115d80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.873+0000 7f65bce00640 1 --2- 172.21.15.90:0/1559066083 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65b8077ff0 0x7f65b8113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.873+0000 7f65bce00640 1 --2- 172.21.15.90:0/1559066083 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65b80776d0 0x7f65b8077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.873+0000 7f65bce00640 1 -- 172.21.15.90:0/1559066083 >> 172.21.15.90:0/1559066083 conn(0x7f65b8100a50 msgr2=0x7f65b8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.873+0000 7f65bce00640 1 -- 172.21.15.90:0/1559066083 shutdown_connections 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.873+0000 7f65bce00640 1 -- 172.21.15.90:0/1559066083 wait complete. 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.874+0000 7f65bce00640 1 Processor -- start 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.874+0000 7f65bce00640 1 -- start start 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65bce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65b80776d0 0x7f65b819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65bce00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65b8077ff0 0x7f65b819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65bce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65b8113990 0x7f65b81a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65bce00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8119260 con 0x7f65b8077ff0 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65bce00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81190e0 con 0x7f65b8113990 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65bce00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81193e0 con 0x7f65b80776d0 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65b5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65b80776d0 0x7f65b819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65ad400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65b8077ff0 0x7f65b819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:03.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65b5e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65b80776d0 0x7f65b819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:56336/0 (socket says 172.21.15.90:56336) 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65b5e00640 1 -- 172.21.15.90:0/2985193194 learned_addr learned my addr 172.21.15.90:0/2985193194 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65b6800640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65b8113990 0x7f65b81a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65ad400640 1 -- 172.21.15.90:0/2985193194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65b80776d0 msgr2=0x7f65b819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65ad400640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65b80776d0 0x7f65b819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65ad400640 1 -- 172.21.15.90:0/2985193194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65b8113990 msgr2=0x7f65b81a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65ad400640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65b8113990 0x7f65b81a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65ad400640 1 -- 172.21.15.90:0/2985193194 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b81a5840 con 0x7f65b8077ff0 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65b5e00640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65b80776d0 0x7f65b819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.875+0000 7f65b6800640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65b8113990 0x7f65b81a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.876+0000 7f65ad400640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65b8077ff0 0x7f65b819fcd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f65a0005fd0 tx=0x7f65a000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.876+0000 7f65af400640 1 -- 172.21.15.90:0/2985193194 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0009c30 con 0x7f65b8077ff0 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.876+0000 7f65af400640 1 -- 172.21.15.90:0/2985193194 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65a00044c0 con 0x7f65b8077ff0 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.876+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b81a5b30 con 0x7f65b8077ff0 2024-09-10T21:53:03.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.876+0000 7f65af400640 1 -- 172.21.15.90:0/2985193194 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0003d70 con 0x7f65b8077ff0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.876+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b81ac1f0 con 0x7f65b8077ff0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.877+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b8077ab0 con 0x7f65b8077ff0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.879+0000 7f65af400640 1 -- 172.21.15.90:0/2985193194 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f65a0023060 con 0x7f65b8077ff0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.880+0000 7f65af400640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f659c0b52a0 0x7f659c0b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.881+0000 7f65af400640 1 -- 172.21.15.90:0/2985193194 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f65a00e34f0 con 0x7f65b8077ff0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.881+0000 7f65b5e00640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f659c0b52a0 0x7f659c0b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.881+0000 7f65b5e00640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f659c0b52a0 0x7f659c0b7760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009fd0 tx=0x7f65a8009720 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:00.884+0000 7f65af400640 1 -- 172.21.15.90:0/2985193194 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f65a0017020 con 0x7f65b8077ff0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.075+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f65b81ac5b0 con 0x7f65b8077ff0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.079+0000 7f65af400640 1 -- 172.21.15.90:0/2985193194 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v32)=0 v32) ==== 121+0+0 (secure 0 0 0) 0x7f65a00abf10 con 0x7f65b8077ff0 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f659c0b52a0 msgr2=0x7f659c0b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f659c0b52a0 0x7f659c0b7760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f65a8009fd0 tx=0x7f65a8009720 comp rx=0 tx=0).stop 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65b8077ff0 msgr2=0x7f65b819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:03.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65b8077ff0 0x7f65b819fcd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f65a0005fd0 tx=0x7f65a000dde0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 shutdown_connections 2024-09-10T21:53:03.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f659c0b52a0 0x7f659c0b7760 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65b8113990 0x7f65b81a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65b8077ff0 0x7f65b819fcd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 --2- 172.21.15.90:0/2985193194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65b80776d0 0x7f65b819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:03.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 >> 172.21.15.90:0/2985193194 conn(0x7f65b8100a50 msgr2=0x7f65b8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:03.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 shutdown_connections 2024-09-10T21:53:03.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:01.082+0000 7f65bce00640 1 -- 172.21.15.90:0/2985193194 wait complete. 2024-09-10T21:53:04.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:04 smithi090 ceph-mon[30350]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:04.617 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:53:04.650 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi090') 2024-09-10T21:53:04.651 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi090') 2024-09-10T21:53:04.651 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi090') 2024-09-10T21:53:04.651 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi090') 2024-09-10T21:53:04.651 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi090') 2024-09-10T21:53:04.651 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi090') 2024-09-10T21:53:04.651 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi090') 2024-09-10T21:53:04.651 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi090') 2024-09-10T21:53:04.652 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi090') 2024-09-10T21:53:04.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:04 smithi154 ceph-mon[34495]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:04 smithi171 ceph-mon[34138]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:05.837 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi090') 2024-09-10T21:53:06.121 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:53:06.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:06 smithi090 ceph-mon[30350]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:06.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:06 smithi154 ceph-mon[34495]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:06.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:06 smithi171 ceph-mon[34138]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:07.186 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:53:07.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.189+0000 7facdc800640 1 -- 172.21.15.90:0/862213685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 msgr2=0x7facd410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.189+0000 7facdc800640 1 --2- 172.21.15.90:0/862213685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 0x7facd410f950 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7facd0009ca0 tx=0x7facd002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.190+0000 7facdc800640 1 -- 172.21.15.90:0/862213685 shutdown_connections 2024-09-10T21:53:07.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.190+0000 7facdc800640 1 --2- 172.21.15.90:0/862213685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 0x7facd410f950 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.190+0000 7facdc800640 1 --2- 172.21.15.90:0/862213685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facd41087d0 0x7facd4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.190+0000 7facdc800640 1 --2- 172.21.15.90:0/862213685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd4102dc0 0x7facd41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.190+0000 7facdc800640 1 -- 172.21.15.90:0/862213685 >> 172.21.15.90:0/862213685 conn(0x7facd40fe830 msgr2=0x7facd4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:07.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.190+0000 7facdc800640 1 -- 172.21.15.90:0/862213685 shutdown_connections 2024-09-10T21:53:07.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.190+0000 7facdc800640 1 -- 172.21.15.90:0/862213685 wait complete. 2024-09-10T21:53:07.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.191+0000 7facdc800640 1 Processor -- start 2024-09-10T21:53:07.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.191+0000 7facdc800640 1 -- start start 2024-09-10T21:53:07.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.191+0000 7facdc800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd4102dc0 0x7facd419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:07.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.191+0000 7facdc800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facd41087d0 0x7facd419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:07.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.191+0000 7facdc800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 0x7facd41a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.191+0000 7facdc800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7facd4112cd0 con 0x7facd4109150 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.191+0000 7facdc800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7facd4112b50 con 0x7facd4102dc0 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.191+0000 7facdc800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7facd4112e50 con 0x7facd41087d0 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.191+0000 7facda800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 0x7facd41a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facda800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 0x7facd41a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42040/0 (socket says 172.21.15.90:42040) 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facda800640 1 -- 172.21.15.90:0/3601152659 learned_addr learned my addr 172.21.15.90:0/3601152659 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facd9400640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facd41087d0 0x7facd419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facd9e00640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd4102dc0 0x7facd419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facda800640 1 -- 172.21.15.90:0/3601152659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facd41087d0 msgr2=0x7facd419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facda800640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facd41087d0 0x7facd419b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facda800640 1 -- 172.21.15.90:0/3601152659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd4102dc0 msgr2=0x7facd419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facda800640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd4102dc0 0x7facd419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facda800640 1 -- 172.21.15.90:0/3601152659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd41a1410 con 0x7facd4109150 2024-09-10T21:53:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facd9400640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facd41087d0 0x7facd419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:53:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.192+0000 7facda800640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 0x7facd41a0c40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7facd0031040 tx=0x7facd0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.193+0000 7faccea00640 1 -- 172.21.15.90:0/3601152659 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0032c90 con 0x7facd4109150 2024-09-10T21:53:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.193+0000 7faccea00640 1 -- 172.21.15.90:0/3601152659 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7facd0032e30 con 0x7facd4109150 2024-09-10T21:53:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.193+0000 7faccea00640 1 -- 172.21.15.90:0/3601152659 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0030a30 con 0x7facd4109150 2024-09-10T21:53:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.193+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd41a16a0 con 0x7facd4109150 2024-09-10T21:53:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.193+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd41a7dc0 con 0x7facd4109150 2024-09-10T21:53:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.194+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd41031a0 con 0x7facd4109150 2024-09-10T21:53:07.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.196+0000 7faccea00640 1 -- 172.21.15.90:0/3601152659 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7facd0030bd0 con 0x7facd4109150 2024-09-10T21:53:07.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.198+0000 7faccea00640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7facbc0b52a0 0x7facbc0b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:07.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.199+0000 7faccea00640 1 -- 172.21.15.90:0/3601152659 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7facd0105350 con 0x7facd4109150 2024-09-10T21:53:07.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.199+0000 7facd9e00640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7facbc0b52a0 0x7facbc0b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:07.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.199+0000 7facd9e00640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7facbc0b52a0 0x7facbc0b7760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7facc800abe0 tx=0x7facc800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:07.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.203+0000 7faccea00640 1 -- 172.21.15.90:0/3601152659 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7facd00cdd70 con 0x7facd4109150 2024-09-10T21:53:07.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.413+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7facd410f3a0 con 0x7facd4109150 2024-09-10T21:53:07.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.418+0000 7faccea00640 1 -- 172.21.15.90:0/3601152659 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v33)=0 v33) ==== 125+0+0 (secure 0 0 0) 0x7facd00cd670 con 0x7facd4109150 2024-09-10T21:53:07.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7facbc0b52a0 msgr2=0x7facbc0b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7facbc0b52a0 0x7facbc0b7760 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7facc800abe0 tx=0x7facc800a2d0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 msgr2=0x7facd41a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 0x7facd41a0c40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7facd0031040 tx=0x7facd0003f80 comp rx=0 tx=0).stop 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 shutdown_connections 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7facbc0b52a0 0x7facbc0b7760 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7facd4109150 0x7facd41a0c40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7facd41087d0 0x7facd419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 --2- 172.21.15.90:0/3601152659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7facd4102dc0 0x7facd419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 >> 172.21.15.90:0/3601152659 conn(0x7facd40fe830 msgr2=0x7facd4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 shutdown_connections 2024-09-10T21:53:07.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.420+0000 7facdc800640 1 -- 172.21.15.90:0/3601152659 wait complete. 2024-09-10T21:53:07.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.452+0000 7f03f3400640 1 -- 172.21.15.90:0/3115596750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec10d120 msgr2=0x7f03ec10f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.452+0000 7f03f3400640 1 --2- 172.21.15.90:0/3115596750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec10d120 0x7f03ec10f580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009ca0 tx=0x7f03e402a8b0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.452+0000 7f03f3400640 1 -- 172.21.15.90:0/3115596750 shutdown_connections 2024-09-10T21:53:07.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.452+0000 7f03f3400640 1 --2- 172.21.15.90:0/3115596750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec10d120 0x7f03ec10f580 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.452+0000 7f03f3400640 1 --2- 172.21.15.90:0/3115596750 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03ec1045e0 0x7f03ec10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.452+0000 7f03f3400640 1 --2- 172.21.15.90:0/3115596750 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03ec103cc0 0x7f03ec1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.452+0000 7f03f3400640 1 -- 172.21.15.90:0/3115596750 >> 172.21.15.90:0/3115596750 conn(0x7f03ec0fd700 msgr2=0x7f03ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:07.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.452+0000 7f03f3400640 1 -- 172.21.15.90:0/3115596750 shutdown_connections 2024-09-10T21:53:07.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.452+0000 7f03f3400640 1 -- 172.21.15.90:0/3115596750 wait complete. 2024-09-10T21:53:07.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.453+0000 7f03f3400640 1 Processor -- start 2024-09-10T21:53:07.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.453+0000 7f03f3400640 1 -- start start 2024-09-10T21:53:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.453+0000 7f03f3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03ec103cc0 0x7f03ec19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.453+0000 7f03f3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec1045e0 0x7f03ec19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.453+0000 7f03f3400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03ec10d120 0x7f03ec1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.453+0000 7f03f3400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec112c90 con 0x7f03ec1045e0 2024-09-10T21:53:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.453+0000 7f03f3400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec112b10 con 0x7f03ec103cc0 2024-09-10T21:53:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.453+0000 7f03f3400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec112e10 con 0x7f03ec10d120 2024-09-10T21:53:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03ebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec1045e0 0x7f03ec19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03ebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec1045e0 0x7f03ec19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42050/0 (socket says 172.21.15.90:42050) 2024-09-10T21:53:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03ebe00640 1 -- 172.21.15.90:0/3869528982 learned_addr learned my addr 172.21.15.90:0/3869528982 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:53:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03f0a00640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03ec103cc0 0x7f03ec19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03f1400640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03ec10d120 0x7f03ec1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03ebe00640 1 -- 172.21.15.90:0/3869528982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03ec10d120 msgr2=0x7f03ec1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03ebe00640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03ec10d120 0x7f03ec1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03ebe00640 1 -- 172.21.15.90:0/3869528982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03ec103cc0 msgr2=0x7f03ec19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03ebe00640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03ec103cc0 0x7f03ec19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.454+0000 7f03ebe00640 1 -- 172.21.15.90:0/3869528982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec1a12e0 con 0x7f03ec1045e0 2024-09-10T21:53:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.455+0000 7f03ebe00640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec1045e0 0x7f03ec19b770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c970 tx=0x7f03d800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.455+0000 7f03e9600640 1 -- 172.21.15.90:0/3869528982 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8007bb0 con 0x7f03ec1045e0 2024-09-10T21:53:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.455+0000 7f03e9600640 1 -- 172.21.15.90:0/3869528982 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03d8007d50 con 0x7f03ec1045e0 2024-09-10T21:53:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.455+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ec1a15d0 con 0x7f03ec1045e0 2024-09-10T21:53:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.455+0000 7f03e9600640 1 -- 172.21.15.90:0/3869528982 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d800f6b0 con 0x7f03ec1045e0 2024-09-10T21:53:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.455+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ec1a7c90 con 0x7f03ec1045e0 2024-09-10T21:53:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.456+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ec1040a0 con 0x7f03ec1045e0 2024-09-10T21:53:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.458+0000 7f03e9600640 1 -- 172.21.15.90:0/3869528982 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f03d8018480 con 0x7f03ec1045e0 2024-09-10T21:53:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.460+0000 7f03e9600640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03d40b52a0 0x7f03d40b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.460+0000 7f03e9600640 1 -- 172.21.15.90:0/3869528982 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f03d80e42b0 con 0x7f03ec1045e0 2024-09-10T21:53:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.460+0000 7f03f0a00640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03d40b52a0 0x7f03d40b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:07.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.461+0000 7f03f0a00640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03d40b52a0 0x7f03d40b7760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f03e000abe0 tx=0x7f03e000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:07.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.463+0000 7f03e9600640 1 -- 172.21.15.90:0/3869528982 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f03d80acd60 con 0x7f03ec1045e0 2024-09-10T21:53:07.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.655+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f03ec19c360 con 0x7f03ec1045e0 2024-09-10T21:53:07.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.659+0000 7f03e9600640 1 -- 172.21.15.90:0/3869528982 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v34)=0 v34) ==== 138+0+0 (secure 0 0 0) 0x7f03d80ac660 con 0x7f03ec1045e0 2024-09-10T21:53:07.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03d40b52a0 msgr2=0x7f03d40b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03d40b52a0 0x7f03d40b7760 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f03e000abe0 tx=0x7f03e000a2d0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec1045e0 msgr2=0x7f03ec19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:07.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec1045e0 0x7f03ec19b770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c970 tx=0x7f03d800cd70 comp rx=0 tx=0).stop 2024-09-10T21:53:07.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 shutdown_connections 2024-09-10T21:53:07.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03d40b52a0 0x7f03d40b7760 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f03ec10d120 0x7f03ec1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f03ec1045e0 0x7f03ec19b770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 --2- 172.21.15.90:0/3869528982 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f03ec103cc0 0x7f03ec19b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:07.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 >> 172.21.15.90:0/3869528982 conn(0x7f03ec0fd700 msgr2=0x7f03ec101850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:07.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 shutdown_connections 2024-09-10T21:53:07.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:07.662+0000 7f03f3400640 1 -- 172.21.15.90:0/3869528982 wait complete. 2024-09-10T21:53:08.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.469+0000 7f0301800640 1 -- 172.21.15.90:0/1101958032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc1087d0 msgr2=0x7f02fc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:08.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.469+0000 7f0301800640 1 --2- 172.21.15.90:0/1101958032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc1087d0 0x7f02fc108c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009900 tx=0x7f02e802a410 comp rx=0 tx=0).stop 2024-09-10T21:53:08.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.469+0000 7f0301800640 1 -- 172.21.15.90:0/1101958032 shutdown_connections 2024-09-10T21:53:08.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.469+0000 7f0301800640 1 --2- 172.21.15.90:0/1101958032 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02fc109150 0x7f02fc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.469+0000 7f0301800640 1 --2- 172.21.15.90:0/1101958032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc1087d0 0x7f02fc108c10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.469+0000 7f0301800640 1 --2- 172.21.15.90:0/1101958032 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02fc102dc0 0x7f02fc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.469+0000 7f0301800640 1 -- 172.21.15.90:0/1101958032 >> 172.21.15.90:0/1101958032 conn(0x7f02fc0fe830 msgr2=0x7f02fc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.469+0000 7f0301800640 1 -- 172.21.15.90:0/1101958032 shutdown_connections 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.469+0000 7f0301800640 1 -- 172.21.15.90:0/1101958032 wait complete. 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.470+0000 7f0301800640 1 Processor -- start 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.470+0000 7f0301800640 1 -- start start 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f0301800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02fc102dc0 0x7f02fc199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f0301800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02fc1087d0 0x7f02fc199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f0301800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc109150 0x7f02fc19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f0301800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc112ca0 con 0x7f02fc109150 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f0301800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc112b20 con 0x7f02fc102dc0 2024-09-10T21:53:08.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f0301800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f02fc112e20 con 0x7f02fc1087d0 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f02fb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc109150 0x7f02fc19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f02fb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc109150 0x7f02fc19ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34624/0 (socket says 172.21.15.90:34624) 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f02fb400640 1 -- 172.21.15.90:0/2654913051 learned_addr learned my addr 172.21.15.90:0/2654913051 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f02faa00640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02fc102dc0 0x7f02fc199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.471+0000 7f02fa000640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02fc1087d0 0x7f02fc199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f02fb400640 1 -- 172.21.15.90:0/2654913051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02fc1087d0 msgr2=0x7f02fc199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f02fb400640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02fc1087d0 0x7f02fc199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f02fb400640 1 -- 172.21.15.90:0/2654913051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02fc102dc0 msgr2=0x7f02fc199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f02fb400640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02fc102dc0 0x7f02fc199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f02fb400640 1 -- 172.21.15.90:0/2654913051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02fc19f1e0 con 0x7f02fc109150 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f02fb400640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc109150 0x7f02fc19ea10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009fd0 tx=0x7f02f400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f02f3400640 1 -- 172.21.15.90:0/2654913051 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4017070 con 0x7f02fc109150 2024-09-10T21:53:08.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f02f3400640 1 -- 172.21.15.90:0/2654913051 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02f4003e00 con 0x7f02fc109150 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f02f3400640 1 -- 172.21.15.90:0/2654913051 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4010460 con 0x7f02fc109150 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02fc19f4d0 con 0x7f02fc109150 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.472+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02fc1a5b90 con 0x7f02fc109150 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.474+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02fc1031a0 con 0x7f02fc109150 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.476+0000 7f02f3400640 1 -- 172.21.15.90:0/2654913051 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f02f4003970 con 0x7f02fc109150 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.477+0000 7f02f3400640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02e00b52a0 0x7f02e00b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.477+0000 7f02f3400640 1 -- 172.21.15.90:0/2654913051 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f02f40e3e30 con 0x7f02fc109150 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.478+0000 7f02faa00640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02e00b52a0 0x7f02e00b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.478+0000 7f02faa00640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02e00b52a0 0x7f02e00b7760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00abe0 tx=0x7f02ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:08.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.481+0000 7f02f3400640 1 -- 172.21.15.90:0/2654913051 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f02f40ac850 con 0x7f02fc109150 2024-09-10T21:53:08.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.668+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f02fc02e5c0 con 0x7f02fc109150 2024-09-10T21:53:08.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.672+0000 7f02f3400640 1 -- 172.21.15.90:0/2654913051 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v35)=0 v35) ==== 119+0+0 (secure 0 0 0) 0x7f02f40ac150 con 0x7f02fc109150 2024-09-10T21:53:08.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:08 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3601152659' entity='client.admin' 2024-09-10T21:53:08.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:08 smithi090 ceph-mon[30350]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:08.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:08 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3869528982' entity='client.admin' 2024-09-10T21:53:08.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:08.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:08.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:08.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02e00b52a0 msgr2=0x7f02e00b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02e00b52a0 0x7f02e00b7760 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f02ec00abe0 tx=0x7f02ec00a2d0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc109150 msgr2=0x7f02fc19ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:08.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc109150 0x7f02fc19ea10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009fd0 tx=0x7f02f400def0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 shutdown_connections 2024-09-10T21:53:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02e00b52a0 0x7f02e00b7760 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02fc109150 0x7f02fc19ea10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f02fc1087d0 0x7f02fc199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 --2- 172.21.15.90:0/2654913051 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f02fc102dc0 0x7f02fc199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 >> 172.21.15.90:0/2654913051 conn(0x7f02fc0fe830 msgr2=0x7f02fc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 shutdown_connections 2024-09-10T21:53:08.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:08.675+0000 7f0301800640 1 -- 172.21.15.90:0/2654913051 wait complete. 2024-09-10T21:53:08.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:08 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3601152659' entity='client.admin' 2024-09-10T21:53:08.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:08 smithi171 ceph-mon[34138]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:08.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:08 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3869528982' entity='client.admin' 2024-09-10T21:53:08.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:08.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:08.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:08.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:08 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3601152659' entity='client.admin' 2024-09-10T21:53:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:08 smithi154 ceph-mon[34495]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:08 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3869528982' entity='client.admin' 2024-09-10T21:53:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:09.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:09 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2654913051' entity='client.admin' 2024-09-10T21:53:09.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.987 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.988 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.988 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.988 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.988 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.988 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '815bde3e-6fbe-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi090') 2024-09-10T21:53:09.988 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T21:53:09.989 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T21:53:09.989 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:53:09.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:09 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2654913051' entity='client.admin' 2024-09-10T21:53:10.025 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:53:10.025 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-09-10T21:53:10.101 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:53:10.101 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:10.101 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 639 Links: 1 2024-09-10T21:53:10.101 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:10.101 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:10.101 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-10 21:51:34.677859210 +0000 2024-09-10T21:53:10.101 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-10 21:46:27.883021387 +0000 2024-09-10T21:53:10.101 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-10 21:46:27.883021387 +0000 2024-09-10T21:53:10.101 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-10 21:46:27.883021387 +0000 2024-09-10T21:53:10.102 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:53:10.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:09 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2654913051' entity='client.admin' 2024-09-10T21:53:10.195 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-10T21:53:10.195 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-10T21:53:10.195 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000271237 s, 1.9 MB/s 2024-09-10T21:53:10.197 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:53:10.234 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-09-10T21:53:10.308 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:53:10.309 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:10.309 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 658 Links: 1 2024-09-10T21:53:10.309 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:10.309 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:10.309 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-10 21:51:34.719858547 +0000 2024-09-10T21:53:10.309 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-10 21:46:28.339014141 +0000 2024-09-10T21:53:10.309 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-10 21:46:28.339014141 +0000 2024-09-10T21:53:10.309 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-10 21:46:28.339014141 +0000 2024-09-10T21:53:10.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:53:10.402 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-10T21:53:10.402 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-10T21:53:10.402 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000292774 s, 1.7 MB/s 2024-09-10T21:53:10.405 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:53:10.441 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-09-10T21:53:10.515 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:53:10.516 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:10.516 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 677 Links: 1 2024-09-10T21:53:10.516 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:10.516 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:10.516 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-10 21:51:34.756857962 +0000 2024-09-10T21:53:10.516 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-10 21:46:28.784007071 +0000 2024-09-10T21:53:10.516 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-10 21:46:28.784007071 +0000 2024-09-10T21:53:10.516 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-10 21:46:28.784007071 +0000 2024-09-10T21:53:10.516 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:53:10.608 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-10T21:53:10.608 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-10T21:53:10.608 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000274399 s, 1.9 MB/s 2024-09-10T21:53:10.611 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:53:10.647 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-09-10T21:53:10.724 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:53:10.724 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:10.724 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 6h/6d Inode: 697 Links: 1 2024-09-10T21:53:10.724 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:10.724 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:10.724 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-10 21:51:34.794857362 +0000 2024-09-10T21:53:10.724 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-10 21:46:29.243999762 +0000 2024-09-10T21:53:10.724 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-10 21:46:29.243999762 +0000 2024-09-10T21:53:10.724 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-09-10 21:46:29.243999762 +0000 2024-09-10T21:53:10.724 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-09-10T21:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-09-10T21:53:10.816 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000278383 s, 1.8 MB/s 2024-09-10T21:53:10.819 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:53:10.855 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-09-10T21:53:10.855 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:53:10.896 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:53:10.896 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-09-10T21:53:10.975 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:53:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:10.976 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-09-10T21:53:10.976 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:10.976 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:10.976 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-10 21:52:45.643759921 +0000 2024-09-10T21:53:10.976 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-10 21:46:27.873538155 +0000 2024-09-10T21:53:10.976 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-10 21:46:27.873538155 +0000 2024-09-10T21:53:10.976 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-09-10 21:46:27.873538155 +0000 2024-09-10T21:53:10.976 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:53:10.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:10 smithi171 ceph-mon[34138]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:11.068 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:10 smithi154 ceph-mon[34495]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:11.075 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-10T21:53:11.075 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-10T21:53:11.075 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000308841 s, 1.7 MB/s 2024-09-10T21:53:11.078 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:53:11.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:10 smithi090 ceph-mon[30350]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:11.116 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-09-10T21:53:11.192 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:53:11.192 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:11.192 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-09-10T21:53:11.193 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:11.193 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:11.193 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-10 21:52:45.644759895 +0000 2024-09-10T21:53:11.193 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-10 21:46:28.321526352 +0000 2024-09-10T21:53:11.193 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-10 21:46:28.321526352 +0000 2024-09-10T21:53:11.193 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-09-10 21:46:28.321526352 +0000 2024-09-10T21:53:11.193 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:53:11.286 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-10T21:53:11.287 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-10T21:53:11.287 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000291981 s, 1.8 MB/s 2024-09-10T21:53:11.289 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:53:11.326 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-09-10T21:53:11.402 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:53:11.402 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:11.402 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-09-10T21:53:11.402 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:11.402 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:11.402 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-10 21:52:45.645759869 +0000 2024-09-10T21:53:11.402 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-10 21:46:28.786514102 +0000 2024-09-10T21:53:11.402 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-10 21:46:28.786514102 +0000 2024-09-10T21:53:11.402 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-09-10 21:46:28.786514102 +0000 2024-09-10T21:53:11.403 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:53:11.495 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-10T21:53:11.495 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-10T21:53:11.495 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000266266 s, 1.9 MB/s 2024-09-10T21:53:11.498 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:53:11.535 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-09-10T21:53:11.612 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:53:11.612 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:11.612 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-09-10T21:53:11.612 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:11.612 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:11.612 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-09-10 21:52:45.646759843 +0000 2024-09-10T21:53:11.613 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-09-10 21:46:29.248501930 +0000 2024-09-10T21:53:11.613 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-09-10 21:46:29.248501930 +0000 2024-09-10T21:53:11.613 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-09-10 21:46:29.248501930 +0000 2024-09-10T21:53:11.613 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:53:11.708 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-09-10T21:53:11.708 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-09-10T21:53:11.708 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000295643 s, 1.7 MB/s 2024-09-10T21:53:11.710 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:53:11.747 DEBUG:teuthology.orchestra.run.smithi171:> set -ex 2024-09-10T21:53:11.748 DEBUG:teuthology.orchestra.run.smithi171:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:53:11.788 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:53:11.788 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_1 2024-09-10T21:53:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:53:11.866 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:11.866 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 639 Links: 1 2024-09-10T21:53:11.866 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:11.866 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:11.866 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-10 21:52:42.709629469 +0000 2024-09-10T21:53:11.867 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-10 21:46:30.987457533 +0000 2024-09-10T21:53:11.867 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-10 21:46:30.987457533 +0000 2024-09-10T21:53:11.867 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-10 21:46:30.987457533 +0000 2024-09-10T21:53:11.867 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:53:11.963 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-10T21:53:11.963 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-10T21:53:11.963 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000296705 s, 1.7 MB/s 2024-09-10T21:53:11.966 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:53:12.003 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_2 2024-09-10T21:53:12.081 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:53:12.081 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:12.081 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 659 Links: 1 2024-09-10T21:53:12.082 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:12.082 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:12.082 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-10 21:52:42.710629448 +0000 2024-09-10T21:53:12.082 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-10 21:46:31.429448361 +0000 2024-09-10T21:53:12.082 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-10 21:46:31.429448361 +0000 2024-09-10T21:53:12.082 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-10 21:46:31.429448361 +0000 2024-09-10T21:53:12.082 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:53:12.175 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-10T21:53:12.175 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-10T21:53:12.175 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000284294 s, 1.8 MB/s 2024-09-10T21:53:12.177 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:53:12.214 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_3 2024-09-10T21:53:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:53:12.290 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:12.290 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 675 Links: 1 2024-09-10T21:53:12.290 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:12.290 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:12.290 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-10 21:52:42.711629428 +0000 2024-09-10T21:53:12.291 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-10 21:46:31.876439086 +0000 2024-09-10T21:53:12.291 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-10 21:46:31.876439086 +0000 2024-09-10T21:53:12.291 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-10 21:46:31.876439086 +0000 2024-09-10T21:53:12.291 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:53:12.382 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-10T21:53:12.382 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-10T21:53:12.382 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000285728 s, 1.8 MB/s 2024-09-10T21:53:12.384 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:53:12.421 DEBUG:teuthology.orchestra.run.smithi171:> stat /dev/vg_nvme/lv_4 2024-09-10T21:53:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:53:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:53:12.497 INFO:teuthology.orchestra.run.smithi171.stdout:Device: 6h/6d Inode: 694 Links: 1 2024-09-10T21:53:12.497 INFO:teuthology.orchestra.run.smithi171.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:53:12.497 INFO:teuthology.orchestra.run.smithi171.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:53:12.497 INFO:teuthology.orchestra.run.smithi171.stdout:Access: 2024-09-10 21:52:42.711629428 +0000 2024-09-10T21:53:12.497 INFO:teuthology.orchestra.run.smithi171.stdout:Modify: 2024-09-10 21:46:32.345429354 +0000 2024-09-10T21:53:12.497 INFO:teuthology.orchestra.run.smithi171.stdout:Change: 2024-09-10 21:46:32.345429354 +0000 2024-09-10T21:53:12.497 INFO:teuthology.orchestra.run.smithi171.stdout: Birth: 2024-09-10 21:46:32.345429354 +0000 2024-09-10T21:53:12.498 DEBUG:teuthology.orchestra.run.smithi171:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:53:12.589 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records in 2024-09-10T21:53:12.589 INFO:teuthology.orchestra.run.smithi171.stderr:1+0 records out 2024-09-10T21:53:12.590 INFO:teuthology.orchestra.run.smithi171.stderr:512 bytes copied, 0.000277075 s, 1.8 MB/s 2024-09-10T21:53:12.592 DEBUG:teuthology.orchestra.run.smithi171:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:53:12.629 INFO:tasks.cephadm:Deploying osd.0 on smithi090 with /dev/vg_nvme/lv_4... 2024-09-10T21:53:12.629 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T21:53:12.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:12 smithi154 ceph-mon[34495]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:12.993 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:53:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:12 smithi171 ceph-mon[34138]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:13.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:12 smithi090 ceph-mon[30350]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:14.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:14 smithi090 ceph-mon[30350]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:14 smithi171 ceph-mon[34138]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:15.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:14 smithi154 ceph-mon[34495]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:15.924 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:53:15.957 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-09-10T21:53:16.314 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:53:16.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:16 smithi090 ceph-mon[30350]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:16.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.849+0000 7fdaa2600640 1 -- 172.21.15.90:0/1092117344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c0754e0 msgr2=0x7fda9c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:16.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.849+0000 7fdaa2600640 1 --2- 172.21.15.90:0/1092117344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c0754e0 0x7fda9c0758c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009ca0 tx=0x7fda8c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:53:16.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.849+0000 7fdaa2600640 1 -- 172.21.15.90:0/1092117344 shutdown_connections 2024-09-10T21:53:16.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.849+0000 7fdaa2600640 1 --2- 172.21.15.90:0/1092117344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda9c10f3b0 0x7fda9c1117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:16.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.849+0000 7fdaa2600640 1 --2- 172.21.15.90:0/1092117344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c075e00 0x7fda9c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:16.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.849+0000 7fdaa2600640 1 --2- 172.21.15.90:0/1092117344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c0754e0 0x7fda9c0758c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:16.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.849+0000 7fdaa2600640 1 -- 172.21.15.90:0/1092117344 >> 172.21.15.90:0/1092117344 conn(0x7fda9c0fe830 msgr2=0x7fda9c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:16.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.849+0000 7fdaa2600640 1 -- 172.21.15.90:0/1092117344 shutdown_connections 2024-09-10T21:53:16.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.849+0000 7fdaa2600640 1 -- 172.21.15.90:0/1092117344 wait complete. 2024-09-10T21:53:16.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.850+0000 7fdaa2600640 1 Processor -- start 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.850+0000 7fdaa2600640 1 -- start start 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.850+0000 7fdaa2600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c0754e0 0x7fda9c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fdaa2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c075e00 0x7fda9c19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fdaa2600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda9c10f3b0 0x7fda9c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fdaa2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c114e70 con 0x7fda9c075e00 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fdaa2600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c114cf0 con 0x7fda9c0754e0 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fdaa2600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c114ff0 con 0x7fda9c10f3b0 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda92a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c075e00 0x7fda9c19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda92a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c075e00 0x7fda9c19b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34630/0 (socket says 172.21.15.90:34630) 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda92a00640 1 -- 172.21.15.90:0/20249554 learned_addr learned my addr 172.21.15.90:0/20249554 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda9b400640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c0754e0 0x7fda9c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:16.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda9be00640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda9c10f3b0 0x7fda9c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:16.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda92a00640 1 -- 172.21.15.90:0/20249554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda9c10f3b0 msgr2=0x7fda9c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:16.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda92a00640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda9c10f3b0 0x7fda9c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:16.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda92a00640 1 -- 172.21.15.90:0/20249554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c0754e0 msgr2=0x7fda9c19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:16.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda92a00640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c0754e0 0x7fda9c19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:16.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda92a00640 1 -- 172.21.15.90:0/20249554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c1a1490 con 0x7fda9c075e00 2024-09-10T21:53:16.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.851+0000 7fda9b400640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c0754e0 0x7fda9c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:53:16.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.852+0000 7fda92a00640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c075e00 0x7fda9c19b920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fda8400d950 tx=0x7fda8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:16.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.852+0000 7fda98c00640 1 -- 172.21.15.90:0/20249554 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84017070 con 0x7fda9c075e00 2024-09-10T21:53:16.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.852+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c1a1780 con 0x7fda9c075e00 2024-09-10T21:53:16.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.852+0000 7fda98c00640 1 -- 172.21.15.90:0/20249554 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda84009c30 con 0x7fda9c075e00 2024-09-10T21:53:16.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.852+0000 7fda98c00640 1 -- 172.21.15.90:0/20249554 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8400f620 con 0x7fda9c075e00 2024-09-10T21:53:16.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.852+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1a7e40 con 0x7fda9c075e00 2024-09-10T21:53:16.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.853+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9c0758c0 con 0x7fda9c075e00 2024-09-10T21:53:16.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.856+0000 7fda98c00640 1 -- 172.21.15.90:0/20249554 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fda84010460 con 0x7fda9c075e00 2024-09-10T21:53:16.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.857+0000 7fda98c00640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda800b5320 0x7fda800b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:16.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.858+0000 7fda98c00640 1 -- 172.21.15.90:0/20249554 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fda840e3d80 con 0x7fda9c075e00 2024-09-10T21:53:16.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.858+0000 7fda9b400640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda800b5320 0x7fda800b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:16.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.858+0000 7fda9b400640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda800b5320 0x7fda800b77e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fda8c005fa0 tx=0x7fda8c009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:16.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:16.861+0000 7fda98c00640 1 -- 172.21.15.90:0/20249554 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fda840e9020 con 0x7fda9c075e00 2024-09-10T21:53:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:16 smithi171 ceph-mon[34138]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:17.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:17.054+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fda9c100b50 con 0x7fda800b5320 2024-09-10T21:53:17.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:16 smithi154 ceph-mon[34495]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:17.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:53:17.992 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:53:17.992 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:17.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:53:17.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:53:17.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:18.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:53:18.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:53:18.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:18.949 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:18 smithi090 ceph-mon[30350]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:53:18.949 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:18 smithi090 ceph-mon[30350]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:18 smithi171 ceph-mon[34138]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:53:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:18 smithi171 ceph-mon[34138]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:19.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:18 smithi154 ceph-mon[34495]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:53:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:18 smithi154 ceph-mon[34495]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:20 smithi171 ceph-mon[34138]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:20 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/463440322' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa"} : dispatch 2024-09-10T21:53:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:20 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/463440322' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa"}]': finished 2024-09-10T21:53:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:20 smithi171 ceph-mon[34138]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T21:53:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:20 smithi090 ceph-mon[30350]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:20 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/463440322' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa"} : dispatch 2024-09-10T21:53:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:20 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/463440322' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa"}]': finished 2024-09-10T21:53:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:20 smithi090 ceph-mon[30350]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T21:53:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:21.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:20 smithi154 ceph-mon[34495]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:21.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:20 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/463440322' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa"} : dispatch 2024-09-10T21:53:21.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:20 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/463440322' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa"}]': finished 2024-09-10T21:53:21.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:20 smithi154 ceph-mon[34495]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T21:53:21.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:21.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:21 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3244980173' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:53:22.041 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:21 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3244980173' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:53:22.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:21 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3244980173' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:53:22.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:22 smithi154 ceph-mon[34495]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:22 smithi171 ceph-mon[34138]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:22 smithi090 ceph-mon[30350]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:24.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:23 smithi090 ceph-mon[30350]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:24.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:23 smithi154 ceph-mon[34495]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:23 smithi171 ceph-mon[34138]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:26.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:26 smithi090 ceph-mon[30350]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:26.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:26 smithi154 ceph-mon[34495]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:26 smithi171 ceph-mon[34138]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:28 smithi090 ceph-mon[30350]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:28.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:28 smithi154 ceph-mon[34495]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:28.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:28 smithi171 ceph-mon[34138]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:30 smithi090 ceph-mon[30350]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T21:53:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:30.921 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:30 smithi154 ceph-mon[34495]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:30.921 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T21:53:30.921 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:30 smithi171 ceph-mon[34138]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T21:53:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:31.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:31 smithi090 ceph-mon[30350]: Deploying daemon osd.0 on smithi090 2024-09-10T21:53:31.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:31 smithi154 ceph-mon[34495]: Deploying daemon osd.0 on smithi090 2024-09-10T21:53:31.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:31 smithi171 ceph-mon[34138]: Deploying daemon osd.0 on smithi090 2024-09-10T21:53:32.765 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:32 smithi090 ceph-mon[30350]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:32.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:32 smithi154 ceph-mon[34495]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:32.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:32 smithi171 ceph-mon[34138]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:34.816 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:34 smithi171 ceph-mon[34138]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:34.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:34 smithi090 ceph-mon[30350]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:34.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:34 smithi154 ceph-mon[34495]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:36 smithi090 ceph-mon[30350]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:36.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:36 smithi154 ceph-mon[34495]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:36 smithi171 ceph-mon[34138]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:38.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:38 smithi090 ceph-mon[30350]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:38.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:38 smithi154 ceph-mon[34495]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:38.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:38 smithi171 ceph-mon[34138]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:39.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:39.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:39.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:39.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:39.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:39.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:39.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:40.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:40 smithi090 ceph-mon[30350]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:40.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:40 smithi154 ceph-mon[34495]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:40 smithi171 ceph-mon[34138]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:42.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:42 smithi090 ceph-mon[30350]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:42 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:42 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:42.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:42 smithi154 ceph-mon[34495]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:42.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:42 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:42.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:42 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:42 smithi171 ceph-mon[34138]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:42 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:42 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:43 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:43 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:43.649 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:43 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:43.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:43 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:43.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:43 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:43.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:43 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:43.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:43 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:43.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:43 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:43.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:43 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:44.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.536+0000 7fda98c00640 1 -- 172.21.15.90:0/20249554 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda9c100b50 con 0x7fda800b5320 2024-09-10T21:53:44.538 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 0 on host 'smithi090' 2024-09-10T21:53:44.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda800b5320 msgr2=0x7fda800b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:44.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda800b5320 0x7fda800b77e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fda8c005fa0 tx=0x7fda8c009360 comp rx=0 tx=0).stop 2024-09-10T21:53:44.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c075e00 msgr2=0x7fda9c19b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:44.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c075e00 0x7fda9c19b920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fda8400d950 tx=0x7fda8400de10 comp rx=0 tx=0).stop 2024-09-10T21:53:44.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 shutdown_connections 2024-09-10T21:53:44.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda800b5320 0x7fda800b77e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:44.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda9c10f3b0 0x7fda9c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:44.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda9c075e00 0x7fda9c19b920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:44.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 --2- 172.21.15.90:0/20249554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda9c0754e0 0x7fda9c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:44.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.538+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 >> 172.21.15.90:0/20249554 conn(0x7fda9c0fe830 msgr2=0x7fda9c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:44.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.539+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 shutdown_connections 2024-09-10T21:53:44.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:53:44.539+0000 7fdaa2600640 1 -- 172.21.15.90:0/20249554 wait complete. 2024-09-10T21:53:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:44 smithi090 ceph-mon[30350]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:44.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:44 smithi154 ceph-mon[34495]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:44.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:44 smithi171 ceph-mon[34138]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:44.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:53:44.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:44.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:45.268 DEBUG:teuthology.orchestra.run.smithi090:osd.0> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T21:53:45.272 INFO:tasks.cephadm:Deploying osd.1 on smithi154 with /dev/vg_nvme/lv_4... 2024-09-10T21:53:45.272 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T21:53:45.635 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:53:45.756 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:45 smithi090 ceph-mon[30350]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T21:53:45.874 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:45 smithi154 ceph-mon[34495]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T21:53:45.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:45 smithi171 ceph-mon[34138]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T21:53:46.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:46 smithi171 ceph-mon[34138]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:46.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:46 smithi171 ceph-mon[34138]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T21:53:46.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:46 smithi171 ceph-mon[34138]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T21:53:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:46 smithi171 ceph-mon[34138]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:53:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:46.796 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:46 smithi154 ceph-mon[34495]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:46.797 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:46 smithi154 ceph-mon[34495]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T21:53:46.797 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:46 smithi154 ceph-mon[34495]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T21:53:46.797 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:46 smithi154 ceph-mon[34495]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:53:46.797 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:46 smithi090 ceph-mon[30350]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:46 smithi090 ceph-mon[30350]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T21:53:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:46 smithi090 ceph-mon[30350]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T21:53:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:46 smithi090 ceph-mon[30350]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:53:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:46.856 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:53:46 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0[48464]: 2024-09-10T21:53:46.611+0000 7f0855000640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T21:53:46.857 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:53:46 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0[48464]: 2024-09-10T21:53:46.614+0000 7f0850000640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:53:47.718 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:47 smithi154 ceph-mon[34495]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:53:47.718 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:47 smithi154 ceph-mon[34495]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T21:53:47.718 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:47.718 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:47.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:47 smithi090 ceph-mon[30350]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:53:47.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:47 smithi090 ceph-mon[30350]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T21:53:47.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:47.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:47 smithi171 ceph-mon[34138]: from='osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:53:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:47 smithi171 ceph-mon[34138]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T21:53:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:53:48.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_4 2024-09-10T21:53:48.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:48 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:53:48.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:48 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:53:48.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:48 smithi090 ceph-mon[30350]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:48.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:48 smithi090 ceph-mon[30350]: osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] boot 2024-09-10T21:53:48.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:48 smithi090 ceph-mon[30350]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T21:53:48.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:48.893 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:53:48.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:48 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:53:48.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:48 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:53:48.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:48 smithi154 ceph-mon[34495]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:48.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:48 smithi154 ceph-mon[34495]: osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] boot 2024-09-10T21:53:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:48 smithi154 ceph-mon[34495]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T21:53:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:48 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:53:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:48 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:53:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:48 smithi171 ceph-mon[34138]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T21:53:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:48 smithi171 ceph-mon[34138]: osd.0 [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] boot 2024-09-10T21:53:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:48 smithi171 ceph-mon[34138]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T21:53:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T21:53:49.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.408+0000 7fd2a3200640 1 -- 172.21.15.154:0/2668928985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108610 msgr2=0x7fd2a4108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:49.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.408+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2668928985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108610 0x7fd2a4108a50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd290009900 tx=0x7fd29002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:53:49.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.408+0000 7fd2a3200640 1 -- 172.21.15.154:0/2668928985 shutdown_connections 2024-09-10T21:53:49.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.408+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2668928985 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2a4108f90 0x7fd2a410f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:49.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.408+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2668928985 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108610 0x7fd2a4108a50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:49.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.408+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2668928985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2a4102c00 0x7fd2a4102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:49.422 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.408+0000 7fd2a3200640 1 -- 172.21.15.154:0/2668928985 >> 172.21.15.154:0/2668928985 conn(0x7fd2a40fe6d0 msgr2=0x7fd2a4100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:53:49.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.408+0000 7fd2a3200640 1 -- 172.21.15.154:0/2668928985 shutdown_connections 2024-09-10T21:53:49.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.408+0000 7fd2a3200640 1 -- 172.21.15.154:0/2668928985 wait complete. 2024-09-10T21:53:49.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.409+0000 7fd2a3200640 1 Processor -- start 2024-09-10T21:53:49.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.409+0000 7fd2a3200640 1 -- start start 2024-09-10T21:53:49.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2a4102c00 0x7fd2a419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:49.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a3200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2a4108610 0x7fd2a419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:49.423 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a3200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108f90 0x7fd2a41a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a3200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4112ae0 con 0x7fd2a4108610 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a3200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4112960 con 0x7fd2a4108f90 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4112c60 con 0x7fd2a4102c00 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a2800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108f90 0x7fd2a41a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a1e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2a4102c00 0x7fd2a419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a2800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108f90 0x7fd2a41a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44930/0 (socket says 172.21.15.154:44930) 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a2800640 1 -- 172.21.15.154:0/2861729928 learned_addr learned my addr 172.21.15.154:0/2861729928 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a1400640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2a4108610 0x7fd2a419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a2800640 1 -- 172.21.15.154:0/2861729928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2a4102c00 msgr2=0x7fd2a419b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a2800640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2a4102c00 0x7fd2a419b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:49.424 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a2800640 1 -- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2a4108610 msgr2=0x7fd2a419b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:53:49.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a2800640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2a4108610 0x7fd2a419b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:53:49.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.410+0000 7fd2a2800640 1 -- 172.21.15.154:0/2861729928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a41a11e0 con 0x7fd2a4108f90 2024-09-10T21:53:49.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.411+0000 7fd2a2800640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108f90 0x7fd2a41a0a10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00eaa0 tx=0x7fd29c00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:49.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.411+0000 7fd29aa00640 1 -- 172.21.15.154:0/2861729928 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c004d20 con 0x7fd2a4108f90 2024-09-10T21:53:49.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.411+0000 7fd29aa00640 1 -- 172.21.15.154:0/2861729928 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd29c005020 con 0x7fd2a4108f90 2024-09-10T21:53:49.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.411+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a41a14d0 con 0x7fd2a4108f90 2024-09-10T21:53:49.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.411+0000 7fd29aa00640 1 -- 172.21.15.154:0/2861729928 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0116d0 con 0x7fd2a4108f90 2024-09-10T21:53:49.425 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.411+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a41a7b90 con 0x7fd2a4108f90 2024-09-10T21:53:49.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.412+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a4102fe0 con 0x7fd2a4108f90 2024-09-10T21:53:49.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.414+0000 7fd29aa00640 1 -- 172.21.15.154:0/2861729928 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7fd29c00f040 con 0x7fd2a4108f90 2024-09-10T21:53:49.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.416+0000 7fd29aa00640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd2880b53c0 0x7fd2880b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:53:49.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.416+0000 7fd29aa00640 1 -- 172.21.15.154:0/2861729928 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1982+0+0 (secure 0 0 0) 0x7fd29c0e3ff0 con 0x7fd2a4108f90 2024-09-10T21:53:49.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.416+0000 7fd2a1e00640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd2880b53c0 0x7fd2880b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:53:49.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.416+0000 7fd2a1e00640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd2880b53c0 0x7fd2880b7880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd29400abe0 tx=0x7fd29400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:53:49.426 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.420+0000 7fd29aa00640 1 -- 172.21.15.154:0/2861729928 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd29c016030 con 0x7fd2a4108f90 2024-09-10T21:53:49.614 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:53:49.613+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd2a4063590 con 0x7fd2880b53c0 2024-09-10T21:53:49.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:49 smithi154 ceph-mon[34495]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T21:53:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:53:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:49 smithi171 ceph-mon[34138]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T21:53:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:53:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:49 smithi090 ceph-mon[30350]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T21:53:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:53:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:53:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:53:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:50 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi090 2024-09-10T21:53:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:50 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi090 to 17089M 2024-09-10T21:53:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:50 smithi154 ceph-mon[34495]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:50 smithi154 ceph-mon[34495]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:53:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:53:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:53:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:50 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi090 2024-09-10T21:53:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:50 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi090 to 17089M 2024-09-10T21:53:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:50 smithi171 ceph-mon[34138]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:50 smithi171 ceph-mon[34138]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:53:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:53:50.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:53:50.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:50 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi090 2024-09-10T21:53:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:50 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi090 to 17089M 2024-09-10T21:53:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:50 smithi090 ceph-mon[30350]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:50 smithi090 ceph-mon[30350]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:53:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:53:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:53:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:53:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:52 smithi154 ceph-mon[34495]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:52 smithi171 ceph-mon[34138]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:52 smithi090 ceph-mon[30350]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:53.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.154:0/852926435' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "417251ca-442c-4c88-881d-c28668067397"} : dispatch 2024-09-10T21:53:53.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:53 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "417251ca-442c-4c88-881d-c28668067397"} : dispatch 2024-09-10T21:53:53.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:53 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "417251ca-442c-4c88-881d-c28668067397"}]': finished 2024-09-10T21:53:53.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:53 smithi154 ceph-mon[34495]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T21:53:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:53:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.154:0/328499456' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:53:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.154:0/852926435' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "417251ca-442c-4c88-881d-c28668067397"} : dispatch 2024-09-10T21:53:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:53 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "417251ca-442c-4c88-881d-c28668067397"} : dispatch 2024-09-10T21:53:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:53 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "417251ca-442c-4c88-881d-c28668067397"}]': finished 2024-09-10T21:53:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:53 smithi171 ceph-mon[34138]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T21:53:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:53:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.154:0/328499456' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:53:54.007 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.154:0/852926435' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "417251ca-442c-4c88-881d-c28668067397"} : dispatch 2024-09-10T21:53:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:53 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "417251ca-442c-4c88-881d-c28668067397"} : dispatch 2024-09-10T21:53:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:53 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "417251ca-442c-4c88-881d-c28668067397"}]': finished 2024-09-10T21:53:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:53 smithi090 ceph-mon[30350]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T21:53:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:53:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.154:0/328499456' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:53:54.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:54 smithi154 ceph-mon[34495]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:54 smithi171 ceph-mon[34138]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:55.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:54 smithi090 ceph-mon[30350]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:56.996 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:56 smithi154 ceph-mon[34495]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:56.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:56 smithi171 ceph-mon[34138]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:57.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:56 smithi090 ceph-mon[30350]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:58.946 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:53:58 smithi154 ceph-mon[34495]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:58.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:53:58 smithi171 ceph-mon[34138]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:53:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:53:58 smithi090 ceph-mon[30350]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:00.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:00 smithi154 ceph-mon[34495]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:00.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:00 smithi171 ceph-mon[34138]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:00 smithi090 ceph-mon[30350]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:01.951 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:01 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T21:54:01.951 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:01 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:01.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:01 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T21:54:01.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:01 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:01 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T21:54:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:01 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:02.955 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:02 smithi154 ceph-mon[34495]: Deploying daemon osd.1 on smithi154 2024-09-10T21:54:02.955 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:02 smithi154 ceph-mon[34495]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:02.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:02 smithi171 ceph-mon[34138]: Deploying daemon osd.1 on smithi154 2024-09-10T21:54:02.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:02 smithi171 ceph-mon[34138]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:02 smithi090 ceph-mon[30350]: Deploying daemon osd.1 on smithi154 2024-09-10T21:54:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:02 smithi090 ceph-mon[30350]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:04.843 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:04 smithi154 ceph-mon[34495]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:04.979 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:04 smithi090 ceph-mon[30350]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:04 smithi171 ceph-mon[34138]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:06.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:06 smithi171 ceph-mon[34138]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:06 smithi090 ceph-mon[30350]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:06 smithi154 ceph-mon[34495]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:08.907 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:08 smithi154 ceph-mon[34495]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:08.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:08 smithi171 ceph-mon[34138]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:08 smithi090 ceph-mon[30350]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:09.984 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:09 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:09.984 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:09 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:09.984 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:09 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:09.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:09 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:09.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:09 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:09.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:09 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:10.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:09 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:10.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:09 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:10.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:09 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:10.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:10 smithi171 ceph-mon[34138]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:11.004 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:10 smithi154 ceph-mon[34495]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:10 smithi090 ceph-mon[30350]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:12 smithi154 ceph-mon[34495]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:12 smithi171 ceph-mon[34138]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:12 smithi090 ceph-mon[30350]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:13.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:13 smithi154 ceph-mon[34495]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:14.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:13 smithi090 ceph-mon[30350]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:13 smithi171 ceph-mon[34138]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:14.385 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 1 on host 'smithi154' 2024-09-10T21:54:14.385 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.383+0000 7fd29aa00640 1 -- 172.21.15.154:0/2861729928 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd2a4063590 con 0x7fd2880b53c0 2024-09-10T21:54:14.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd2880b53c0 msgr2=0x7fd2880b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:14.387 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd2880b53c0 0x7fd2880b7880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd29400abe0 tx=0x7fd29400a2d0 comp rx=0 tx=0).stop 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108f90 msgr2=0x7fd2a41a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108f90 0x7fd2a41a0a10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd29c00eaa0 tx=0x7fd29c00eea0 comp rx=0 tx=0).stop 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 shutdown_connections 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd2880b53c0 0x7fd2880b7880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd2a4108f90 0x7fd2a41a0a10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2a4108610 0x7fd2a419b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 --2- 172.21.15.154:0/2861729928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd2a4102c00 0x7fd2a419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 >> 172.21.15.154:0/2861729928 conn(0x7fd2a40fe6d0 msgr2=0x7fd2a4105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 shutdown_connections 2024-09-10T21:54:14.388 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:54:14.386+0000 7fd2a3200640 1 -- 172.21.15.154:0/2861729928 wait complete. 2024-09-10T21:54:15.089 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:15.089 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:15.089 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:15.130 DEBUG:teuthology.orchestra.run.smithi154:osd.1> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T21:54:15.134 INFO:tasks.cephadm:Deploying osd.2 on smithi171 with /dev/vg_nvme/lv_4... 2024-09-10T21:54:15.134 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T21:54:15.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:15.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:15.188 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:15.507 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:54:15.895 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:15 smithi171 ceph-mon[34138]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:16.078 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:15 smithi154 ceph-mon[34495]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:16.079 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:15 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1[40185]: 2024-09-10T21:54:15.775+0000 7f65be622640 -1 osd.1 0 log_to_monitors true 2024-09-10T21:54:16.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:15 smithi090 ceph-mon[30350]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:16.961 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:16 smithi171 ceph-mon[34138]: from='osd.1 [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T21:54:16.961 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:16 smithi171 ceph-mon[34138]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T21:54:17.008 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:16 smithi154 ceph-mon[34495]: from='osd.1 [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T21:54:17.008 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:16 smithi154 ceph-mon[34495]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T21:54:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:16 smithi090 ceph-mon[30350]: from='osd.1 [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T21:54:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:16 smithi090 ceph-mon[30350]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T21:54:17.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:17 smithi171 ceph-mon[34138]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T21:54:17.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:17 smithi171 ceph-mon[34138]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T21:54:17.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:17.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:17 smithi171 ceph-mon[34138]: from='osd.1 [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:54:17.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:17 smithi171 ceph-mon[34138]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:54:17.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:17 smithi171 ceph-mon[34138]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:18.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:17 smithi090 ceph-mon[30350]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T21:54:18.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:17 smithi090 ceph-mon[30350]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T21:54:18.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:18.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:17 smithi090 ceph-mon[30350]: from='osd.1 [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:54:18.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:17 smithi090 ceph-mon[30350]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:54:18.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:17 smithi090 ceph-mon[30350]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:18.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:17 smithi154 ceph-mon[34495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T21:54:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:17 smithi154 ceph-mon[34495]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T21:54:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:17 smithi154 ceph-mon[34495]: from='osd.1 [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:54:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:17 smithi154 ceph-mon[34495]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:54:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:17 smithi154 ceph-mon[34495]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:18.497 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:54:18.529 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_4 2024-09-10T21:54:18.667 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1[40185]: 2024-09-10T21:54:18.325+0000 7f65b9c00640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T21:54:18.668 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1[40185]: 2024-09-10T21:54:18.338+0000 7f65af200640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:54:18.891 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:54:19.086 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:18 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:54:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:18 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:54:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:18 smithi171 ceph-mon[34138]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:54:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:18 smithi171 ceph-mon[34138]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T21:54:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:18 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:18 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:18 smithi171 ceph-mon[34138]: from='osd.1 ' entity='osd.1' 2024-09-10T21:54:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:18 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:19.087 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:18 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:18 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:54:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:18 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:54:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:18 smithi090 ceph-mon[30350]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:54:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:18 smithi090 ceph-mon[30350]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T21:54:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:18 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:18 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:18 smithi090 ceph-mon[30350]: from='osd.1 ' entity='osd.1' 2024-09-10T21:54:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:18 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:18 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:19.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:54:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:54:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-mon[34495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:54:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-mon[34495]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T21:54:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-mon[34495]: from='osd.1 ' entity='osd.1' 2024-09-10T21:54:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:18 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:19.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.513+0000 7f13dd400640 1 -- 172.21.15.171:0/2911525073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13d8103cc0 msgr2=0x7f13d81040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:19.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.513+0000 7f13dd400640 1 --2- 172.21.15.171:0/2911525073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13d8103cc0 0x7f13d81040a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009900 tx=0x7f13c802a3f0 comp rx=0 tx=0).stop 2024-09-10T21:54:19.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.513+0000 7f13dd400640 1 -- 172.21.15.171:0/2911525073 shutdown_connections 2024-09-10T21:54:19.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.513+0000 7f13dd400640 1 --2- 172.21.15.171:0/2911525073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d810d120 0x7f13d810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:19.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.513+0000 7f13dd400640 1 --2- 172.21.15.171:0/2911525073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d81045e0 0x7f13d810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:19.515 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.513+0000 7f13dd400640 1 --2- 172.21.15.171:0/2911525073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13d8103cc0 0x7f13d81040a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:19.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.513+0000 7f13dd400640 1 -- 172.21.15.171:0/2911525073 >> 172.21.15.171:0/2911525073 conn(0x7f13d80fd700 msgr2=0x7f13d80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:54:19.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.513+0000 7f13dd400640 1 -- 172.21.15.171:0/2911525073 shutdown_connections 2024-09-10T21:54:19.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.513+0000 7f13dd400640 1 -- 172.21.15.171:0/2911525073 wait complete. 2024-09-10T21:54:19.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.514+0000 7f13dd400640 1 Processor -- start 2024-09-10T21:54:19.516 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.514+0000 7f13dd400640 1 -- start start 2024-09-10T21:54:19.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.514+0000 7f13dd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8103cc0 0x7f13d8101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.514+0000 7f13dd400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d81045e0 0x7f13d8101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.514+0000 7f13dd400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13d810d120 0x7f13d81023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.514+0000 7f13dd400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8112c40 con 0x7f13d8103cc0 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.514+0000 7f13dd400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8112ac0 con 0x7f13d81045e0 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.514+0000 7f13dd400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f13d8112dc0 con 0x7f13d810d120 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d81045e0 0x7f13d8101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13d810d120 0x7f13d81023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d81045e0 0x7f13d8101eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.171:39392/0 (socket says 172.21.15.171:39392) 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d5e00640 1 -- 172.21.15.171:0/3856491458 learned_addr learned my addr 172.21.15.171:0/3856491458 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d6800640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8103cc0 0x7f13d8101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d7200640 1 -- 172.21.15.171:0/3856491458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d81045e0 msgr2=0x7f13d8101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:19.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d7200640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d81045e0 0x7f13d8101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d7200640 1 -- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8103cc0 msgr2=0x7f13d8101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d7200640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8103cc0 0x7f13d8101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d7200640 1 -- 172.21.15.171:0/3856491458 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d81a7550 con 0x7f13d810d120 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d5e00640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d81045e0 0x7f13d8101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.515+0000 7f13d7200640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13d810d120 0x7f13d81023f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f13d000e940 tx=0x7f13d000ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.516+0000 7f13cf400640 1 -- 172.21.15.171:0/3856491458 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0004d20 con 0x7f13d810d120 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.516+0000 7f13cf400640 1 -- 172.21.15.171:0/3856491458 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f13d0005020 con 0x7f13d810d120 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.516+0000 7f13cf400640 1 -- 172.21.15.171:0/3856491458 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d00116d0 con 0x7f13d810d120 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.516+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d81a7840 con 0x7f13d810d120 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.516+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d81a7d30 con 0x7f13d810d120 2024-09-10T21:54:19.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.517+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d81040a0 con 0x7f13d810d120 2024-09-10T21:54:19.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.519+0000 7f13cf400640 1 -- 172.21.15.171:0/3856491458 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f13d000f040 con 0x7f13d810d120 2024-09-10T21:54:19.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.521+0000 7f13cf400640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f13bc0b52a0 0x7f13bc0b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:54:19.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.521+0000 7f13cf400640 1 -- 172.21.15.171:0/3856491458 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2524+0+0 (secure 0 0 0) 0x7f13d00e4640 con 0x7f13d810d120 2024-09-10T21:54:19.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.521+0000 7f13d6800640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f13bc0b52a0 0x7f13bc0b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:54:19.523 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.522+0000 7f13d6800640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f13bc0b52a0 0x7f13bc0b7760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f13c80057c0 tx=0x7f13c80058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:54:19.524 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.523+0000 7f13cf400640 1 -- 172.21.15.171:0/3856491458 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f13d0016030 con 0x7f13d810d120 2024-09-10T21:54:19.718 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:19.716+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f13d80ffa20 con 0x7f13bc0b52a0 2024-09-10T21:54:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi154 2024-09-10T21:54:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:54:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi154 to 17089M 2024-09-10T21:54:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: osd.1 [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] boot 2024-09-10T21:54:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T21:54:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: pgmap v55: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:54:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:54:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:54:19.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi154 2024-09-10T21:54:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:54:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi154 to 17089M 2024-09-10T21:54:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: osd.1 [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] boot 2024-09-10T21:54:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T21:54:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: pgmap v55: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:54:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:54:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:54:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi154 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi154 to 17089M 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: osd.1 [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] boot 2024-09-10T21:54:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T21:54:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T21:54:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: pgmap v55: 0 pgs: ; 0 B data, 27 MiB used, 89 GiB / 89 GiB avail 2024-09-10T21:54:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:54:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:54:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:54:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:21.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:21 smithi171 ceph-mon[34138]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T21:54:21.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:21 smithi171 ceph-mon[34138]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:21 smithi090 ceph-mon[30350]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T21:54:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:21 smithi090 ceph-mon[30350]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:22.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:21 smithi154 ceph-mon[34495]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T21:54:22.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:21 smithi154 ceph-mon[34495]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:22 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/712208383' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "529eeb1a-fe6a-4077-b8a5-97246ab90ec5"} : dispatch 2024-09-10T21:54:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:22 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "529eeb1a-fe6a-4077-b8a5-97246ab90ec5"} : dispatch 2024-09-10T21:54:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:22 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "529eeb1a-fe6a-4077-b8a5-97246ab90ec5"}]': finished 2024-09-10T21:54:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:22 smithi090 ceph-mon[30350]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T21:54:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:23.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:22 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/712208383' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "529eeb1a-fe6a-4077-b8a5-97246ab90ec5"} : dispatch 2024-09-10T21:54:23.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:22 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "529eeb1a-fe6a-4077-b8a5-97246ab90ec5"} : dispatch 2024-09-10T21:54:23.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:22 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "529eeb1a-fe6a-4077-b8a5-97246ab90ec5"}]': finished 2024-09-10T21:54:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:22 smithi154 ceph-mon[34495]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T21:54:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:22 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/712208383' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "529eeb1a-fe6a-4077-b8a5-97246ab90ec5"} : dispatch 2024-09-10T21:54:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:22 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "529eeb1a-fe6a-4077-b8a5-97246ab90ec5"} : dispatch 2024-09-10T21:54:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:22 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "529eeb1a-fe6a-4077-b8a5-97246ab90ec5"}]': finished 2024-09-10T21:54:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:22 smithi171 ceph-mon[34138]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T21:54:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:24.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:23 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/720112405' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:54:24.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:23 smithi090 ceph-mon[30350]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:24.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:23 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/720112405' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:54:24.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:23 smithi154 ceph-mon[34495]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:23 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/720112405' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:54:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:23 smithi171 ceph-mon[34138]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:26 smithi090 ceph-mon[30350]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:26.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:26 smithi154 ceph-mon[34495]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:26 smithi171 ceph-mon[34138]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:28.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:28 smithi171 ceph-mon[34138]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:28 smithi090 ceph-mon[30350]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:28.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:28 smithi154 ceph-mon[34495]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:30.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:30 smithi171 ceph-mon[34138]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:30 smithi090 ceph-mon[30350]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:30.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:30 smithi154 ceph-mon[34495]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:31.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:31 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T21:54:31.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:31 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:31.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:31 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T21:54:31.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:31 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:31.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:31 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T21:54:31.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:31 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:32.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:32 smithi171 ceph-mon[34138]: Deploying daemon osd.2 on smithi171 2024-09-10T21:54:32.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:32 smithi171 ceph-mon[34138]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:32.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:32 smithi090 ceph-mon[30350]: Deploying daemon osd.2 on smithi171 2024-09-10T21:54:32.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:32 smithi090 ceph-mon[30350]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:32.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:32 smithi154 ceph-mon[34495]: Deploying daemon osd.2 on smithi171 2024-09-10T21:54:32.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:32 smithi154 ceph-mon[34495]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:34.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:34 smithi090 ceph-mon[30350]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:34.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:34 smithi154 ceph-mon[34495]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:34.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:34 smithi171 ceph-mon[34138]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:36 smithi090 ceph-mon[30350]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:36.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:36 smithi154 ceph-mon[34495]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:36 smithi171 ceph-mon[34138]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:38.779 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:38 smithi171 ceph-mon[34138]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:38.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:38 smithi090 ceph-mon[30350]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:38.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:38 smithi154 ceph-mon[34495]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:39.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:39.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:39.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:39 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:39.856 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:39.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:39.857 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:39 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:39.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:39.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:39.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:39 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:40 smithi090 ceph-mon[30350]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:40.863 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:40 smithi171 ceph-mon[34138]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:40.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:40 smithi154 ceph-mon[34495]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:42 smithi090 ceph-mon[30350]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:42 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:42 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:42.895 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:42 smithi154 ceph-mon[34495]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:42.895 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:42 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:42.895 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:42 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:42 smithi171 ceph-mon[34138]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:42 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:42 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:43.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:43 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:43.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:43 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:43.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:43 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:43.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:43 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:43.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:43 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:43.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:43 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:43.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:43 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:43.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:43 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:43.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:43 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:43.975 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 2 on host 'smithi171' 2024-09-10T21:54:43.975 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.974+0000 7f13cf400640 1 -- 172.21.15.171:0/3856491458 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f13d80ffa20 con 0x7f13bc0b52a0 2024-09-10T21:54:43.977 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f13bc0b52a0 msgr2=0x7f13bc0b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:43.978 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f13bc0b52a0 0x7f13bc0b7760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f13c80057c0 tx=0x7f13c80058d0 comp rx=0 tx=0).stop 2024-09-10T21:54:43.978 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13d810d120 msgr2=0x7f13d81023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:43.978 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13d810d120 0x7f13d81023f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f13d000e940 tx=0x7f13d000ee00 comp rx=0 tx=0).stop 2024-09-10T21:54:43.978 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 shutdown_connections 2024-09-10T21:54:43.978 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f13bc0b52a0 0x7f13bc0b7760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:43.978 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f13d810d120 0x7f13d81023f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:43.978 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f13d81045e0 0x7f13d8101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:43.978 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 --2- 172.21.15.171:0/3856491458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f13d8103cc0 0x7f13d8101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:43.978 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.977+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 >> 172.21.15.171:0/3856491458 conn(0x7f13d80fd700 msgr2=0x7f13d80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:54:43.979 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.978+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 shutdown_connections 2024-09-10T21:54:43.979 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:54:43.978+0000 7f13dd400640 1 -- 172.21.15.171:0/3856491458 wait complete. 2024-09-10T21:54:44.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:44 smithi154 ceph-mon[34495]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:44.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:44.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:44.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:44.685 DEBUG:teuthology.orchestra.run.smithi171:osd.2> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T21:54:44.691 INFO:tasks.cephadm:Deploying osd.3 on smithi090 with /dev/vg_nvme/lv_3... 2024-09-10T21:54:44.691 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T21:54:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:44 smithi090 ceph-mon[30350]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:44.860 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:44 smithi171 ceph-mon[34138]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:44.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:54:44.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:44.861 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:45.047 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:54:45.180 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2[40414]: 2024-09-10T21:54:44.993+0000 7f6ad4e41640 -1 osd.2 0 log_to_monitors true 2024-09-10T21:54:45.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:45 smithi090 ceph-mon[30350]: from='osd.2 [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T21:54:45.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:45 smithi090 ceph-mon[30350]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T21:54:45.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:45 smithi171 ceph-mon[34138]: from='osd.2 [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T21:54:45.903 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:45 smithi171 ceph-mon[34138]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T21:54:45.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:45 smithi154 ceph-mon[34495]: from='osd.2 [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T21:54:45.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:45 smithi154 ceph-mon[34495]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T21:54:46.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:46 smithi171 ceph-mon[34138]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:46 smithi171 ceph-mon[34138]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T21:54:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:46 smithi171 ceph-mon[34138]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T21:54:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:46 smithi171 ceph-mon[34138]: from='osd.2 [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:54:46.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:46 smithi171 ceph-mon[34138]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:54:46.748 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2[40414]: 2024-09-10T21:54:46.625+0000 7f6acfa00640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T21:54:46.748 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2[40414]: 2024-09-10T21:54:46.630+0000 7f6acaa00640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:54:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:46 smithi090 ceph-mon[30350]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:46 smithi090 ceph-mon[30350]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T21:54:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:46 smithi090 ceph-mon[30350]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T21:54:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:46.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:46 smithi090 ceph-mon[30350]: from='osd.2 [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:54:46.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:46 smithi090 ceph-mon[30350]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:54:46.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:46 smithi154 ceph-mon[34495]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:46 smithi154 ceph-mon[34495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T21:54:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:46 smithi154 ceph-mon[34495]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T21:54:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:46 smithi154 ceph-mon[34495]: from='osd.2 [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:54:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:46 smithi154 ceph-mon[34495]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:54:47.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:47 smithi154 ceph-mon[34495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:54:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:47 smithi154 ceph-mon[34495]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T21:54:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:47 smithi171 ceph-mon[34138]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:54:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:47 smithi171 ceph-mon[34138]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T21:54:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:47.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:47 smithi090 ceph-mon[30350]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:54:47.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:47 smithi090 ceph-mon[30350]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T21:54:47.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:47.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:48.448 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:54:48.489 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-09-10T21:54:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:54:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:54:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: osd.2 [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] boot 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.861 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: osd.2 [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] boot 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:54:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:54:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: osd.2 [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] boot 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:54:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.407+0000 7f23e2e00640 1 -- 172.21.15.90:0/3618917781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc10b1a0 msgr2=0x7f23dc1119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.407+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3618917781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc10b1a0 0x7f23dc1119a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009ca0 tx=0x7f23d402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:54:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.407+0000 7f23e2e00640 1 -- 172.21.15.90:0/3618917781 shutdown_connections 2024-09-10T21:54:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.407+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3618917781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc10b1a0 0x7f23dc1119a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.407+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3618917781 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc10a820 0x7f23dc10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:49.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.407+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3618917781 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23dc104e10 0x7f23dc1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.407+0000 7f23e2e00640 1 -- 172.21.15.90:0/3618917781 >> 172.21.15.90:0/3618917781 conn(0x7f23dc100840 msgr2=0x7f23dc102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.408+0000 7f23e2e00640 1 -- 172.21.15.90:0/3618917781 shutdown_connections 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.408+0000 7f23e2e00640 1 -- 172.21.15.90:0/3618917781 wait complete. 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.408+0000 7f23e2e00640 1 Processor -- start 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e2e00640 1 -- start start 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e2e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc104e10 0x7f23dc19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e2e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc10a820 0x7f23dc19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e2e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23dc10b1a0 0x7f23dc1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e2e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc114d20 con 0x7f23dc104e10 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e2e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc114ba0 con 0x7f23dc10b1a0 2024-09-10T21:54:49.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e2e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc114ea0 con 0x7f23dc10a820 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e1a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc104e10 0x7f23dc19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e1a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc104e10 0x7f23dc19b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55612/0 (socket says 172.21.15.90:55612) 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e1a00640 1 -- 172.21.15.90:0/3439053279 learned_addr learned my addr 172.21.15.90:0/3439053279 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e1000640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc10a820 0x7f23dc19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.409+0000 7f23e2400640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23dc10b1a0 0x7f23dc1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23e1a00640 1 -- 172.21.15.90:0/3439053279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc10a820 msgr2=0x7f23dc19b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23e1a00640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc10a820 0x7f23dc19b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23e1a00640 1 -- 172.21.15.90:0/3439053279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23dc10b1a0 msgr2=0x7f23dc1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23e1a00640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23dc10b1a0 0x7f23dc1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23e1a00640 1 -- 172.21.15.90:0/3439053279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc1a1220 con 0x7f23dc104e10 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23e1a00640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc104e10 0x7f23dc19b170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f23d000ea60 tx=0x7f23d000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23da000640 1 -- 172.21.15.90:0/3439053279 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d000cde0 con 0x7f23dc104e10 2024-09-10T21:54:49.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc1a1510 con 0x7f23dc104e10 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23da000640 1 -- 172.21.15.90:0/3439053279 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f23d00035f0 con 0x7f23dc104e10 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23da000640 1 -- 172.21.15.90:0/3439053279 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0010690 con 0x7f23dc104e10 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.410+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc1a7bd0 con 0x7f23dc104e10 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.412+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc1051f0 con 0x7f23dc104e10 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.414+0000 7f23da000640 1 -- 172.21.15.90:0/3439053279 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 153211+0+0 (secure 0 0 0) 0x7f23d0004000 con 0x7f23dc104e10 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.415+0000 7f23da000640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f23c40b5340 0x7f23c40b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.415+0000 7f23da000640 1 -- 172.21.15.90:0/3439053279 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3441+0+0 (secure 0 0 0) 0x7f23d00e4ba0 con 0x7f23dc104e10 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.415+0000 7f23e1000640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f23c40b5340 0x7f23c40b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.416+0000 7f23e1000640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f23c40b5340 0x7f23c40b7800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f23dc1031c0 tx=0x7f23c8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:54:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.419+0000 7f23da000640 1 -- 172.21.15.90:0/3439053279 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f23d00acda0 con 0x7f23dc104e10 2024-09-10T21:54:49.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:49.618+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f23dc063590 con 0x7f23c40b5340 2024-09-10T21:54:49.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:49 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi171 2024-09-10T21:54:49.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:49 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi171 to 17089M 2024-09-10T21:54:49.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T21:54:49.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:49 smithi171 ceph-mon[34138]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T21:54:49.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T21:54:49.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:54:49.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:54:49.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:49 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi171 2024-09-10T21:54:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:49 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi171 to 17089M 2024-09-10T21:54:49.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T21:54:49.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:49 smithi090 ceph-mon[30350]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T21:54:49.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T21:54:49.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:54:49.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:54:49.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:49.858 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:54:49 smithi090 sudo[52932]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T21:54:49.858 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:54:49 smithi090 sudo[52932]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:49.858 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:54:49 smithi090 sudo[52932]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:49.858 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:54:49 smithi090 sudo[52932]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:49.858 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:54:49 smithi090 sudo[52956]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T21:54:49.858 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:54:49 smithi090 sudo[52956]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:49.859 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:54:49 smithi090 sudo[52956]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:49.859 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 21:54:49 smithi090 sudo[52956]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:49.918 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:49 smithi154 sudo[43981]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T21:54:49.918 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:49 smithi154 sudo[43981]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:49.918 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:49 smithi154 sudo[43981]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:49.918 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:49 smithi154 sudo[43981]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:49.918 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:49 smithi154 sudo[43984]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T21:54:49.919 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:49 smithi154 sudo[43984]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:49.919 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:49 smithi154 sudo[43984]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:49.919 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 21:54:49 smithi154 sudo[43984]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:49.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:49 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi171 2024-09-10T21:54:49.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:49 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi171 to 17089M 2024-09-10T21:54:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T21:54:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:49 smithi154 ceph-mon[34495]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T21:54:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T21:54:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:54:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:54:49.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:54:50.085 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:49 smithi171 sudo[44127]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T21:54:50.085 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:49 smithi171 sudo[44127]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:50.086 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:49 smithi171 sudo[44127]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:50.086 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:49 smithi171 sudo[44127]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:50.086 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:49 smithi171 sudo[44130]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T21:54:50.086 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:49 smithi171 sudo[44130]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:50.086 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:49 smithi171 sudo[44130]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:50.086 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 21:54:49 smithi171 sudo[44130]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 sudo[52995]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T21:54:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 sudo[52995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 sudo[52995]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:50.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 sudo[52995]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:50.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 sudo[53009]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T21:54:50.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 sudo[53009]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:50.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 sudo[53009]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:50.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 sudo[53009]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:50.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 sudo[44138]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T21:54:50.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 sudo[44138]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:50.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 sudo[44138]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:50.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 sudo[44138]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:50.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 sudo[44141]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T21:54:50.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 sudo[44141]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:50.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 sudo[44141]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:50.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 sudo[44141]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:50.636 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 sudo[43987]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T21:54:50.637 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 sudo[43987]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:50.637 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 sudo[43987]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:50.637 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 sudo[43987]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:50.637 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 sudo[43990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T21:54:50.637 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 sudo[43990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T21:54:50.637 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 sudo[43990]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T21:54:50.637 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 sudo[43990]: pam_unix(sudo:session): session closed for user root 2024-09-10T21:54:50.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:54:50.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:54:50.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T21:54:50.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:54:50.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:54:50.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:54:50.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T21:54:50.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T21:54:50.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:54:50.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:54:50.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:54:50.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:50 smithi090 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T21:54:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:54:50.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:50 smithi154 ceph-mon[34495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: pgmap v76: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T21:54:50.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T21:54:50.923 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:50 smithi171 ceph-mon[34138]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T21:54:51.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:54:51.659+0000 7f23da000640 1 -- 172.21.15.90:0/3439053279 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f23d0004320 con 0x7f23dc104e10 2024-09-10T21:54:51.757 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:51 smithi090 ceph-mon[30350]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T21:54:51.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:51 smithi154 ceph-mon[34495]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T21:54:51.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:51 smithi171 ceph-mon[34138]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T21:54:52.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:52 smithi090 ceph-mon[30350]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:54:52.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:52 smithi090 ceph-mon[30350]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T21:54:52.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:52 smithi090 ceph-mon[30350]: Cluster is now healthy 2024-09-10T21:54:52.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:52 smithi090 ceph-mon[30350]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-10T21:54:52.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:52 smithi154 ceph-mon[34495]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:54:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:52 smithi154 ceph-mon[34495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T21:54:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:52 smithi154 ceph-mon[34495]: Cluster is now healthy 2024-09-10T21:54:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:52 smithi154 ceph-mon[34495]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-10T21:54:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:52 smithi171 ceph-mon[34138]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:54:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:52 smithi171 ceph-mon[34138]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T21:54:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:52 smithi171 ceph-mon[34138]: Cluster is now healthy 2024-09-10T21:54:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:52 smithi171 ceph-mon[34138]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-10T21:54:53.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3499210666' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d6629c1-3c41-462a-84ce-21a906d40bb0"} : dispatch 2024-09-10T21:54:53.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3499210666' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d6629c1-3c41-462a-84ce-21a906d40bb0"}]': finished 2024-09-10T21:54:53.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:53 smithi090 ceph-mon[30350]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T21:54:53.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:54:53.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3882883989' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:54:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3499210666' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d6629c1-3c41-462a-84ce-21a906d40bb0"} : dispatch 2024-09-10T21:54:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3499210666' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d6629c1-3c41-462a-84ce-21a906d40bb0"}]': finished 2024-09-10T21:54:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:53 smithi154 ceph-mon[34495]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T21:54:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:54:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3882883989' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:54:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3499210666' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5d6629c1-3c41-462a-84ce-21a906d40bb0"} : dispatch 2024-09-10T21:54:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3499210666' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d6629c1-3c41-462a-84ce-21a906d40bb0"}]': finished 2024-09-10T21:54:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:53 smithi171 ceph-mon[34138]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T21:54:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:54:53.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3882883989' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:54:54.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:54 smithi154 ceph-mon[34495]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:54:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:54 smithi171 ceph-mon[34138]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:54:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:54 smithi090 ceph-mon[30350]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:54:56.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:56 smithi154 ceph-mon[34495]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 6.2 KiB/s rd, 269 KiB/s wr, 18 op/s 2024-09-10T21:54:56.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:56 smithi171 ceph-mon[34138]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 6.2 KiB/s rd, 269 KiB/s wr, 18 op/s 2024-09-10T21:54:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:56 smithi090 ceph-mon[30350]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 6.2 KiB/s rd, 269 KiB/s wr, 18 op/s 2024-09-10T21:54:58.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:54:58 smithi154 ceph-mon[34495]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 4.4 KiB/s rd, 190 KiB/s wr, 13 op/s 2024-09-10T21:54:58.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:54:58 smithi171 ceph-mon[34138]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 4.4 KiB/s rd, 190 KiB/s wr, 13 op/s 2024-09-10T21:54:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:54:58 smithi090 ceph-mon[30350]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 4.4 KiB/s rd, 190 KiB/s wr, 13 op/s 2024-09-10T21:55:00.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:00 smithi090 ceph-mon[30350]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 147 KiB/s wr, 10 op/s 2024-09-10T21:55:00.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:00 smithi171 ceph-mon[34138]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 147 KiB/s wr, 10 op/s 2024-09-10T21:55:01.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:00 smithi154 ceph-mon[34495]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 147 KiB/s wr, 10 op/s 2024-09-10T21:55:02.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:02 smithi090 ceph-mon[30350]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-10T21:55:02.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:02 smithi171 ceph-mon[34138]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-10T21:55:03.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:02 smithi154 ceph-mon[34495]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-09-10T21:55:03.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:03 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T21:55:03.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:03 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:03.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:03 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T21:55:03.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:03 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:03 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T21:55:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:03 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:04 smithi171 ceph-mon[34138]: Deploying daemon osd.3 on smithi090 2024-09-10T21:55:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:04 smithi171 ceph-mon[34138]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 123 KiB/s wr, 8 op/s 2024-09-10T21:55:05.058 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:04 smithi090 ceph-mon[30350]: Deploying daemon osd.3 on smithi090 2024-09-10T21:55:05.058 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:04 smithi090 ceph-mon[30350]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 123 KiB/s wr, 8 op/s 2024-09-10T21:55:05.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:04 smithi154 ceph-mon[34495]: Deploying daemon osd.3 on smithi090 2024-09-10T21:55:05.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:04 smithi154 ceph-mon[34495]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 123 KiB/s wr, 8 op/s 2024-09-10T21:55:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:06 smithi090 ceph-mon[30350]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-10T21:55:06.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:06 smithi171 ceph-mon[34138]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-10T21:55:07.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:06 smithi154 ceph-mon[34495]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-09-10T21:55:08.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:08 smithi171 ceph-mon[34138]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:08 smithi090 ceph-mon[30350]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:08 smithi154 ceph-mon[34495]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:10.932 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:10 smithi090 ceph-mon[30350]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:10.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:10 smithi171 ceph-mon[34138]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:11.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:10 smithi154 ceph-mon[34495]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:11.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:11 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:11.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:11 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:11.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:11 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:11.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:11 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:11.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:11 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:11.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:11 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:12.069 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:11 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:12.069 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:11 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:12.069 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:11 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:12.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:12 smithi090 ceph-mon[30350]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:12 smithi171 ceph-mon[34138]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:13.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:12 smithi154 ceph-mon[34495]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:14 smithi171 ceph-mon[34138]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:14 smithi090 ceph-mon[30350]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:14 smithi154 ceph-mon[34495]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:16.603 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 3 on host 'smithi090' 2024-09-10T21:55:16.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.601+0000 7f23da000640 1 -- 172.21.15.90:0/3439053279 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f23dc063590 con 0x7f23c40b5340 2024-09-10T21:55:16.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f23c40b5340 msgr2=0x7f23c40b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:16.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f23c40b5340 0x7f23c40b7800 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f23dc1031c0 tx=0x7f23c8005f90 comp rx=0 tx=0).stop 2024-09-10T21:55:16.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc104e10 msgr2=0x7f23dc19b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:16.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc104e10 0x7f23dc19b170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f23d000ea60 tx=0x7f23d000ef20 comp rx=0 tx=0).stop 2024-09-10T21:55:16.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 shutdown_connections 2024-09-10T21:55:16.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f23c40b5340 0x7f23c40b7800 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:16.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f23dc10b1a0 0x7f23dc1a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:16.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f23dc10a820 0x7f23dc19b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:16.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 --2- 172.21.15.90:0/3439053279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23dc104e10 0x7f23dc19b170 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:16.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 >> 172.21.15.90:0/3439053279 conn(0x7f23dc100840 msgr2=0x7f23dc100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:55:16.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 shutdown_connections 2024-09-10T21:55:16.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:55:16.604+0000 7f23e2e00640 1 -- 172.21.15.90:0/3439053279 wait complete. 2024-09-10T21:55:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:16 smithi090 ceph-mon[30350]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:16 smithi171 ceph-mon[34138]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:17.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:16 smithi154 ceph-mon[34495]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:17.514 DEBUG:teuthology.orchestra.run.smithi090:osd.3> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T21:55:17.519 INFO:tasks.cephadm:Deploying osd.4 on smithi154 with /dev/vg_nvme/lv_3... 2024-09-10T21:55:17.519 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T21:55:17.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:17 smithi090 ceph-mon[30350]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T21:55:17.856 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 21:55:17 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T21:55:17.644+0000 7facbbc2f640 -1 osd.3 0 log_to_monitors true 2024-09-10T21:55:17.898 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:55:17.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:17 smithi154 ceph-mon[34495]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T21:55:17.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:17 smithi171 ceph-mon[34138]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T21:55:18.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:18 smithi090 ceph-mon[30350]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:18.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:18 smithi090 ceph-mon[30350]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T21:55:18.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:18 smithi090 ceph-mon[30350]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T21:55:18.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:18 smithi090 ceph-mon[30350]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:55:18.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:18 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:18 smithi171 ceph-mon[34138]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:18 smithi171 ceph-mon[34138]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T21:55:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:18 smithi171 ceph-mon[34138]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T21:55:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:18 smithi171 ceph-mon[34138]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:55:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:18 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:19.038 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 21:55:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T21:55:18.744+0000 7facb6400640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T21:55:19.039 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 21:55:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T21:55:18.750+0000 7facb1400640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:55:19.054 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:18 smithi154 ceph-mon[34495]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:19.054 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:18 smithi154 ceph-mon[34495]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T21:55:19.055 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:18 smithi154 ceph-mon[34495]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T21:55:19.055 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:18 smithi154 ceph-mon[34495]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:55:19.055 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:18 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:19.977 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:19 smithi154 ceph-mon[34495]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:55:19.977 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:19 smithi154 ceph-mon[34495]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T21:55:19.977 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:19.977 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:19.977 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:19 smithi154 ceph-mon[34495]: osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] boot 2024-09-10T21:55:19.977 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:19 smithi154 ceph-mon[34495]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T21:55:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:19 smithi171 ceph-mon[34138]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:55:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:19 smithi171 ceph-mon[34138]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T21:55:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:19 smithi171 ceph-mon[34138]: osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] boot 2024-09-10T21:55:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:19 smithi171 ceph-mon[34138]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T21:55:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:19 smithi090 ceph-mon[30350]: from='osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:55:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:19 smithi090 ceph-mon[30350]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T21:55:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:19 smithi090 ceph-mon[30350]: osd.3 [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] boot 2024-09-10T21:55:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:19 smithi090 ceph-mon[30350]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T21:55:20.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:20 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:55:20.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:20 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:55:20.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:20 smithi090 ceph-mon[30350]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:20.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:20.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:20 smithi090 ceph-mon[30350]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T21:55:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:20 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:55:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:20 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:55:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:20 smithi171 ceph-mon[34138]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:20 smithi171 ceph-mon[34138]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T21:55:21.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:20 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:55:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:20 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:55:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:20 smithi154 ceph-mon[34495]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T21:55:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T21:55:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:20 smithi154 ceph-mon[34495]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T21:55:21.321 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:55:21.352 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_3 2024-09-10T21:55:21.716 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:55:22.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi090 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi090 to 8544M 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T21:55:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:22 smithi154 ceph-mon[34495]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:22.309 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.280+0000 7fcce1c00640 1 -- 172.21.15.154:0/4135976192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10a800 msgr2=0x7fccdc10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.280+0000 7fcce1c00640 1 --2- 172.21.15.154:0/4135976192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10a800 0x7fccdc10ac40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009900 tx=0x7fccc802a410 comp rx=0 tx=0).stop 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.280+0000 7fcce1c00640 1 -- 172.21.15.154:0/4135976192 shutdown_connections 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.280+0000 7fcce1c00640 1 --2- 172.21.15.154:0/4135976192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc10b180 0x7fccdc111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.280+0000 7fcce1c00640 1 --2- 172.21.15.154:0/4135976192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10a800 0x7fccdc10ac40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.280+0000 7fcce1c00640 1 --2- 172.21.15.154:0/4135976192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc104df0 0x7fccdc1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.280+0000 7fcce1c00640 1 -- 172.21.15.154:0/4135976192 >> 172.21.15.154:0/4135976192 conn(0x7fccdc100840 msgr2=0x7fccdc102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.280+0000 7fcce1c00640 1 -- 172.21.15.154:0/4135976192 shutdown_connections 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.280+0000 7fcce1c00640 1 -- 172.21.15.154:0/4135976192 wait complete. 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.281+0000 7fcce1c00640 1 Processor -- start 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.281+0000 7fcce1c00640 1 -- start start 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fcce1c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc104df0 0x7fccdc19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fcce1c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc10a800 0x7fccdc19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fcce1c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10b180 0x7fccdc1a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:55:22.310 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fcce1c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc114d00 con 0x7fccdc10a800 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fcce1c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc114b80 con 0x7fccdc10b180 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fcce1c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fccdc114e80 con 0x7fccdc104df0 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fcce1200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10b180 0x7fccdc1a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fccdbe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc104df0 0x7fccdc19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fcce1200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10b180 0x7fccdc1a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:52098/0 (socket says 172.21.15.154:52098) 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fcce1200640 1 -- 172.21.15.154:0/171546884 learned_addr learned my addr 172.21.15.154:0/171546884 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.282+0000 7fccdb400640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc10a800 0x7fccdc19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fccdbe00640 1 -- 172.21.15.154:0/171546884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10b180 msgr2=0x7fccdc1a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fccdbe00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10b180 0x7fccdc1a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fccdbe00640 1 -- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc10a800 msgr2=0x7fccdc19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fccdbe00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc10a800 0x7fccdc19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fccdbe00640 1 -- 172.21.15.154:0/171546884 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccdc1a11c0 con 0x7fccdc104df0 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fcce1200640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10b180 0x7fccdc1a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:55:22.311 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fccdbe00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc104df0 0x7fccdc19b110 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00d720 tx=0x7fcccc00dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fccd8c00640 1 -- 172.21.15.154:0/171546884 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc017070 con 0x7fccdc104df0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccdc1a14b0 con 0x7fccdc104df0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fccd8c00640 1 -- 172.21.15.154:0/171546884 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcccc00de30 con 0x7fccdc104df0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fccd8c00640 1 -- 172.21.15.154:0/171546884 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc004ca0 con 0x7fccdc104df0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.283+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccdc1a7b70 con 0x7fccdc104df0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.284+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccdc1051d0 con 0x7fccdc104df0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.286+0000 7fccd8c00640 1 -- 172.21.15.154:0/171546884 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fcccc004e40 con 0x7fccdc104df0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.288+0000 7fccd8c00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fccc00b53e0 0x7fccc00b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.288+0000 7fccd8c00640 1 -- 172.21.15.154:0/171546884 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3884+0+0 (secure 0 0 0) 0x7fcccc0e4410 con 0x7fccdc104df0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.288+0000 7fccdb400640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fccc00b53e0 0x7fccc00b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.289+0000 7fccdb400640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fccc00b53e0 0x7fccc00b78a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fccc8006010 tx=0x7fccc8005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:55:22.312 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.292+0000 7fccd8c00640 1 -- 172.21.15.154:0/171546884 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcccc0abd30 con 0x7fccdc104df0 2024-09-10T21:55:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi090 2024-09-10T21:55:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi090 to 8544M 2024-09-10T21:55:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T21:55:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:22 smithi090 ceph-mon[30350]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:22.489 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:22.488+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fccdc063590 con 0x7fccc00b53e0 2024-09-10T21:55:22.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi090 2024-09-10T21:55:22.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:22.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:22.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi090 to 8544M 2024-09-10T21:55:22.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:22.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:22.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:22.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T21:55:22.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:22 smithi171 ceph-mon[34138]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:23.537 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:23 smithi154 ceph-mon[34495]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:55:23.538 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:55:23.538 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:55:23.538 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:23.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:23 smithi171 ceph-mon[34138]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:55:23.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:55:23.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:55:23.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:23.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:23 smithi090 ceph-mon[30350]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:55:23.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:55:23.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:55:23.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:24.467 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:24 smithi154 ceph-mon[34495]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:24.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:24 smithi171 ceph-mon[34138]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:24.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:24 smithi090 ceph-mon[30350]: pgmap v101: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:26 smithi090 ceph-mon[30350]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:55:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:26 smithi090 ceph-mon[30350]: from='client.? 172.21.15.154:0/1136284177' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "99cff575-9f60-486c-b99a-48d5c3cf531c"} : dispatch 2024-09-10T21:55:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:26 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "99cff575-9f60-486c-b99a-48d5c3cf531c"} : dispatch 2024-09-10T21:55:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:26 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99cff575-9f60-486c-b99a-48d5c3cf531c"}]': finished 2024-09-10T21:55:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:26 smithi090 ceph-mon[30350]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T21:55:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:26 smithi090 ceph-mon[30350]: from='client.? 172.21.15.154:0/2756381577' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:55:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:26 smithi154 ceph-mon[34495]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:55:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:26 smithi154 ceph-mon[34495]: from='client.? 172.21.15.154:0/1136284177' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "99cff575-9f60-486c-b99a-48d5c3cf531c"} : dispatch 2024-09-10T21:55:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:26 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "99cff575-9f60-486c-b99a-48d5c3cf531c"} : dispatch 2024-09-10T21:55:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:26 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99cff575-9f60-486c-b99a-48d5c3cf531c"}]': finished 2024-09-10T21:55:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:26 smithi154 ceph-mon[34495]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T21:55:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:26 smithi154 ceph-mon[34495]: from='client.? 172.21.15.154:0/2756381577' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:55:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:26 smithi171 ceph-mon[34138]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:55:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:26 smithi171 ceph-mon[34138]: from='client.? 172.21.15.154:0/1136284177' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "99cff575-9f60-486c-b99a-48d5c3cf531c"} : dispatch 2024-09-10T21:55:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:26 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "99cff575-9f60-486c-b99a-48d5c3cf531c"} : dispatch 2024-09-10T21:55:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:26 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "99cff575-9f60-486c-b99a-48d5c3cf531c"}]': finished 2024-09-10T21:55:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:26 smithi171 ceph-mon[34138]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T21:55:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:26 smithi171 ceph-mon[34138]: from='client.? 172.21.15.154:0/2756381577' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:55:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:28 smithi090 ceph-mon[30350]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-10T21:55:28.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:28 smithi154 ceph-mon[34495]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-10T21:55:28.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:28 smithi171 ceph-mon[34138]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-10T21:55:30.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:30 smithi154 ceph-mon[34495]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-10T21:55:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:30 smithi090 ceph-mon[30350]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-10T21:55:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:30 smithi171 ceph-mon[34138]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-10T21:55:32.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:32 smithi090 ceph-mon[30350]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:55:32.902 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:32 smithi154 ceph-mon[34495]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:55:32.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:32 smithi171 ceph-mon[34138]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:55:34.822 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:34 smithi154 ceph-mon[34495]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:55:34.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:34 smithi090 ceph-mon[30350]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:55:34.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:34 smithi171 ceph-mon[34138]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:55:35.808 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T21:55:35.809 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:35.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T21:55:35.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:35.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T21:55:35.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:36.837 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:36 smithi154 ceph-mon[34495]: Deploying daemon osd.4 on smithi154 2024-09-10T21:55:36.837 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:36 smithi154 ceph-mon[34495]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:36 smithi090 ceph-mon[30350]: Deploying daemon osd.4 on smithi154 2024-09-10T21:55:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:36 smithi090 ceph-mon[30350]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:36 smithi171 ceph-mon[34138]: Deploying daemon osd.4 on smithi154 2024-09-10T21:55:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:36 smithi171 ceph-mon[34138]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:38.698 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:38 smithi154 ceph-mon[34495]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:38.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:38 smithi090 ceph-mon[30350]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:38.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:38 smithi171 ceph-mon[34138]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:40 smithi090 ceph-mon[30350]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:40 smithi154 ceph-mon[34495]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:40 smithi171 ceph-mon[34138]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:42.857 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:42 smithi154 ceph-mon[34495]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:42 smithi171 ceph-mon[34138]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:42 smithi090 ceph-mon[30350]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:43.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:43 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:43.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:43 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:43.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:43 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:43.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:43 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:43.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:43 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:43.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:43 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:44.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:43 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:44.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:43 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:44.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:43 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:44.900 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:44 smithi154 ceph-mon[34495]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:44.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:44 smithi171 ceph-mon[34138]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:44 smithi090 ceph-mon[30350]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:46.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:46 smithi154 ceph-mon[34495]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:46.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:46 smithi171 ceph-mon[34138]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:46.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:46.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:46 smithi090 ceph-mon[30350]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:47.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:48.138 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 4 on host 'smithi154' 2024-09-10T21:55:48.138 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.136+0000 7fccd8c00640 1 -- 172.21.15.154:0/171546884 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fccdc063590 con 0x7fccc00b53e0 2024-09-10T21:55:48.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fccc00b53e0 msgr2=0x7fccc00b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:48.140 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fccc00b53e0 0x7fccc00b78a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fccc8006010 tx=0x7fccc8005e80 comp rx=0 tx=0).stop 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc104df0 msgr2=0x7fccdc19b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc104df0 0x7fccdc19b110 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00d720 tx=0x7fcccc00dbe0 comp rx=0 tx=0).stop 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 shutdown_connections 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fccc00b53e0 0x7fccc00b78a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fccdc10b180 0x7fccdc1a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc10a800 0x7fccdc19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 --2- 172.21.15.154:0/171546884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fccdc104df0 0x7fccdc19b110 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 >> 172.21.15.154:0/171546884 conn(0x7fccdc100840 msgr2=0x7fccdc102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 shutdown_connections 2024-09-10T21:55:48.141 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:55:48.139+0000 7fcce1c00640 1 -- 172.21.15.154:0/171546884 wait complete. 2024-09-10T21:55:48.905 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:48 smithi154 ceph-mon[34495]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:48.905 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:48.905 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:48.905 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:48.947 DEBUG:teuthology.orchestra.run.smithi154:osd.4> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T21:55:48.952 INFO:tasks.cephadm:Deploying osd.5 on smithi171 with /dev/vg_nvme/lv_3... 2024-09-10T21:55:48.952 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T21:55:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:48 smithi171 ceph-mon[34138]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:48 smithi090 ceph-mon[30350]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:55:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:49.323 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:55:49.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:49 smithi171 ceph-mon[34138]: from='osd.4 [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T21:55:49.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:49 smithi171 ceph-mon[34138]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T21:55:49.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:49 smithi154 ceph-mon[34495]: from='osd.4 [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T21:55:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:49 smithi154 ceph-mon[34495]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T21:55:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:49 smithi090 ceph-mon[30350]: from='osd.4 [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T21:55:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:49 smithi090 ceph-mon[30350]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T21:55:50.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:50 smithi171 ceph-mon[34138]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:50.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:50 smithi171 ceph-mon[34138]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T21:55:50.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:50 smithi171 ceph-mon[34138]: from='osd.4 [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:55:50.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:50 smithi171 ceph-mon[34138]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T21:55:50.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:50.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:50 smithi171 ceph-mon[34138]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:55:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:50 smithi154 ceph-mon[34495]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:50 smithi154 ceph-mon[34495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T21:55:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:50 smithi154 ceph-mon[34495]: from='osd.4 [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:55:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:50 smithi154 ceph-mon[34495]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T21:55:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:50 smithi154 ceph-mon[34495]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:55:50.918 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 21:55:50 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T21:55:50.648+0000 7ff64a800640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T21:55:50.919 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 21:55:50 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T21:55:50.655+0000 7ff645800640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:55:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:50 smithi090 ceph-mon[30350]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:50 smithi090 ceph-mon[30350]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T21:55:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:50 smithi090 ceph-mon[30350]: from='osd.4 [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:55:51.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:50 smithi090 ceph-mon[30350]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T21:55:51.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:51.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:50 smithi090 ceph-mon[30350]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:55:51.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:51 smithi154 ceph-mon[34495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:55:51.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:51 smithi154 ceph-mon[34495]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T21:55:51.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:51.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:51.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:51 smithi171 ceph-mon[34138]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:55:51.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:51 smithi171 ceph-mon[34138]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T21:55:51.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:51.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:51 smithi090 ceph-mon[30350]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:55:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:51 smithi090 ceph-mon[30350]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T21:55:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:52.475 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:55:52.509 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_3 2024-09-10T21:55:52.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:52 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:55:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:52 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:55:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:52 smithi171 ceph-mon[34138]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:52 smithi171 ceph-mon[34138]: osd.4 [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] boot 2024-09-10T21:55:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:52 smithi171 ceph-mon[34138]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T21:55:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:52.917 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:55:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:52 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:55:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:52 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:55:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:52 smithi154 ceph-mon[34495]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:52 smithi154 ceph-mon[34495]: osd.4 [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] boot 2024-09-10T21:55:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:52 smithi154 ceph-mon[34495]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T21:55:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:52 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:55:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:52 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:55:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:52 smithi090 ceph-mon[30350]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T21:55:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:52 smithi090 ceph-mon[30350]: osd.4 [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] boot 2024-09-10T21:55:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:52 smithi090 ceph-mon[30350]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T21:55:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T21:55:53.477 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.476+0000 7f5842800640 1 -- 172.21.15.171:0/2870381302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c1087b0 msgr2=0x7f583c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:53.478 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.476+0000 7f5842800640 1 --2- 172.21.15.171:0/2870381302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c1087b0 0x7f583c108bf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5828009900 tx=0x7f582802a410 comp rx=0 tx=0).stop 2024-09-10T21:55:53.478 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.476+0000 7f5842800640 1 -- 172.21.15.171:0/2870381302 shutdown_connections 2024-09-10T21:55:53.478 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.476+0000 7f5842800640 1 --2- 172.21.15.171:0/2870381302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583c109130 0x7f583c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:53.478 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.476+0000 7f5842800640 1 --2- 172.21.15.171:0/2870381302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c1087b0 0x7f583c108bf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:53.478 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.476+0000 7f5842800640 1 --2- 172.21.15.171:0/2870381302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583c102da0 0x7f583c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:53.478 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.476+0000 7f5842800640 1 -- 172.21.15.171:0/2870381302 >> 172.21.15.171:0/2870381302 conn(0x7f583c0fe810 msgr2=0x7f583c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:55:53.478 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.476+0000 7f5842800640 1 -- 172.21.15.171:0/2870381302 shutdown_connections 2024-09-10T21:55:53.478 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.476+0000 7f5842800640 1 -- 172.21.15.171:0/2870381302 wait complete. 2024-09-10T21:55:53.479 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f5842800640 1 Processor -- start 2024-09-10T21:55:53.479 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f5842800640 1 -- start start 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f5842800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c102da0 0x7f583c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f5842800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583c1087b0 0x7f583c19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f5842800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583c109130 0x7f583c1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f5842800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f583c112c80 con 0x7f583c109130 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f5842800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f583c112b00 con 0x7f583c1087b0 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f5842800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f583c112e00 con 0x7f583c102da0 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f583b400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c102da0 0x7f583c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f583b400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c102da0 0x7f583c19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:43136/0 (socket says 172.21.15.171:43136) 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f583b400640 1 -- 172.21.15.171:0/2691855747 learned_addr learned my addr 172.21.15.171:0/2691855747 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.477+0000 7f583aa00640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583c1087b0 0x7f583c19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:55:53.480 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f583be00640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583c109130 0x7f583c1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:55:53.481 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f583b400640 1 -- 172.21.15.171:0/2691855747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583c1087b0 msgr2=0x7f583c19b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:53.481 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f583b400640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583c1087b0 0x7f583c19b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:53.481 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f583b400640 1 -- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583c109130 msgr2=0x7f583c1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:55:53.481 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f583b400640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583c109130 0x7f583c1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:55:53.481 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f583b400640 1 -- 172.21.15.171:0/2691855747 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f583c1a1490 con 0x7f583c102da0 2024-09-10T21:55:53.481 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f583b400640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c102da0 0x7f583c19b3e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f582c00d950 tx=0x7f582c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:55:53.482 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f5833e00640 1 -- 172.21.15.171:0/2691855747 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c017030 con 0x7f583c102da0 2024-09-10T21:55:53.482 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f5833e00640 1 -- 172.21.15.171:0/2691855747 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f582c004020 con 0x7f583c102da0 2024-09-10T21:55:53.482 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f5833e00640 1 -- 172.21.15.171:0/2691855747 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582c0055d0 con 0x7f583c102da0 2024-09-10T21:55:53.482 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.478+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f583c1a1780 con 0x7f583c102da0 2024-09-10T21:55:53.482 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.479+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f583c1a7e40 con 0x7f583c102da0 2024-09-10T21:55:53.482 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.480+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f583c103180 con 0x7f583c102da0 2024-09-10T21:55:53.483 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.482+0000 7f5833e00640 1 -- 172.21.15.171:0/2691855747 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f582c009a30 con 0x7f583c102da0 2024-09-10T21:55:53.485 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.483+0000 7f5833e00640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58200b5330 0x7f58200b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:55:53.485 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.484+0000 7f5833e00640 1 -- 172.21.15.171:0/2691855747 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4316+0+0 (secure 0 0 0) 0x7f582c0e4e30 con 0x7f583c102da0 2024-09-10T21:55:53.485 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.484+0000 7f583aa00640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58200b5330 0x7f58200b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:55:53.486 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.485+0000 7f583aa00640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58200b5330 0x7f58200b77f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5828006010 tx=0x7f5828005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:55:53.487 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.486+0000 7f5833e00640 1 -- 172.21.15.171:0/2691855747 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f582c0acbf0 con 0x7f583c102da0 2024-09-10T21:55:53.685 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:55:53.683+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f583c063590 con 0x7f58200b5330 2024-09-10T21:55:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:53 smithi154 ceph-mon[34495]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T21:55:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:53 smithi171 ceph-mon[34138]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T21:55:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:53.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:53.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:53.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:54.007 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:53 smithi090 ceph-mon[30350]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T21:55:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:55:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:55:54.008 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:55:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:54 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi154 2024-09-10T21:55:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:54 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi154 to 8544M 2024-09-10T21:55:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:54 smithi154 ceph-mon[34495]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:55:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:54 smithi154 ceph-mon[34495]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:55:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:55:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:55:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:54 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi154 2024-09-10T21:55:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:54 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi154 to 8544M 2024-09-10T21:55:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:54 smithi171 ceph-mon[34138]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:55:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:54 smithi171 ceph-mon[34138]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:55:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:55:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:55:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:54 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi154 2024-09-10T21:55:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:54 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi154 to 8544M 2024-09-10T21:55:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:54 smithi090 ceph-mon[30350]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:55:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:54 smithi090 ceph-mon[30350]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:55:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:55:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:55:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:55:56.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:56 smithi154 ceph-mon[34495]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:55:56.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:56 smithi171 ceph-mon[34138]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:55:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:56 smithi090 ceph-mon[30350]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:55:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:57 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/1907148523' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f7931a53-09a3-408d-91c4-b85dd2fec784"} : dispatch 2024-09-10T21:55:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:57 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f7931a53-09a3-408d-91c4-b85dd2fec784"} : dispatch 2024-09-10T21:55:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:57 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7931a53-09a3-408d-91c4-b85dd2fec784"}]': finished 2024-09-10T21:55:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:57 smithi154 ceph-mon[34495]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T21:55:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:55:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:57 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/2128919763' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:55:57.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:57 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/1907148523' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f7931a53-09a3-408d-91c4-b85dd2fec784"} : dispatch 2024-09-10T21:55:57.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:57 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f7931a53-09a3-408d-91c4-b85dd2fec784"} : dispatch 2024-09-10T21:55:57.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:57 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7931a53-09a3-408d-91c4-b85dd2fec784"}]': finished 2024-09-10T21:55:57.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:57 smithi171 ceph-mon[34138]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T21:55:57.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:55:57.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:57 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/2128919763' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:55:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:57 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/1907148523' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f7931a53-09a3-408d-91c4-b85dd2fec784"} : dispatch 2024-09-10T21:55:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:57 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f7931a53-09a3-408d-91c4-b85dd2fec784"} : dispatch 2024-09-10T21:55:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:57 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f7931a53-09a3-408d-91c4-b85dd2fec784"}]': finished 2024-09-10T21:55:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:57 smithi090 ceph-mon[30350]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T21:55:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:55:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:57 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/2128919763' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:55:58.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:55:58 smithi154 ceph-mon[34495]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:55:58.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:55:58 smithi171 ceph-mon[34138]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:55:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:55:58 smithi090 ceph-mon[30350]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:00.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:00 smithi171 ceph-mon[34138]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:00 smithi090 ceph-mon[30350]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:01.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:00 smithi154 ceph-mon[34495]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:02.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:02 smithi171 ceph-mon[34138]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:02 smithi090 ceph-mon[30350]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:02 smithi154 ceph-mon[34495]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:04.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:04 smithi171 ceph-mon[34138]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:04 smithi090 ceph-mon[30350]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:05.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:04 smithi154 ceph-mon[34495]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:06.718 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:06 smithi171 ceph-mon[34138]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:06.718 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:06 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T21:56:06.718 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:06 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:06 smithi090 ceph-mon[30350]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:06 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T21:56:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:06 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:07.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:06 smithi154 ceph-mon[34495]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:06 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T21:56:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:06 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:07.872 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:07 smithi171 ceph-mon[34138]: Deploying daemon osd.5 on smithi171 2024-09-10T21:56:08.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:07 smithi090 ceph-mon[30350]: Deploying daemon osd.5 on smithi171 2024-09-10T21:56:08.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:07 smithi154 ceph-mon[34495]: Deploying daemon osd.5 on smithi171 2024-09-10T21:56:08.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:08 smithi171 ceph-mon[34138]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:08 smithi090 ceph-mon[30350]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:08 smithi154 ceph-mon[34495]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:10.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:10 smithi171 ceph-mon[34138]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:10 smithi090 ceph-mon[30350]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:11.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:10 smithi154 ceph-mon[34495]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:12.970 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:12 smithi154 ceph-mon[34495]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:12.981 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:12 smithi171 ceph-mon[34138]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:12 smithi090 ceph-mon[30350]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:14 smithi171 ceph-mon[34138]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:14 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:14 smithi090 ceph-mon[30350]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:14 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:15.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:14 smithi154 ceph-mon[34495]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:14 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:16 smithi171 ceph-mon[34138]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:16 smithi090 ceph-mon[30350]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:16 smithi154 ceph-mon[34495]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:18.503 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.502+0000 7f5833e00640 1 -- 172.21.15.171:0/2691855747 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f583c063590 con 0x7f58200b5330 2024-09-10T21:56:18.503 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 5 on host 'smithi171' 2024-09-10T21:56:18.505 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.504+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58200b5330 msgr2=0x7f58200b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:18.505 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.504+0000 7f5842800640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58200b5330 0x7f58200b77f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5828006010 tx=0x7f5828005e80 comp rx=0 tx=0).stop 2024-09-10T21:56:18.505 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c102da0 msgr2=0x7f583c19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:18.505 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c102da0 0x7f583c19b3e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f582c00d950 tx=0x7f582c00de10 comp rx=0 tx=0).stop 2024-09-10T21:56:18.505 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 shutdown_connections 2024-09-10T21:56:18.506 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58200b5330 0x7f58200b77f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:18.506 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f583c109130 0x7f583c1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:18.506 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f583c1087b0 0x7f583c19b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:18.506 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 --2- 172.21.15.171:0/2691855747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f583c102da0 0x7f583c19b3e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:18.506 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 >> 172.21.15.171:0/2691855747 conn(0x7f583c0fe810 msgr2=0x7f583c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:56:18.506 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 shutdown_connections 2024-09-10T21:56:18.506 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:56:18.505+0000 7f5842800640 1 -- 172.21.15.171:0/2691855747 wait complete. 2024-09-10T21:56:18.962 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:18 smithi171 ceph-mon[34138]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:18.962 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:18 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:18.963 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:18 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:18.963 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:18 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:18 smithi090 ceph-mon[30350]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:18 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:18 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:18 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:19.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:18 smithi154 ceph-mon[34495]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:18 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:18 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:18 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:19.319 DEBUG:teuthology.orchestra.run.smithi171:osd.5> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T21:56:19.323 INFO:tasks.cephadm:Deploying osd.6 on smithi090 with /dev/vg_nvme/lv_2... 2024-09-10T21:56:19.323 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T21:56:19.552 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 21:56:19 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T21:56:19.446+0000 7f8f65c11640 -1 osd.5 0 log_to_monitors true 2024-09-10T21:56:19.695 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:56:19.873 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:19 smithi171 ceph-mon[34138]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T21:56:20.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:19 smithi090 ceph-mon[30350]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T21:56:20.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:19 smithi154 ceph-mon[34495]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T21:56:20.914 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:20 smithi090 ceph-mon[30350]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:20.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:20 smithi090 ceph-mon[30350]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T21:56:20.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:20 smithi090 ceph-mon[30350]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T21:56:20.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:20 smithi090 ceph-mon[30350]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:56:20.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:20.976 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:20 smithi171 ceph-mon[34138]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:20.977 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:20 smithi171 ceph-mon[34138]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T21:56:20.977 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:20 smithi171 ceph-mon[34138]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T21:56:20.977 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:20 smithi171 ceph-mon[34138]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:56:20.977 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:20 smithi154 ceph-mon[34495]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:20 smithi154 ceph-mon[34495]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T21:56:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:20 smithi154 ceph-mon[34495]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T21:56:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:20 smithi154 ceph-mon[34495]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:56:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:21.228 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 21:56:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T21:56:21.207+0000 7f8f60800640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T21:56:21.228 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 21:56:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T21:56:21.227+0000 7f8f56800640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:56:21.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:21 smithi171 ceph-mon[34138]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:56:21.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:21 smithi171 ceph-mon[34138]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T21:56:21.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:21 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:21.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:21 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:21.788 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:21 smithi171 ceph-mon[34138]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' 2024-09-10T21:56:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:21 smithi090 ceph-mon[30350]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:56:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:21 smithi090 ceph-mon[30350]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T21:56:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:21 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:21 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:21 smithi090 ceph-mon[30350]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' 2024-09-10T21:56:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:21 smithi154 ceph-mon[34495]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:56:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:21 smithi154 ceph-mon[34495]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T21:56:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:21 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:21 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:21 smithi154 ceph-mon[34495]: from='osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699]' entity='osd.5' 2024-09-10T21:56:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:22 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:56:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:22 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:56:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:22 smithi171 ceph-mon[34138]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:22.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:22 smithi171 ceph-mon[34138]: osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] boot 2024-09-10T21:56:22.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:22 smithi171 ceph-mon[34138]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T21:56:22.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:22 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:56:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:22 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:56:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:22 smithi090 ceph-mon[30350]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:22 smithi090 ceph-mon[30350]: osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] boot 2024-09-10T21:56:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:22 smithi090 ceph-mon[30350]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T21:56:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:22 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:56:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:22 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:56:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:22 smithi154 ceph-mon[34495]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T21:56:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:22 smithi154 ceph-mon[34495]: osd.5 [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] boot 2024-09-10T21:56:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:22 smithi154 ceph-mon[34495]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T21:56:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T21:56:23.248 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:56:23.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-09-10T21:56:23.644 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:56:24.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T21:56:24.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi171 2024-09-10T21:56:24.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:24.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:24.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi171 to 8544M 2024-09-10T21:56:24.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:24.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:24.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:23 smithi090 ceph-mon[30350]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T21:56:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi171 2024-09-10T21:56:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi171 to 8544M 2024-09-10T21:56:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:24.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:24.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:23 smithi154 ceph-mon[34495]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.174+0000 7fe01f200640 1 -- 172.21.15.90:0/3832109073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe01810a9a0 msgr2=0x7fe01810ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.174+0000 7fe01f200640 1 --2- 172.21.15.90:0/3832109073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe01810a9a0 0x7fe01810ade0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe000009900 tx=0x7fe00002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:56:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.174+0000 7fe01f200640 1 -- 172.21.15.90:0/3832109073 shutdown_connections 2024-09-10T21:56:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.174+0000 7fe01f200640 1 --2- 172.21.15.90:0/3832109073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01810b320 0x7fe018111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.174+0000 7fe01f200640 1 --2- 172.21.15.90:0/3832109073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe01810a9a0 0x7fe01810ade0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.174+0000 7fe01f200640 1 --2- 172.21.15.90:0/3832109073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe018104f90 0x7fe018105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.174+0000 7fe01f200640 1 -- 172.21.15.90:0/3832109073 >> 172.21.15.90:0/3832109073 conn(0x7fe018100a00 msgr2=0x7fe018102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:56:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.174+0000 7fe01f200640 1 -- 172.21.15.90:0/3832109073 shutdown_connections 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.174+0000 7fe01f200640 1 -- 172.21.15.90:0/3832109073 wait complete. 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe01f200640 1 Processor -- start 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe01f200640 1 -- start start 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe01f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe018104f90 0x7fe01819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe01f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe01810a9a0 0x7fe01819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe01f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01810b320 0x7fe0181a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe01f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe018114e20 con 0x7fe018104f90 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe01f200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe018114ca0 con 0x7fe01810b320 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe01f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe018114fa0 con 0x7fe01810a9a0 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe017e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe018104f90 0x7fe01819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe017e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe018104f90 0x7fe01819b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54846/0 (socket says 172.21.15.90:54846) 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.175+0000 7fe017e00640 1 -- 172.21.15.90:0/3224313238 learned_addr learned my addr 172.21.15.90:0/3224313238 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:56:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe00fe00640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe01810a9a0 0x7fe01819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe01d200640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01810b320 0x7fe0181a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe017e00640 1 -- 172.21.15.90:0/3224313238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe01810a9a0 msgr2=0x7fe01819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe017e00640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe01810a9a0 0x7fe01819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe017e00640 1 -- 172.21.15.90:0/3224313238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01810b320 msgr2=0x7fe0181a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe017e00640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01810b320 0x7fe0181a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe017e00640 1 -- 172.21.15.90:0/3224313238 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0181a13c0 con 0x7fe018104f90 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe017e00640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe018104f90 0x7fe01819b310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c920 tx=0x7fe00800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe015600640 1 -- 172.21.15.90:0/3224313238 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008007c70 con 0x7fe018104f90 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe015600640 1 -- 172.21.15.90:0/3224313238 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe008007e10 con 0x7fe018104f90 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0181a16b0 con 0x7fe018104f90 2024-09-10T21:56:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.176+0000 7fe015600640 1 -- 172.21.15.90:0/3224313238 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008018710 con 0x7fe018104f90 2024-09-10T21:56:24.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.177+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0181a7f40 con 0x7fe018104f90 2024-09-10T21:56:24.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.178+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe018105370 con 0x7fe018104f90 2024-09-10T21:56:24.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.180+0000 7fe015600640 1 -- 172.21.15.90:0/3224313238 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fe00800f450 con 0x7fe018104f90 2024-09-10T21:56:24.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.181+0000 7fe015600640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdffc0b53b0 0x7fdffc0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:56:24.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.181+0000 7fe015600640 1 -- 172.21.15.90:0/3224313238 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4748+0+0 (secure 0 0 0) 0x7fe0080e4fa0 con 0x7fe018104f90 2024-09-10T21:56:24.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.182+0000 7fe00fe00640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdffc0b53b0 0x7fdffc0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:56:24.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.182+0000 7fe00fe00640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdffc0b53b0 0x7fdffc0b7870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe000006010 tx=0x7fe000005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:56:24.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.185+0000 7fe015600640 1 -- 172.21.15.90:0/3224313238 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe008014030 con 0x7fe018104f90 2024-09-10T21:56:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T21:56:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi171 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi171 to 8544M 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:23 smithi171 ceph-mon[34138]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:24.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:24.382+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe018063590 con 0x7fdffc0b53b0 2024-09-10T21:56:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:24 smithi154 ceph-mon[34495]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T21:56:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:24 smithi154 ceph-mon[34495]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:56:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:56:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:56:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:24.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:24 smithi090 ceph-mon[30350]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T21:56:24.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:24 smithi090 ceph-mon[30350]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:56:24.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:56:24.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:56:24.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:24 smithi171 ceph-mon[34138]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T21:56:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:24 smithi171 ceph-mon[34138]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:56:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:56:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:56:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:26.010 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:25 smithi090 ceph-mon[30350]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-10T21:56:26.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:25 smithi154 ceph-mon[34495]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-10T21:56:26.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:25 smithi171 ceph-mon[34138]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-09-10T21:56:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:28 smithi090 ceph-mon[30350]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:56:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:28 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3653185284' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2bdf9899-a84f-42fd-a62f-6c19c39f8ba4"} : dispatch 2024-09-10T21:56:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:28 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3653185284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bdf9899-a84f-42fd-a62f-6c19c39f8ba4"}]': finished 2024-09-10T21:56:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:28 smithi090 ceph-mon[30350]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T21:56:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:28 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:28.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:28 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4034569484' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:56:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:28 smithi154 ceph-mon[34495]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:56:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:28 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3653185284' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2bdf9899-a84f-42fd-a62f-6c19c39f8ba4"} : dispatch 2024-09-10T21:56:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:28 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3653185284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bdf9899-a84f-42fd-a62f-6c19c39f8ba4"}]': finished 2024-09-10T21:56:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:28 smithi154 ceph-mon[34495]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T21:56:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:28 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:28 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4034569484' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:56:28.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:28 smithi171 ceph-mon[34138]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:56:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:28 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3653185284' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2bdf9899-a84f-42fd-a62f-6c19c39f8ba4"} : dispatch 2024-09-10T21:56:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:28 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3653185284' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bdf9899-a84f-42fd-a62f-6c19c39f8ba4"}]': finished 2024-09-10T21:56:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:28 smithi171 ceph-mon[34138]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T21:56:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:28 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:28 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4034569484' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:56:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:30 smithi090 ceph-mon[30350]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T21:56:30.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:30 smithi154 ceph-mon[34495]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T21:56:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:30 smithi171 ceph-mon[34138]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T21:56:32.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:32 smithi090 ceph-mon[30350]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T21:56:32.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:32 smithi154 ceph-mon[34495]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T21:56:32.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:32 smithi171 ceph-mon[34138]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T21:56:34.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:34 smithi090 ceph-mon[30350]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T21:56:34.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:34 smithi154 ceph-mon[34495]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T21:56:34.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:34 smithi171 ceph-mon[34138]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T21:56:36.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:36 smithi171 ceph-mon[34138]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:36 smithi090 ceph-mon[30350]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:36 smithi154 ceph-mon[34495]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:37.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T21:56:37.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:37.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T21:56:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T21:56:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:38.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:38 smithi090 ceph-mon[30350]: Deploying daemon osd.6 on smithi090 2024-09-10T21:56:38.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:38 smithi090 ceph-mon[30350]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:38.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:38 smithi154 ceph-mon[34495]: Deploying daemon osd.6 on smithi090 2024-09-10T21:56:38.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:38 smithi154 ceph-mon[34495]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:38.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:38 smithi171 ceph-mon[34138]: Deploying daemon osd.6 on smithi090 2024-09-10T21:56:38.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:38 smithi171 ceph-mon[34138]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:40.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:40 smithi090 ceph-mon[30350]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:40 smithi154 ceph-mon[34495]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:40 smithi171 ceph-mon[34138]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:42.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:42 smithi154 ceph-mon[34495]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:42 smithi171 ceph-mon[34138]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:42 smithi090 ceph-mon[30350]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:44.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:44 smithi154 ceph-mon[34495]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:44.967 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:44 smithi090 ceph-mon[30350]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:44.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:44 smithi171 ceph-mon[34138]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:45.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:45 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:45.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:45 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:45.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:45 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:45.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:45 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:45.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:45 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:45.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:45 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:46.066 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:45 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:46.066 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:45 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:46.066 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:45 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:46.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:46 smithi090 ceph-mon[30350]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:46 smithi154 ceph-mon[34495]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:46.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:46 smithi171 ceph-mon[34138]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:48 smithi090 ceph-mon[30350]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:48.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:48 smithi154 ceph-mon[34495]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:48 smithi171 ceph-mon[34138]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:50.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.078+0000 7fe015600640 1 -- 172.21.15.90:0/3224313238 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe018063590 con 0x7fdffc0b53b0 2024-09-10T21:56:50.080 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 6 on host 'smithi090' 2024-09-10T21:56:50.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdffc0b53b0 msgr2=0x7fdffc0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:50.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdffc0b53b0 0x7fdffc0b7870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe000006010 tx=0x7fe000005e80 comp rx=0 tx=0).stop 2024-09-10T21:56:50.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe018104f90 msgr2=0x7fe01819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:50.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe018104f90 0x7fe01819b310 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe00800c920 tx=0x7fe00800cde0 comp rx=0 tx=0).stop 2024-09-10T21:56:50.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 shutdown_connections 2024-09-10T21:56:50.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdffc0b53b0 0x7fdffc0b7870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:50.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe01810b320 0x7fe0181a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:50.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe01810a9a0 0x7fe01819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:50.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 --2- 172.21.15.90:0/3224313238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe018104f90 0x7fe01819b310 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:50.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 >> 172.21.15.90:0/3224313238 conn(0x7fe018100a00 msgr2=0x7fe018102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:56:50.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 shutdown_connections 2024-09-10T21:56:50.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:56:50.082+0000 7fe01f200640 1 -- 172.21.15.90:0/3224313238 wait complete. 2024-09-10T21:56:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:50 smithi090 ceph-mon[30350]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:50.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:50 smithi154 ceph-mon[34495]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:50.930 DEBUG:teuthology.orchestra.run.smithi090:osd.6> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T21:56:50.934 INFO:tasks.cephadm:Deploying osd.7 on smithi154 with /dev/vg_nvme/lv_2... 2024-09-10T21:56:50.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T21:56:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:50 smithi171 ceph-mon[34138]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:56:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:51.296 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:56:51.754 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:51 smithi154 ceph-mon[34495]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T21:56:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:51 smithi090 ceph-mon[30350]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T21:56:51.755 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 21:56:51 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T21:56:51.440+0000 7f63f342d640 -1 osd.6 0 log_to_monitors true 2024-09-10T21:56:51.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:51 smithi171 ceph-mon[34138]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T21:56:52.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:52 smithi154 ceph-mon[34495]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:52 smithi154 ceph-mon[34495]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T21:56:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:52 smithi154 ceph-mon[34495]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T21:56:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:52 smithi154 ceph-mon[34495]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:56:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:52 smithi171 ceph-mon[34138]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:52.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:52 smithi171 ceph-mon[34138]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T21:56:52.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:52 smithi171 ceph-mon[34138]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T21:56:52.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:52 smithi171 ceph-mon[34138]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:56:52.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:53.006 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:52 smithi090 ceph-mon[30350]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:53.006 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:52 smithi090 ceph-mon[30350]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T21:56:53.006 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:52 smithi090 ceph-mon[30350]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T21:56:53.006 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:52 smithi090 ceph-mon[30350]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:56:53.006 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:53.356 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 21:56:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T21:56:53.159+0000 7f63edc00640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T21:56:53.356 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 21:56:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T21:56:53.178+0000 7f63e3c00640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:56:53.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:53 smithi154 ceph-mon[34495]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:56:53.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:53 smithi154 ceph-mon[34495]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T21:56:53.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:53.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:53.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:53 smithi154 ceph-mon[34495]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' 2024-09-10T21:56:53.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:53 smithi090 ceph-mon[30350]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:56:53.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:53 smithi090 ceph-mon[30350]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T21:56:53.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:53.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:53.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:53 smithi090 ceph-mon[30350]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' 2024-09-10T21:56:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:53 smithi171 ceph-mon[34138]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:56:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:53 smithi171 ceph-mon[34138]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T21:56:53.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:53.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:53.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:53 smithi171 ceph-mon[34138]: from='osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003]' entity='osd.6' 2024-09-10T21:56:54.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:54 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:56:54.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:54 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:56:54.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:54 smithi154 ceph-mon[34495]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:54.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:54.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:54 smithi154 ceph-mon[34495]: osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] boot 2024-09-10T21:56:54.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:54 smithi154 ceph-mon[34495]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T21:56:54.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:54.826 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:56:54.858 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_2 2024-09-10T21:56:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:54 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:56:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:54 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:56:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:54 smithi171 ceph-mon[34138]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:54 smithi171 ceph-mon[34138]: osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] boot 2024-09-10T21:56:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:54 smithi171 ceph-mon[34138]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T21:56:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:54 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:56:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:54 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:56:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:54 smithi090 ceph-mon[30350]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-10T21:56:55.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:55.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:54 smithi090 ceph-mon[30350]: osd.6 [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] boot 2024-09-10T21:56:55.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:54 smithi090 ceph-mon[30350]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T21:56:55.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T21:56:55.228 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:56:55.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.899+0000 7fd537e00640 1 -- 172.21.15.154:0/1955306462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538102c10 msgr2=0x7fd538102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:55.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.899+0000 7fd537e00640 1 --2- 172.21.15.154:0/1955306462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538102c10 0x7fd538102ff0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd528009900 tx=0x7fd52802a410 comp rx=0 tx=0).stop 2024-09-10T21:56:55.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.899+0000 7fd537e00640 1 -- 172.21.15.154:0/1955306462 shutdown_connections 2024-09-10T21:56:55.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.899+0000 7fd537e00640 1 --2- 172.21.15.154:0/1955306462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd538108fa0 0x7fd53810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:55.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.899+0000 7fd537e00640 1 --2- 172.21.15.154:0/1955306462 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd538108620 0x7fd538108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:55.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.899+0000 7fd537e00640 1 --2- 172.21.15.154:0/1955306462 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538102c10 0x7fd538102ff0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:55.910 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.899+0000 7fd537e00640 1 -- 172.21.15.154:0/1955306462 >> 172.21.15.154:0/1955306462 conn(0x7fd5380fe660 msgr2=0x7fd538100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.899+0000 7fd537e00640 1 -- 172.21.15.154:0/1955306462 shutdown_connections 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.899+0000 7fd537e00640 1 -- 172.21.15.154:0/1955306462 wait complete. 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.901+0000 7fd537e00640 1 Processor -- start 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.901+0000 7fd537e00640 1 -- start start 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.901+0000 7fd537e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd538102c10 0x7fd53819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.901+0000 7fd537e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd538108620 0x7fd53819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.901+0000 7fd537e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538108fa0 0x7fd5381a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.901+0000 7fd537e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd538112b60 con 0x7fd538102c10 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.901+0000 7fd537e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd5381129e0 con 0x7fd538108fa0 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.901+0000 7fd537e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd538112ce0 con 0x7fd538108620 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd537400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538108fa0 0x7fd5381a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:56:55.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd536a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd538102c10 0x7fd53819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd537400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538108fa0 0x7fd5381a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.154:44754/0 (socket says 172.21.15.154:44754) 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd536a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd538102c10 0x7fd53819b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.154:33740/0 (socket says 172.21.15.154:33740) 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd537400640 1 -- 172.21.15.154:0/2985144978 learned_addr learned my addr 172.21.15.154:0/2985144978 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd536000640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd538108620 0x7fd53819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd537400640 1 -- 172.21.15.154:0/2985144978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd538108620 msgr2=0x7fd53819b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd537400640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd538108620 0x7fd53819b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd537400640 1 -- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd538102c10 msgr2=0x7fd53819b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd537400640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd538102c10 0x7fd53819b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd537400640 1 -- 172.21.15.154:0/2985144978 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5381a1240 con 0x7fd538108fa0 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd536000640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd538108620 0x7fd53819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd536a00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd538102c10 0x7fd53819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.902+0000 7fd537400640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538108fa0 0x7fd5381a0a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd53000eb10 tx=0x7fd53000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:56:55.912 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.903+0000 7fd53ca00640 1 -- 172.21.15.154:0/2985144978 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530004b20 con 0x7fd538108fa0 2024-09-10T21:56:55.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.903+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5381a1530 con 0x7fd538108fa0 2024-09-10T21:56:55.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.903+0000 7fd53ca00640 1 -- 172.21.15.154:0/2985144978 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd530004cc0 con 0x7fd538108fa0 2024-09-10T21:56:55.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.903+0000 7fd53ca00640 1 -- 172.21.15.154:0/2985144978 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530010690 con 0x7fd538108fa0 2024-09-10T21:56:55.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.903+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5381a7bf0 con 0x7fd538108fa0 2024-09-10T21:56:55.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.904+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd538102ff0 con 0x7fd538108fa0 2024-09-10T21:56:55.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.906+0000 7fd53ca00640 1 -- 172.21.15.154:0/2985144978 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fd530005000 con 0x7fd538108fa0 2024-09-10T21:56:55.913 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.909+0000 7fd53ca00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd51c0b53e0 0x7fd51c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:56:55.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.909+0000 7fd53ca00640 1 -- 172.21.15.154:0/2985144978 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5180+0+0 (secure 0 0 0) 0x7fd5300e58d0 con 0x7fd538108fa0 2024-09-10T21:56:55.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.909+0000 7fd536a00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd51c0b53e0 0x7fd51c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:56:55.914 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.910+0000 7fd536a00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd51c0b53e0 0x7fd51c0b78a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd528006010 tx=0x7fd528000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:56:55.915 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:55.911+0000 7fd53ca00640 1 -- 172.21.15.154:0/2985144978 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd530056220 con 0x7fd538108fa0 2024-09-10T21:56:56.107 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:56:56.105+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd538063590 con 0x7fd51c0b53e0 2024-09-10T21:56:56.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T21:56:56.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi090 2024-09-10T21:56:56.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:56.431 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi090 to 5696M 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:56:56.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:56.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi090 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi090 to 5696M 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:56:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:56:56.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi090 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi090 to 5696M 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:56.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:56:56.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:56:56.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:56:56.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:56:56.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:56:56.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:56:57.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:57 smithi090 ceph-mon[30350]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:56:57.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:57 smithi154 ceph-mon[34495]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:56:57.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:57 smithi171 ceph-mon[34138]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:56:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:58 smithi154 ceph-mon[34495]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:56:58.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:58 smithi090 ceph-mon[30350]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:56:58.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:58 smithi171 ceph-mon[34138]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:56:59.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:59 smithi154 ceph-mon[34495]: from='client.? 172.21.15.154:0/2957505595' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "627062fc-f4ab-4e64-a68a-33beb4664f3e"} : dispatch 2024-09-10T21:56:59.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:59 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "627062fc-f4ab-4e64-a68a-33beb4664f3e"} : dispatch 2024-09-10T21:56:59.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:59 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "627062fc-f4ab-4e64-a68a-33beb4664f3e"}]': finished 2024-09-10T21:56:59.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:59 smithi154 ceph-mon[34495]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T21:56:59.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:56:59 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:56:59.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:59 smithi171 ceph-mon[34138]: from='client.? 172.21.15.154:0/2957505595' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "627062fc-f4ab-4e64-a68a-33beb4664f3e"} : dispatch 2024-09-10T21:56:59.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:59 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "627062fc-f4ab-4e64-a68a-33beb4664f3e"} : dispatch 2024-09-10T21:56:59.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:59 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "627062fc-f4ab-4e64-a68a-33beb4664f3e"}]': finished 2024-09-10T21:56:59.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:59 smithi171 ceph-mon[34138]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T21:56:59.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:56:59 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:56:59.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:59 smithi090 ceph-mon[30350]: from='client.? 172.21.15.154:0/2957505595' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "627062fc-f4ab-4e64-a68a-33beb4664f3e"} : dispatch 2024-09-10T21:56:59.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:59 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "627062fc-f4ab-4e64-a68a-33beb4664f3e"} : dispatch 2024-09-10T21:56:59.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:59 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "627062fc-f4ab-4e64-a68a-33beb4664f3e"}]': finished 2024-09-10T21:56:59.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:59 smithi090 ceph-mon[30350]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T21:56:59.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:56:59 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:00.686 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:00 smithi154 ceph-mon[34495]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:00.686 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:00 smithi154 ceph-mon[34495]: from='client.? 172.21.15.154:0/834548636' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:57:00.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:00 smithi171 ceph-mon[34138]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:00.749 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:00 smithi171 ceph-mon[34138]: from='client.? 172.21.15.154:0/834548636' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:57:00.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:00 smithi090 ceph-mon[30350]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:00.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:00 smithi090 ceph-mon[30350]: from='client.? 172.21.15.154:0/834548636' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:57:02.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:02 smithi090 ceph-mon[30350]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:02.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:02 smithi154 ceph-mon[34495]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:02.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:02 smithi171 ceph-mon[34138]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:04.830 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:04 smithi154 ceph-mon[34495]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:04.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:04 smithi090 ceph-mon[30350]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:04 smithi171 ceph-mon[34138]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:06 smithi090 ceph-mon[30350]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:06.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:06 smithi154 ceph-mon[34495]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:06.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:06 smithi171 ceph-mon[34138]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:08.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:08 smithi154 ceph-mon[34495]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:08.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T21:57:08.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:08.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:08 smithi090 ceph-mon[30350]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:08.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T21:57:08.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:08.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:08 smithi171 ceph-mon[34138]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:08.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T21:57:08.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:09.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:09 smithi154 ceph-mon[34495]: Deploying daemon osd.7 on smithi154 2024-09-10T21:57:09.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:09 smithi090 ceph-mon[30350]: Deploying daemon osd.7 on smithi154 2024-09-10T21:57:09.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:09 smithi171 ceph-mon[34138]: Deploying daemon osd.7 on smithi154 2024-09-10T21:57:10.845 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:10 smithi154 ceph-mon[34495]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:10.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:10 smithi171 ceph-mon[34138]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:10 smithi090 ceph-mon[30350]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:12.790 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:12 smithi154 ceph-mon[34495]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:12 smithi171 ceph-mon[34138]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:12 smithi090 ceph-mon[30350]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:14.779 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:14 smithi154 ceph-mon[34495]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:14 smithi171 ceph-mon[34138]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:14 smithi090 ceph-mon[30350]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:16.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:16 smithi154 ceph-mon[34495]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:16.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:16.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:16.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:16 smithi171 ceph-mon[34138]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:16 smithi090 ceph-mon[30350]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:17.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:18.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:18 smithi154 ceph-mon[34495]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:18 smithi171 ceph-mon[34138]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:18 smithi090 ceph-mon[30350]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:20.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:20 smithi154 ceph-mon[34495]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:20.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:20.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:20.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:20.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:20.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:20.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:20 smithi171 ceph-mon[34138]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:20.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:20.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:20 smithi090 ceph-mon[30350]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:21.363 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.361+0000 7fd53ca00640 1 -- 172.21.15.154:0/2985144978 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd538063590 con 0x7fd51c0b53e0 2024-09-10T21:57:21.364 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 7 on host 'smithi154' 2024-09-10T21:57:21.365 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd51c0b53e0 msgr2=0x7fd51c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd51c0b53e0 0x7fd51c0b78a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd528006010 tx=0x7fd528000bc0 comp rx=0 tx=0).stop 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538108fa0 msgr2=0x7fd5381a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538108fa0 0x7fd5381a0a70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd53000eb10 tx=0x7fd53000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 shutdown_connections 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd51c0b53e0 0x7fd51c0b78a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd538108fa0 0x7fd5381a0a70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd538108620 0x7fd53819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 --2- 172.21.15.154:0/2985144978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd538102c10 0x7fd53819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 >> 172.21.15.154:0/2985144978 conn(0x7fd5380fe660 msgr2=0x7fd538104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 shutdown_connections 2024-09-10T21:57:21.366 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:57:21.364+0000 7fd537e00640 1 -- 172.21.15.154:0/2985144978 wait complete. 2024-09-10T21:57:21.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:21 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:21.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:21 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:21.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:21 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:21.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:21 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:21.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:21 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:21.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:21 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:21.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:21 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:21.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:21 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:21.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:21 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:22.218 DEBUG:teuthology.orchestra.run.smithi154:osd.7> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T21:57:22.223 INFO:tasks.cephadm:Deploying osd.8 on smithi171 with /dev/vg_nvme/lv_2... 2024-09-10T21:57:22.223 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T21:57:22.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:22 smithi154 ceph-mon[34495]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:22.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:22 smithi154 ceph-mon[34495]: from='osd.7 [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T21:57:22.421 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:22 smithi154 ceph-mon[34495]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T21:57:22.421 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 21:57:22 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T21:57:22.290+0000 7fd36a845640 -1 osd.7 0 log_to_monitors true 2024-09-10T21:57:22.598 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:57:22.737 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:22 smithi171 ceph-mon[34138]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:22.737 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:22 smithi171 ceph-mon[34138]: from='osd.7 [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T21:57:22.737 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:22 smithi171 ceph-mon[34138]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T21:57:22.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:22 smithi090 ceph-mon[30350]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:22.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:22 smithi090 ceph-mon[30350]: from='osd.7 [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T21:57:22.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:22 smithi090 ceph-mon[30350]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T21:57:23.659 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 21:57:23 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T21:57:23.419+0000 7fd365000640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T21:57:23.659 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 21:57:23 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T21:57:23.451+0000 7fd360000640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:57:23.660 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:23 smithi154 ceph-mon[34495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T21:57:23.660 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:23 smithi154 ceph-mon[34495]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T21:57:23.660 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:23.660 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:23 smithi154 ceph-mon[34495]: from='osd.7 [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:57:23.660 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:23 smithi154 ceph-mon[34495]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:57:23.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:23 smithi171 ceph-mon[34138]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T21:57:23.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:23 smithi171 ceph-mon[34138]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T21:57:23.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:23.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:23 smithi171 ceph-mon[34138]: from='osd.7 [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:57:23.664 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:23 smithi171 ceph-mon[34138]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:57:23.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:23 smithi090 ceph-mon[30350]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T21:57:23.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:23 smithi090 ceph-mon[30350]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T21:57:23.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:23.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:23 smithi090 ceph-mon[30350]: from='osd.7 [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:57:23.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:23 smithi090 ceph-mon[30350]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:57:24.584 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:24 smithi171 ceph-mon[34138]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:57:24.584 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:24 smithi171 ceph-mon[34138]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T21:57:24.584 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:24.584 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:24.584 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:24 smithi171 ceph-mon[34138]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:24.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:24 smithi154 ceph-mon[34495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:57:24.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:24 smithi154 ceph-mon[34495]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T21:57:24.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:24.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:24.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:24 smithi154 ceph-mon[34495]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:24.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:24 smithi090 ceph-mon[30350]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:57:24.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:24 smithi090 ceph-mon[30350]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T21:57:24.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:24.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:24.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:24 smithi090 ceph-mon[30350]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-10T21:57:25.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:25 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:57:25.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:25 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:57:25.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:25 smithi171 ceph-mon[34138]: osd.7 [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] boot 2024-09-10T21:57:25.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:25 smithi171 ceph-mon[34138]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T21:57:25.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:25 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:25.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:25 smithi171 ceph-mon[34138]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T21:57:25.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:25 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:57:25.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:25 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:57:25.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:25 smithi090 ceph-mon[30350]: osd.7 [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] boot 2024-09-10T21:57:25.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:25 smithi090 ceph-mon[30350]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T21:57:25.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:25 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:25.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:25 smithi090 ceph-mon[30350]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T21:57:25.912 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:57:25.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:25 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:57:25.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:25 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:57:25.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:25 smithi154 ceph-mon[34495]: osd.7 [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] boot 2024-09-10T21:57:25.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:25 smithi154 ceph-mon[34495]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T21:57:25.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:25 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T21:57:25.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:25 smithi154 ceph-mon[34495]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T21:57:25.945 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_2 2024-09-10T21:57:26.337 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:57:26.693 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:26.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:26.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:26.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:26.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:26.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:26.694 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.922+0000 7fc49b200640 1 -- 172.21.15.171:0/2429026224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c10b1a0 msgr2=0x7fc49c1119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.922+0000 7fc49b200640 1 --2- 172.21.15.171:0/2429026224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c10b1a0 0x7fc49c1119a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc494009ca0 tx=0x7fc49402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.922+0000 7fc49b200640 1 -- 172.21.15.171:0/2429026224 shutdown_connections 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.922+0000 7fc49b200640 1 --2- 172.21.15.171:0/2429026224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c10b1a0 0x7fc49c1119a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.922+0000 7fc49b200640 1 --2- 172.21.15.171:0/2429026224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c10a820 0x7fc49c10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.922+0000 7fc49b200640 1 --2- 172.21.15.171:0/2429026224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc49c104e10 0x7fc49c1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.922+0000 7fc49b200640 1 -- 172.21.15.171:0/2429026224 >> 172.21.15.171:0/2429026224 conn(0x7fc49c100840 msgr2=0x7fc49c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.922+0000 7fc49b200640 1 -- 172.21.15.171:0/2429026224 shutdown_connections 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.922+0000 7fc49b200640 1 -- 172.21.15.171:0/2429026224 wait complete. 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.923+0000 7fc49b200640 1 Processor -- start 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.923+0000 7fc49b200640 1 -- start start 2024-09-10T21:57:26.927 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc49b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c104e10 0x7fc49c19b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc49b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc49c10a820 0x7fc49c19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc49b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c10b1a0 0x7fc49c1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc49b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c114d20 con 0x7fc49c10a820 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc49b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c114ba0 con 0x7fc49c10b1a0 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc49b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc49c114ea0 con 0x7fc49c104e10 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc499e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c104e10 0x7fc49c19b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc499e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c104e10 0x7fc49c19b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:48278/0 (socket says 172.21.15.171:48278) 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc499e00640 1 -- 172.21.15.171:0/2827092856 learned_addr learned my addr 172.21.15.171:0/2827092856 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc49a800640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c10b1a0 0x7fc49c1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc499400640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc49c10a820 0x7fc49c19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc499e00640 1 -- 172.21.15.171:0/2827092856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c10b1a0 msgr2=0x7fc49c1a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:26.928 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc499e00640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c10b1a0 0x7fc49c1a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc499e00640 1 -- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc49c10a820 msgr2=0x7fc49c19b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc499e00640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc49c10a820 0x7fc49c19b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.924+0000 7fc499e00640 1 -- 172.21.15.171:0/2827092856 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49c1a1120 con 0x7fc49c104e10 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.925+0000 7fc499e00640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c104e10 0x7fc49c19b070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c920 tx=0x7fc48c00cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.925+0000 7fc492a00640 1 -- 172.21.15.171:0/2827092856 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c007c70 con 0x7fc49c104e10 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.925+0000 7fc492a00640 1 -- 172.21.15.171:0/2827092856 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc48c007e10 con 0x7fc49c104e10 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.925+0000 7fc492a00640 1 -- 172.21.15.171:0/2827092856 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c018710 con 0x7fc49c104e10 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.925+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49c1a1410 con 0x7fc49c104e10 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.925+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc49c1a7ad0 con 0x7fc49c104e10 2024-09-10T21:57:26.929 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.926+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49c1051f0 con 0x7fc49c104e10 2024-09-10T21:57:26.930 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.928+0000 7fc492a00640 1 -- 172.21.15.171:0/2827092856 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fc48c00f450 con 0x7fc49c104e10 2024-09-10T21:57:26.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.930+0000 7fc492a00640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc4800b5460 0x7fc4800b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:57:26.931 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.930+0000 7fc492a00640 1 -- 172.21.15.171:0/2827092856 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5612+0+0 (secure 0 0 0) 0x7fc48c0e56a0 con 0x7fc49c104e10 2024-09-10T21:57:26.932 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.930+0000 7fc499400640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc4800b5460 0x7fc4800b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:57:26.932 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.931+0000 7fc499400640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc4800b5460 0x7fc4800b7920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc49c107310 tx=0x7fc488005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:57:26.933 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:26.932+0000 7fc492a00640 1 -- 172.21.15.171:0/2827092856 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc48c014030 con 0x7fc49c104e10 2024-09-10T21:57:27.128 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:27.127+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc49c109e60 con 0x7fc4800b5460 2024-09-10T21:57:27.719 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:27 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi154 2024-09-10T21:57:27.719 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:27 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi154 to 5696M 2024-09-10T21:57:27.720 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:27 smithi171 ceph-mon[34138]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T21:57:27.720 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:27 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:57:27.720 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:27 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:57:27.720 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:27 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:27 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi154 2024-09-10T21:57:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:27 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi154 to 5696M 2024-09-10T21:57:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:27 smithi090 ceph-mon[30350]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T21:57:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:27 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:57:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:27 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:57:27.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:27 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:27 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi154 2024-09-10T21:57:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:27 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi154 to 5696M 2024-09-10T21:57:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:27 smithi154 ceph-mon[34495]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T21:57:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:27 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:57:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:27 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:57:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:27 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:28.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:28 smithi171 ceph-mon[34138]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:57:28.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:28 smithi171 ceph-mon[34138]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:28 smithi090 ceph-mon[30350]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:57:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:28 smithi090 ceph-mon[30350]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:28 smithi154 ceph-mon[34495]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:57:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:28 smithi154 ceph-mon[34495]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:30 smithi090 ceph-mon[30350]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:57:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:30 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/1129675718' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6ec3145c-deaf-467c-800c-257e339ebde5"} : dispatch 2024-09-10T21:57:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:30 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6ec3145c-deaf-467c-800c-257e339ebde5"} : dispatch 2024-09-10T21:57:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:30 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ec3145c-deaf-467c-800c-257e339ebde5"}]': finished 2024-09-10T21:57:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:30 smithi090 ceph-mon[30350]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T21:57:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:30.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:30 smithi154 ceph-mon[34495]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:57:30.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:30 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/1129675718' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6ec3145c-deaf-467c-800c-257e339ebde5"} : dispatch 2024-09-10T21:57:30.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:30 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6ec3145c-deaf-467c-800c-257e339ebde5"} : dispatch 2024-09-10T21:57:30.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:30 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ec3145c-deaf-467c-800c-257e339ebde5"}]': finished 2024-09-10T21:57:30.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:30 smithi154 ceph-mon[34495]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T21:57:30.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:30 smithi171 ceph-mon[34138]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:57:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:30 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/1129675718' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6ec3145c-deaf-467c-800c-257e339ebde5"} : dispatch 2024-09-10T21:57:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:30 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6ec3145c-deaf-467c-800c-257e339ebde5"} : dispatch 2024-09-10T21:57:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:30 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ec3145c-deaf-467c-800c-257e339ebde5"}]': finished 2024-09-10T21:57:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:30 smithi171 ceph-mon[34138]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T21:57:30.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:31.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:31 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/3396003069' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:57:31.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:31 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/3396003069' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:57:31.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:31 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/3396003069' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:57:32.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:32 smithi090 ceph-mon[30350]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-10T21:57:32.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:32 smithi154 ceph-mon[34495]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-10T21:57:32.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:32 smithi171 ceph-mon[34138]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-10T21:57:34.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:34 smithi090 ceph-mon[30350]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T21:57:34.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:34 smithi154 ceph-mon[34495]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T21:57:34.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:34 smithi171 ceph-mon[34138]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T21:57:36.878 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:36 smithi171 ceph-mon[34138]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T21:57:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:36 smithi154 ceph-mon[34495]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T21:57:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:36 smithi090 ceph-mon[30350]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T21:57:38.890 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:38 smithi171 ceph-mon[34138]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:57:38.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:38 smithi154 ceph-mon[34495]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:57:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:38 smithi090 ceph-mon[30350]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:57:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:40 smithi154 ceph-mon[34495]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T21:57:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:40 smithi154 ceph-mon[34495]: Deploying daemon osd.8 on smithi171 2024-09-10T21:57:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:40 smithi171 ceph-mon[34138]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T21:57:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:40 smithi171 ceph-mon[34138]: Deploying daemon osd.8 on smithi171 2024-09-10T21:57:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:40 smithi090 ceph-mon[30350]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T21:57:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:40 smithi090 ceph-mon[30350]: Deploying daemon osd.8 on smithi171 2024-09-10T21:57:42.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:42 smithi154 ceph-mon[34495]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:42 smithi171 ceph-mon[34138]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:42 smithi090 ceph-mon[30350]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:44 smithi171 ceph-mon[34138]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:44 smithi154 ceph-mon[34495]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:44 smithi090 ceph-mon[30350]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:46.902 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:46 smithi171 ceph-mon[34138]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:46 smithi154 ceph-mon[34495]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:46 smithi090 ceph-mon[30350]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:47.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:47.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:48 smithi154 ceph-mon[34495]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:48.938 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:48 smithi171 ceph-mon[34138]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:48 smithi090 ceph-mon[30350]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:50.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:50 smithi171 ceph-mon[34138]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:50.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:50.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:50.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:50 smithi154 ceph-mon[34495]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:50 smithi090 ceph-mon[30350]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:51.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:51.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:51.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:51.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:51.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:51.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:52.050 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.049+0000 7fc492a00640 1 -- 172.21.15.171:0/2827092856 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc49c109e60 con 0x7fc4800b5460 2024-09-10T21:57:52.050 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 8 on host 'smithi171' 2024-09-10T21:57:52.052 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc4800b5460 msgr2=0x7fc4800b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:52.052 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc4800b5460 0x7fc4800b7920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc49c107310 tx=0x7fc488005f90 comp rx=0 tx=0).stop 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c104e10 msgr2=0x7fc49c19b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c104e10 0x7fc49c19b070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c920 tx=0x7fc48c00cde0 comp rx=0 tx=0).stop 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 shutdown_connections 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc4800b5460 0x7fc4800b7920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc49c10b1a0 0x7fc49c1a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc49c10a820 0x7fc49c19b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 --2- 172.21.15.171:0/2827092856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc49c104e10 0x7fc49c19b070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 >> 172.21.15.171:0/2827092856 conn(0x7fc49c100840 msgr2=0x7fc49c103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 shutdown_connections 2024-09-10T21:57:52.053 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:57:52.052+0000 7fc49b200640 1 -- 172.21.15.171:0/2827092856 wait complete. 2024-09-10T21:57:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:52.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:52 smithi171 ceph-mon[34138]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:52.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:52.858 DEBUG:teuthology.orchestra.run.smithi171:osd.8> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T21:57:52.862 INFO:tasks.cephadm:Deploying osd.9 on smithi090 with /dev/vg_nvme/lv_1... 2024-09-10T21:57:52.862 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T21:57:52.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:52 smithi090 ceph-mon[30350]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:52.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:52.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:52.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:52 smithi154 ceph-mon[34495]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:57:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:53.224 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:57:53.640 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 21:57:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T21:57:53.610+0000 7fcddd734640 -1 osd.8 0 log_to_monitors true 2024-09-10T21:57:53.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:53 smithi171 ceph-mon[34138]: from='osd.8 [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T21:57:53.891 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:53 smithi171 ceph-mon[34138]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T21:57:53.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:53 smithi154 ceph-mon[34495]: from='osd.8 [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T21:57:53.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:53 smithi154 ceph-mon[34495]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T21:57:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:53 smithi090 ceph-mon[30350]: from='osd.8 [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T21:57:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:53 smithi090 ceph-mon[30350]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T21:57:54.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:54 smithi090 ceph-mon[30350]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:54.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:54 smithi090 ceph-mon[30350]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T21:57:54.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:54 smithi090 ceph-mon[30350]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T21:57:54.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:54.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:54 smithi090 ceph-mon[30350]: from='osd.8 [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:57:54.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:54 smithi090 ceph-mon[30350]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:57:54.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 21:57:54 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T21:57:54.661+0000 7fcdd8600640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T21:57:54.883 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 21:57:54 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T21:57:54.673+0000 7fcdd3600640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:57:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:54 smithi171 ceph-mon[34138]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:54 smithi171 ceph-mon[34138]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T21:57:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:54 smithi171 ceph-mon[34138]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T21:57:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:54 smithi171 ceph-mon[34138]: from='osd.8 [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:57:54.884 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:54 smithi171 ceph-mon[34138]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:57:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:54 smithi154 ceph-mon[34495]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:54 smithi154 ceph-mon[34495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T21:57:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:54 smithi154 ceph-mon[34495]: osdmap e52: 9 total, 8 up, 9 in 2024-09-10T21:57:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:54 smithi154 ceph-mon[34495]: from='osd.8 [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:57:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:54 smithi154 ceph-mon[34495]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:57:55.728 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:55 smithi090 ceph-mon[30350]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:57:55.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:55 smithi090 ceph-mon[30350]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T21:57:55.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:55.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:55.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:55 smithi090 ceph-mon[30350]: osd.8 [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] boot 2024-09-10T21:57:55.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:55 smithi090 ceph-mon[30350]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T21:57:55.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:55.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:55 smithi154 ceph-mon[34495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:57:55.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:55 smithi154 ceph-mon[34495]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T21:57:55.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:55.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:55.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:55 smithi154 ceph-mon[34495]: osd.8 [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] boot 2024-09-10T21:57:55.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:55 smithi154 ceph-mon[34495]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T21:57:55.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:55.980 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:55 smithi171 ceph-mon[34138]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:57:55.980 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:55 smithi171 ceph-mon[34138]: osdmap e53: 9 total, 8 up, 9 in 2024-09-10T21:57:55.980 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:55.980 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:55.980 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:55 smithi171 ceph-mon[34138]: osd.8 [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] boot 2024-09-10T21:57:55.980 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:55 smithi171 ceph-mon[34138]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T21:57:55.980 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T21:57:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:56 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:57:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:56 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:57:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:56 smithi171 ceph-mon[34138]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:56 smithi171 ceph-mon[34138]: osdmap e55: 9 total, 9 up, 9 in 2024-09-10T21:57:56.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:56 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:57:56.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:56 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:57:56.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:56 smithi090 ceph-mon[30350]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:56.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:56 smithi090 ceph-mon[30350]: osdmap e55: 9 total, 9 up, 9 in 2024-09-10T21:57:56.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:56 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:57:56.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:56 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:57:56.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:56 smithi154 ceph-mon[34495]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T21:57:56.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:56 smithi154 ceph-mon[34495]: osdmap e55: 9 total, 9 up, 9 in 2024-09-10T21:57:57.178 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:57:57.209 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-09-10T21:57:57.575 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:57:58.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi171 2024-09-10T21:57:58.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:58.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:58.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:58.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi171 to 5696M 2024-09-10T21:57:58.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:58.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:58.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T21:57:58.020 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:57 smithi090 ceph-mon[30350]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:57:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.130+0000 7f5481e00640 1 -- 172.21.15.90:0/1728239246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 msgr2=0x7f547c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.130+0000 7f5481e00640 1 --2- 172.21.15.90:0/1728239246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 0x7f547c103180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f546c009900 tx=0x7f546c02a410 comp rx=0 tx=0).stop 2024-09-10T21:57:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.131+0000 7f5481e00640 1 -- 172.21.15.90:0/1728239246 shutdown_connections 2024-09-10T21:57:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.131+0000 7f5481e00640 1 --2- 172.21.15.90:0/1728239246 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f547c109130 0x7f547c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.131+0000 7f5481e00640 1 --2- 172.21.15.90:0/1728239246 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f547c1087b0 0x7f547c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.131+0000 7f5481e00640 1 --2- 172.21.15.90:0/1728239246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 0x7f547c103180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.131+0000 7f5481e00640 1 -- 172.21.15.90:0/1728239246 >> 172.21.15.90:0/1728239246 conn(0x7f547c0fe810 msgr2=0x7f547c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:57:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.131+0000 7f5481e00640 1 -- 172.21.15.90:0/1728239246 shutdown_connections 2024-09-10T21:57:58.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.131+0000 7f5481e00640 1 -- 172.21.15.90:0/1728239246 wait complete. 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.132+0000 7f5481e00640 1 Processor -- start 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.132+0000 7f5481e00640 1 -- start start 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.132+0000 7f5481e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 0x7f547c109920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.132+0000 7f5481e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f547c1087b0 0x7f547c109e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.132+0000 7f5481e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f547c109130 0x7f547c10a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.132+0000 7f5481e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f547c112ca0 con 0x7f547c102da0 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.132+0000 7f5481e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f547c112b20 con 0x7f547c109130 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.132+0000 7f5481e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f547c112e20 con 0x7f547c1087b0 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 0x7f547c109920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 0x7f547c109920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45638/0 (socket says 172.21.15.90:45638) 2024-09-10T21:57:58.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547aa00640 1 -- 172.21.15.90:0/1033073009 learned_addr learned my addr 172.21.15.90:0/1033073009 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:57:58.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547b400640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f547c109130 0x7f547c10a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:57:58.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547a000640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f547c1087b0 0x7f547c109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:57:58.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547aa00640 1 -- 172.21.15.90:0/1033073009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f547c1087b0 msgr2=0x7f547c109e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:58.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547aa00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f547c1087b0 0x7f547c109e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:58.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547aa00640 1 -- 172.21.15.90:0/1033073009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f547c109130 msgr2=0x7f547c10a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:57:58.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547aa00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f547c109130 0x7f547c10a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:57:58.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547aa00640 1 -- 172.21.15.90:0/1033073009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c1ab970 con 0x7f547c102da0 2024-09-10T21:57:58.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.133+0000 7f547aa00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 0x7f547c109920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f546c009230 tx=0x7f546c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:57:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.134+0000 7f5480a00640 1 -- 172.21.15.90:0/1033073009 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c03f030 con 0x7f547c102da0 2024-09-10T21:57:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.134+0000 7f5480a00640 1 -- 172.21.15.90:0/1033073009 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f546c003a80 con 0x7f547c102da0 2024-09-10T21:57:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.134+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c1abc00 con 0x7f547c102da0 2024-09-10T21:57:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.134+0000 7f5480a00640 1 -- 172.21.15.90:0/1033073009 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c035070 con 0x7f547c102da0 2024-09-10T21:57:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.134+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c1ac130 con 0x7f547c102da0 2024-09-10T21:57:58.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.135+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547c103180 con 0x7f547c102da0 2024-09-10T21:57:58.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.137+0000 7f5480a00640 1 -- 172.21.15.90:0/1033073009 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f546c03bc30 con 0x7f547c102da0 2024-09-10T21:57:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.139+0000 7f5480a00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f54600b53b0 0x7f54600b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:57:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.140+0000 7f5480a00640 1 -- 172.21.15.90:0/1033073009 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6044+0+0 (secure 0 0 0) 0x7f546c1060e0 con 0x7f547c102da0 2024-09-10T21:57:58.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.140+0000 7f547a000640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f54600b53b0 0x7f54600b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:57:58.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.140+0000 7f547a000640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f54600b53b0 0x7f54600b7870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f547c10d430 tx=0x7f546800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:57:58.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.142+0000 7f5480a00640 1 -- 172.21.15.90:0/1033073009 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f546c0cd190 con 0x7f547c102da0 2024-09-10T21:57:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi171 2024-09-10T21:57:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi171 to 5696M 2024-09-10T21:57:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:58.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:58.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T21:57:58.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:57 smithi154 ceph-mon[34495]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:57:58.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi171 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi171 to 5696M 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: osdmap e56: 9 total, 9 up, 9 in 2024-09-10T21:57:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:57 smithi171 ceph-mon[34138]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:57:58.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:57:58.333+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f547c100650 con 0x7f54600b53b0 2024-09-10T21:57:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:58 smithi154 ceph-mon[34495]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:57:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:57:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:57:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:58 smithi171 ceph-mon[34138]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:57:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:57:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:57:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:57:59.248 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:58 smithi090 ceph-mon[30350]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:57:59.248 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:57:59.248 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:57:59.248 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:57:59 smithi090 ceph-mon[30350]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-10T21:58:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:57:59 smithi154 ceph-mon[34495]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-10T21:58:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:57:59 smithi171 ceph-mon[34138]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-10T21:58:01.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:01 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/952856613' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7e58c2f4-87df-4021-98ff-fd65c0a4fbcf"} : dispatch 2024-09-10T21:58:01.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:01 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/952856613' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e58c2f4-87df-4021-98ff-fd65c0a4fbcf"}]': finished 2024-09-10T21:58:01.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:01 smithi154 ceph-mon[34495]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T21:58:01.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:01 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:01.987 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:01 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/952856613' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7e58c2f4-87df-4021-98ff-fd65c0a4fbcf"} : dispatch 2024-09-10T21:58:01.987 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:01 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/952856613' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e58c2f4-87df-4021-98ff-fd65c0a4fbcf"}]': finished 2024-09-10T21:58:01.987 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:01 smithi090 ceph-mon[30350]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T21:58:01.988 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:01 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:01.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:01 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/952856613' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7e58c2f4-87df-4021-98ff-fd65c0a4fbcf"} : dispatch 2024-09-10T21:58:01.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:01 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/952856613' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e58c2f4-87df-4021-98ff-fd65c0a4fbcf"}]': finished 2024-09-10T21:58:01.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:01 smithi171 ceph-mon[34138]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T21:58:01.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:01 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:02.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:02 smithi154 ceph-mon[34495]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-10T21:58:02.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:02 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3476971509' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:58:02.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:02 smithi171 ceph-mon[34138]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-10T21:58:02.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:02 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3476971509' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:58:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:02 smithi090 ceph-mon[30350]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-10T21:58:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:02 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3476971509' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:58:04.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:04 smithi154 ceph-mon[34495]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-10T21:58:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:04 smithi171 ceph-mon[34138]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-10T21:58:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:04 smithi090 ceph-mon[30350]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-10T21:58:06.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:06 smithi154 ceph-mon[34495]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-10T21:58:06.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:06 smithi171 ceph-mon[34138]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-10T21:58:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:06 smithi090 ceph-mon[30350]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-10T21:58:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:08 smithi154 ceph-mon[34495]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:58:08.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:08 smithi171 ceph-mon[34138]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:58:09.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:08 smithi090 ceph-mon[30350]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:58:10.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:10 smithi154 ceph-mon[34495]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:10.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:10 smithi171 ceph-mon[34138]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:10 smithi090 ceph-mon[30350]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:11.926 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:11 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T21:58:11.926 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:11 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:11.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:11 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T21:58:11.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:11 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:12.002 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:11 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T21:58:12.002 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:11 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:12.837 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:12 smithi090 ceph-mon[30350]: Deploying daemon osd.9 on smithi090 2024-09-10T21:58:12.837 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:12 smithi090 ceph-mon[30350]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:12 smithi171 ceph-mon[34138]: Deploying daemon osd.9 on smithi090 2024-09-10T21:58:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:12 smithi171 ceph-mon[34138]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:12 smithi154 ceph-mon[34495]: Deploying daemon osd.9 on smithi090 2024-09-10T21:58:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:12 smithi154 ceph-mon[34495]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:14.770 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:14 smithi090 ceph-mon[30350]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:14.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:14 smithi154 ceph-mon[34495]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:14 smithi171 ceph-mon[34138]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:16.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:16 smithi154 ceph-mon[34495]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:16.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:16 smithi171 ceph-mon[34138]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:16.810 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:16 smithi090 ceph-mon[30350]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:18.728 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:18 smithi090 ceph-mon[30350]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:18.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:18 smithi154 ceph-mon[34495]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:18 smithi171 ceph-mon[34138]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:19.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:19.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:19.817 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:19.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:19.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:19.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:20.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:20 smithi090 ceph-mon[30350]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:20 smithi154 ceph-mon[34495]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:20 smithi171 ceph-mon[34138]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:22.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:22 smithi154 ceph-mon[34495]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:22.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:22.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:22.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:22.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:58:22.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:22.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:22 smithi090 ceph-mon[30350]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:22.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:22.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:22.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:22.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:58:22.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:22 smithi171 ceph-mon[34138]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:58:22.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:23.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.400+0000 7f5480a00640 1 -- 172.21.15.90:0/1033073009 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f547c100650 con 0x7f54600b53b0 2024-09-10T21:58:23.402 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 9 on host 'smithi090' 2024-09-10T21:58:23.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f54600b53b0 msgr2=0x7f54600b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:58:23.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f54600b53b0 0x7f54600b7870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f547c10d430 tx=0x7f546800b390 comp rx=0 tx=0).stop 2024-09-10T21:58:23.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 msgr2=0x7f547c109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:58:23.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 0x7f547c109920 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f546c009230 tx=0x7f546c004270 comp rx=0 tx=0).stop 2024-09-10T21:58:23.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 shutdown_connections 2024-09-10T21:58:23.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f54600b53b0 0x7f54600b7870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:23.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f547c109130 0x7f547c10a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:23.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f547c1087b0 0x7f547c109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:23.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 --2- 172.21.15.90:0/1033073009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f547c102da0 0x7f547c109920 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:23.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 >> 172.21.15.90:0/1033073009 conn(0x7f547c0fe810 msgr2=0x7f547c0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:58:23.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 shutdown_connections 2024-09-10T21:58:23.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:58:23.403+0000 7f5481e00640 1 -- 172.21.15.90:0/1033073009 wait complete. 2024-09-10T21:58:23.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:23.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:23.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:23 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:23.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:23.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:23.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:23 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:24.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:24.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:24.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:23 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:24.278 DEBUG:teuthology.orchestra.run.smithi090:osd.9> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T21:58:24.281 INFO:tasks.cephadm:Deploying osd.10 on smithi154 with /dev/vg_nvme/lv_1... 2024-09-10T21:58:24.282 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T21:58:24.659 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:58:24.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:24 smithi090 ceph-mon[30350]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:24.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:24 smithi090 ceph-mon[30350]: from='osd.9 [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T21:58:24.708 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:24 smithi090 ceph-mon[30350]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T21:58:24.708 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 21:58:24 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T21:58:24.396+0000 7f0b2822f640 -1 osd.9 0 log_to_monitors true 2024-09-10T21:58:24.872 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:24 smithi154 ceph-mon[34495]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:24.872 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:24 smithi154 ceph-mon[34495]: from='osd.9 [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T21:58:24.872 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:24 smithi154 ceph-mon[34495]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T21:58:24.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:24 smithi171 ceph-mon[34138]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:24 smithi171 ceph-mon[34138]: from='osd.9 [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T21:58:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:24 smithi171 ceph-mon[34138]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T21:58:26.573 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:26 smithi154 ceph-mon[34495]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T21:58:26.574 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:26 smithi154 ceph-mon[34495]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T21:58:26.574 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:26.574 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:26 smithi154 ceph-mon[34495]: from='osd.9 [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:58:26.574 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:26 smithi154 ceph-mon[34495]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:58:26.574 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:26 smithi154 ceph-mon[34495]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:26.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:26 smithi171 ceph-mon[34138]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T21:58:26.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:26 smithi171 ceph-mon[34138]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T21:58:26.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:26.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:26 smithi171 ceph-mon[34138]: from='osd.9 [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:58:26.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:26 smithi171 ceph-mon[34138]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:58:26.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:26 smithi171 ceph-mon[34138]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:26 smithi090 ceph-mon[30350]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T21:58:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:26 smithi090 ceph-mon[30350]: osdmap e58: 10 total, 9 up, 10 in 2024-09-10T21:58:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:26 smithi090 ceph-mon[30350]: from='osd.9 [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:58:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:26 smithi090 ceph-mon[30350]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]} : dispatch 2024-09-10T21:58:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:26 smithi090 ceph-mon[30350]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:26.857 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 21:58:26 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T21:58:26.414+0000 7f0b22a00640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T21:58:26.857 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 21:58:26 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T21:58:26.446+0000 7f0b1da00640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:58:27.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:27 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:58:27.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:27 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:58:27.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:27 smithi154 ceph-mon[34495]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:58:27.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:27 smithi154 ceph-mon[34495]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T21:58:27.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:27 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:27.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:27 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:27 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:58:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:27 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:58:27.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:27 smithi171 ceph-mon[34138]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:58:27.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:27 smithi171 ceph-mon[34138]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T21:58:27.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:27 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:27.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:27 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:27 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:58:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:27 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:58:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:27 smithi090 ceph-mon[30350]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-09-10T21:58:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:27 smithi090 ceph-mon[30350]: osdmap e59: 10 total, 9 up, 10 in 2024-09-10T21:58:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:27 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:27.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:27 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:28.405 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-09-10T21:58:28.444 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi154:vg_nvme/lv_1 2024-09-10T21:58:28.667 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:28 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:28.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:28 smithi154 ceph-mon[34495]: osd.9 [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] boot 2024-09-10T21:58:28.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:28 smithi154 ceph-mon[34495]: osdmap e60: 10 total, 10 up, 10 in 2024-09-10T21:58:28.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:28 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:28.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:28 smithi154 ceph-mon[34495]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:28.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:28 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:28.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:28 smithi090 ceph-mon[30350]: osd.9 [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] boot 2024-09-10T21:58:28.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:28 smithi090 ceph-mon[30350]: osdmap e60: 10 total, 10 up, 10 in 2024-09-10T21:58:28.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:28 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:28.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:28 smithi090 ceph-mon[30350]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:28.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:28 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:28.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:28 smithi171 ceph-mon[34138]: osd.9 [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] boot 2024-09-10T21:58:28.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:28 smithi171 ceph-mon[34138]: osdmap e60: 10 total, 10 up, 10 in 2024-09-10T21:58:28.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:28 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:58:28.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:28 smithi171 ceph-mon[34138]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:58:28.812 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:58:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.456+0000 7f6949a00640 1 -- 172.21.15.154:0/362507271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69441137e0 msgr2=0x7f6944115bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:58:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.456+0000 7f6949a00640 1 --2- 172.21.15.154:0/362507271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69441137e0 0x7f6944115bd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f693c009ca0 tx=0x7f693c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:58:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.456+0000 7f6949a00640 1 -- 172.21.15.154:0/362507271 shutdown_connections 2024-09-10T21:58:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.456+0000 7f6949a00640 1 --2- 172.21.15.154:0/362507271 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69441137e0 0x7f6944115bd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.456+0000 7f6949a00640 1 --2- 172.21.15.154:0/362507271 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6944077ff0 0x7f69441132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:29.470 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.456+0000 7f6949a00640 1 --2- 172.21.15.154:0/362507271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69440776d0 0x7f6944077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.456+0000 7f6949a00640 1 -- 172.21.15.154:0/362507271 >> 172.21.15.154:0/362507271 conn(0x7f6944100840 msgr2=0x7f6944102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.456+0000 7f6949a00640 1 -- 172.21.15.154:0/362507271 shutdown_connections 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.456+0000 7f6949a00640 1 -- 172.21.15.154:0/362507271 wait complete. 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.457+0000 7f6949a00640 1 Processor -- start 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.457+0000 7f6949a00640 1 -- start start 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.457+0000 7f6949a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69440776d0 0x7f694419f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.457+0000 7f6949a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944077ff0 0x7f694419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.457+0000 7f6949a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69441137e0 0x7f69441a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.457+0000 7f6949a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6944119110 con 0x7f6944077ff0 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.457+0000 7f6949a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6944118f90 con 0x7f69441137e0 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.457+0000 7f6949a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6944119290 con 0x7f69440776d0 2024-09-10T21:58:29.471 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944077ff0 0x7f694419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944077ff0 0x7f694419fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.154:53132/0 (socket says 172.21.15.154:53132) 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943400640 1 -- 172.21.15.154:0/1553753203 learned_addr learned my addr 172.21.15.154:0/1553753203 (peer_addr_for_me v2:172.21.15.154:0/0) 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6949000640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69441137e0 0x7f69441a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943e00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69440776d0 0x7f694419f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943400640 1 -- 172.21.15.154:0/1553753203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69440776d0 msgr2=0x7f694419f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943400640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69440776d0 0x7f694419f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943400640 1 -- 172.21.15.154:0/1553753203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69441137e0 msgr2=0x7f69441a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943400640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69441137e0 0x7f69441a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943400640 1 -- 172.21.15.154:0/1553753203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69441a55f0 con 0x7f6944077ff0 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.458+0000 7f6943e00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69440776d0 0x7f694419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.459+0000 7f6943400640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944077ff0 0x7f694419fa80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f693000b4a0 tx=0x7f693000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.459+0000 7f6940c00640 1 -- 172.21.15.154:0/1553753203 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930014070 con 0x7f6944077ff0 2024-09-10T21:58:29.472 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.459+0000 7f6940c00640 1 -- 172.21.15.154:0/1553753203 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6930010aa0 con 0x7f6944077ff0 2024-09-10T21:58:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.459+0000 7f6940c00640 1 -- 172.21.15.154:0/1553753203 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930017b70 con 0x7f6944077ff0 2024-09-10T21:58:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.459+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69441a58e0 con 0x7f6944077ff0 2024-09-10T21:58:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.459+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69441abfa0 con 0x7f6944077ff0 2024-09-10T21:58:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.460+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6944077ab0 con 0x7f6944077ff0 2024-09-10T21:58:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.463+0000 7f6940c00640 1 -- 172.21.15.154:0/1553753203 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f6930010610 con 0x7f6944077ff0 2024-09-10T21:58:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.464+0000 7f6940c00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69280b5460 0x7f69280b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:58:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.464+0000 7f6940c00640 1 -- 172.21.15.154:0/1553753203 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6476+0+0 (secure 0 0 0) 0x7f69300e4df0 con 0x7f6944077ff0 2024-09-10T21:58:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.465+0000 7f6943e00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69280b5460 0x7f69280b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:58:29.473 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.465+0000 7f6943e00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69280b5460 0x7f69280b7920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6934009c20 tx=0x7f6934005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:58:29.474 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.465+0000 7f6940c00640 1 -- 172.21.15.154:0/1553753203 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f69300ac340 con 0x7f6944077ff0 2024-09-10T21:58:29.660 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:29.659+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f69441158d0 con 0x7f69280b5460 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: osdmap e61: 10 total, 10 up, 10 in 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi090 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi090 to 4272M 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:58:29.661 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: osdmap e61: 10 total, 10 up, 10 in 2024-09-10T21:58:29.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi090 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi090 to 4272M 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:58:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: osdmap e61: 10 total, 10 up, 10 in 2024-09-10T21:58:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi090 2024-09-10T21:58:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:58:29.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi090 to 4272M 2024-09-10T21:58:29.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:29.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:29.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:58:29.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:30.600 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:30 smithi154 ceph-mon[34495]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:30.600 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:30 smithi154 ceph-mon[34495]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:58:30.600 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:58:30.600 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:58:30.600 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:30.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:30 smithi171 ceph-mon[34138]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:30.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:30 smithi171 ceph-mon[34138]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:58:30.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:58:30.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:58:30.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:30 smithi090 ceph-mon[30350]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:30 smithi090 ceph-mon[30350]: from='client.14667 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi154:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:58:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:58:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:58:30.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:32.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:32 smithi090 ceph-mon[30350]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:32.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:32 smithi154 ceph-mon[34495]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:32.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:32 smithi171 ceph-mon[34138]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:33.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:33 smithi154 ceph-mon[34495]: from='client.? 172.21.15.154:0/4294773805' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e57aa10e-348d-4de3-be31-927241f583e0"} : dispatch 2024-09-10T21:58:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:33 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e57aa10e-348d-4de3-be31-927241f583e0"} : dispatch 2024-09-10T21:58:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:33 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e57aa10e-348d-4de3-be31-927241f583e0"}]': finished 2024-09-10T21:58:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:33 smithi154 ceph-mon[34495]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T21:58:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:33.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:33 smithi154 ceph-mon[34495]: from='client.? 172.21.15.154:0/3467833175' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:58:33.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:33 smithi171 ceph-mon[34138]: from='client.? 172.21.15.154:0/4294773805' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e57aa10e-348d-4de3-be31-927241f583e0"} : dispatch 2024-09-10T21:58:33.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:33 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e57aa10e-348d-4de3-be31-927241f583e0"} : dispatch 2024-09-10T21:58:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:33 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e57aa10e-348d-4de3-be31-927241f583e0"}]': finished 2024-09-10T21:58:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:33 smithi171 ceph-mon[34138]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T21:58:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:33 smithi171 ceph-mon[34138]: from='client.? 172.21.15.154:0/3467833175' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:58:34.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:33 smithi090 ceph-mon[30350]: from='client.? 172.21.15.154:0/4294773805' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e57aa10e-348d-4de3-be31-927241f583e0"} : dispatch 2024-09-10T21:58:34.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:33 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e57aa10e-348d-4de3-be31-927241f583e0"} : dispatch 2024-09-10T21:58:34.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:33 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e57aa10e-348d-4de3-be31-927241f583e0"}]': finished 2024-09-10T21:58:34.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:33 smithi090 ceph-mon[30350]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T21:58:34.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:34.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:33 smithi090 ceph-mon[30350]: from='client.? 172.21.15.154:0/3467833175' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:58:34.899 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:34 smithi171 ceph-mon[34138]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:34.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:34 smithi154 ceph-mon[34495]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:34 smithi090 ceph-mon[30350]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:36.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:36 smithi090 ceph-mon[30350]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:36.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:36 smithi154 ceph-mon[34495]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:36 smithi171 ceph-mon[34138]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:38.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:38 smithi154 ceph-mon[34495]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:38.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:38 smithi171 ceph-mon[34138]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:38 smithi090 ceph-mon[30350]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:40.910 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:40 smithi154 ceph-mon[34495]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:40 smithi171 ceph-mon[34138]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:40 smithi090 ceph-mon[30350]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:42.699 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:42 smithi154 ceph-mon[34495]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:42.699 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:42 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T21:58:42.699 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:42 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:42 smithi171 ceph-mon[34138]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:42.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:42 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T21:58:42.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:42 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:42 smithi090 ceph-mon[30350]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:42 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T21:58:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:42 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:43.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:43 smithi154 ceph-mon[34495]: Deploying daemon osd.10 on smithi154 2024-09-10T21:58:43.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:43 smithi171 ceph-mon[34138]: Deploying daemon osd.10 on smithi154 2024-09-10T21:58:44.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:43 smithi090 ceph-mon[30350]: Deploying daemon osd.10 on smithi154 2024-09-10T21:58:44.820 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:44 smithi154 ceph-mon[34495]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:44.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:44 smithi171 ceph-mon[34138]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:44 smithi090 ceph-mon[30350]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:46.794 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:46 smithi154 ceph-mon[34495]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:46.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:46 smithi171 ceph-mon[34138]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:46 smithi090 ceph-mon[30350]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:48 smithi154 ceph-mon[34495]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:48 smithi171 ceph-mon[34138]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:48 smithi090 ceph-mon[30350]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:50 smithi154 ceph-mon[34495]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:50 smithi171 ceph-mon[34138]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:50.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:50 smithi090 ceph-mon[30350]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:52.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:52 smithi154 ceph-mon[34495]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:52 smithi171 ceph-mon[34138]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:52 smithi090 ceph-mon[30350]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:54.801 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.800+0000 7f6940c00640 1 -- 172.21.15.154:0/1553753203 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f69441158d0 con 0x7f69280b5460 2024-09-10T21:58:54.802 INFO:teuthology.orchestra.run.smithi154.stdout:Created osd(s) 10 on host 'smithi154' 2024-09-10T21:58:54.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.802+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69280b5460 msgr2=0x7f69280b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:58:54.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.802+0000 7f6949a00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69280b5460 0x7f69280b7920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6934009c20 tx=0x7f6934005f90 comp rx=0 tx=0).stop 2024-09-10T21:58:54.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.803+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944077ff0 msgr2=0x7f694419fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:58:54.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.803+0000 7f6949a00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944077ff0 0x7f694419fa80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f693000b4a0 tx=0x7f693000b960 comp rx=0 tx=0).stop 2024-09-10T21:58:54.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.804+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 shutdown_connections 2024-09-10T21:58:54.805 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.804+0000 7f6949a00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69280b5460 0x7f69280b7920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:54.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.804+0000 7f6949a00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69441137e0 0x7f69441a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:54.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.804+0000 7f6949a00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944077ff0 0x7f694419fa80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:54.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.804+0000 7f6949a00640 1 --2- 172.21.15.154:0/1553753203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69440776d0 0x7f694419f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:58:54.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.804+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 >> 172.21.15.154:0/1553753203 conn(0x7f6944100840 msgr2=0x7f6944102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:58:54.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.805+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 shutdown_connections 2024-09-10T21:58:54.806 INFO:teuthology.orchestra.run.smithi154.stderr:2024-09-10T21:58:54.805+0000 7f6949a00640 1 -- 172.21.15.154:0/1553753203 wait complete. 2024-09-10T21:58:54.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:54 smithi090 ceph-mon[30350]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:54.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:54.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:54.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:54 smithi154 ceph-mon[34495]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:54.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:54 smithi171 ceph-mon[34138]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.662 DEBUG:teuthology.orchestra.run.smithi154:osd.10> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T21:58:55.666 INFO:tasks.cephadm:Deploying osd.11 on smithi171 with /dev/vg_nvme/lv_1... 2024-09-10T21:58:55.666 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T21:58:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:58:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:55.875 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:55.875 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:58:55.875 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.876 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.876 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.876 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:55.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:58:55.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:58:55.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:58:55.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:58:56.031 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:58:56.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 21:58:55 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T21:58:55.914+0000 7f1caf841640 -1 osd.10 0 log_to_monitors true 2024-09-10T21:58:56.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:56 smithi171 ceph-mon[34138]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:56.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:56 smithi171 ceph-mon[34138]: from='osd.10 [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:58:56.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:56 smithi171 ceph-mon[34138]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:58:56.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:56 smithi090 ceph-mon[30350]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:56.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:56 smithi090 ceph-mon[30350]: from='osd.10 [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:58:56.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:56 smithi090 ceph-mon[30350]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:58:56.907 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:56 smithi154 ceph-mon[34495]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:56.907 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:56 smithi154 ceph-mon[34495]: from='osd.10 [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:58:56.907 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:56 smithi154 ceph-mon[34495]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:58:57.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 21:58:57 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T21:58:57.820+0000 7f1caa400640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T21:58:57.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 21:58:57 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T21:58:57.833+0000 7f1ca5400640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:58:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:57 smithi154 ceph-mon[34495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T21:58:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:57 smithi154 ceph-mon[34495]: osdmap e63: 11 total, 10 up, 11 in 2024-09-10T21:58:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:57.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:57 smithi154 ceph-mon[34495]: from='osd.10 [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:58:57.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:57 smithi154 ceph-mon[34495]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:58:57.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:57 smithi154 ceph-mon[34495]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:57 smithi090 ceph-mon[30350]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T21:58:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:57 smithi090 ceph-mon[30350]: osdmap e63: 11 total, 10 up, 11 in 2024-09-10T21:58:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:57 smithi090 ceph-mon[30350]: from='osd.10 [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:58:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:57 smithi090 ceph-mon[30350]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:58:58.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:57 smithi090 ceph-mon[30350]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:58.121 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:57 smithi171 ceph-mon[34138]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T21:58:58.121 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:57 smithi171 ceph-mon[34138]: osdmap e63: 11 total, 10 up, 11 in 2024-09-10T21:58:58.121 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:58.121 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:57 smithi171 ceph-mon[34138]: from='osd.10 [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:58:58.121 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:57 smithi171 ceph-mon[34138]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]} : dispatch 2024-09-10T21:58:58.121 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:57 smithi171 ceph-mon[34138]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:58:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:58 smithi090 ceph-mon[30350]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:58:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:58 smithi090 ceph-mon[30350]: osdmap e64: 11 total, 10 up, 11 in 2024-09-10T21:58:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:59.116 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:58 smithi171 ceph-mon[34138]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:58:59.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:58 smithi171 ceph-mon[34138]: osdmap e64: 11 total, 10 up, 11 in 2024-09-10T21:58:59.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:59.117 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:58 smithi154 ceph-mon[34495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi154", "root=default"]}]': finished 2024-09-10T21:58:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:58 smithi154 ceph-mon[34495]: osdmap e64: 11 total, 10 up, 11 in 2024-09-10T21:58:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:58:59.498 INFO:teuthology.orchestra.run.smithi171.stdout: 2024-09-10T21:58:59.530 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi171:vg_nvme/lv_1 2024-09-10T21:58:59.904 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:59:00.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:59 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:59:00.102 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:59 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:59:00.103 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:59 smithi171 ceph-mon[34138]: osd.10 [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] boot 2024-09-10T21:59:00.103 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:59 smithi171 ceph-mon[34138]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T21:59:00.103 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:59 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:59:00.103 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:58:59 smithi171 ceph-mon[34138]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:59:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:59 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:59:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:59 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:59:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:59 smithi090 ceph-mon[30350]: osd.10 [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] boot 2024-09-10T21:59:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:59 smithi090 ceph-mon[30350]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T21:59:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:59 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:59:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:58:59 smithi090 ceph-mon[30350]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:59:00.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:59 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:59:00.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:59 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:59:00.167 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:59 smithi154 ceph-mon[34495]: osd.10 [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] boot 2024-09-10T21:59:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:59 smithi154 ceph-mon[34495]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T21:59:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:59 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:59:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:58:59 smithi154 ceph-mon[34495]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:59:00.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.515+0000 7fc8a0c00640 1 -- 172.21.15.171:0/883771938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c1087b0 msgr2=0x7fc89c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:00.517 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.515+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/883771938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c1087b0 0x7fc89c108bf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc888009900 tx=0x7fc88802a410 comp rx=0 tx=0).stop 2024-09-10T21:59:00.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.515+0000 7fc8a0c00640 1 -- 172.21.15.171:0/883771938 shutdown_connections 2024-09-10T21:59:00.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.515+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/883771938 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc89c109130 0x7fc89c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:00.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.515+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/883771938 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c1087b0 0x7fc89c108bf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:00.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.515+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/883771938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc89c102da0 0x7fc89c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:00.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.515+0000 7fc8a0c00640 1 -- 172.21.15.171:0/883771938 >> 172.21.15.171:0/883771938 conn(0x7fc89c0fe810 msgr2=0x7fc89c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:00.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.516+0000 7fc8a0c00640 1 -- 172.21.15.171:0/883771938 shutdown_connections 2024-09-10T21:59:00.518 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.516+0000 7fc8a0c00640 1 -- 172.21.15.171:0/883771938 wait complete. 2024-09-10T21:59:00.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.516+0000 7fc8a0c00640 1 Processor -- start 2024-09-10T21:59:00.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc8a0c00640 1 -- start start 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc8a0c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc89c102da0 0x7fc89c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc8a0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc89c1087b0 0x7fc89c19b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc8a0c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c109130 0x7fc89c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc8a0c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c112c60 con 0x7fc89c102da0 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc8a0c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c112ae0 con 0x7fc89c1087b0 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc8a0c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c112de0 con 0x7fc89c109130 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc89a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c109130 0x7fc89c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc89a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c109130 0x7fc89c1a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:48614/0 (socket says 172.21.15.171:48614) 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc899400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc89c1087b0 0x7fc89c19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc89a800640 1 -- 172.21.15.171:0/2114594530 learned_addr learned my addr 172.21.15.171:0/2114594530 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-10T21:59:00.520 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.517+0000 7fc899e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc89c102da0 0x7fc89c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:00.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc89a800640 1 -- 172.21.15.171:0/2114594530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc89c1087b0 msgr2=0x7fc89c19b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:00.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc89a800640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc89c1087b0 0x7fc89c19b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:00.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc89a800640 1 -- 172.21.15.171:0/2114594530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc89c102da0 msgr2=0x7fc89c19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:00.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc89a800640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc89c102da0 0x7fc89c19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:00.521 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc89a800640 1 -- 172.21.15.171:0/2114594530 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c1a1480 con 0x7fc89c109130 2024-09-10T21:59:00.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc89a800640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c109130 0x7fc89c1a0cb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc89400eaa0 tx=0x7fc89400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:00.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc892a00640 1 -- 172.21.15.171:0/2114594530 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89400cdb0 con 0x7fc89c109130 2024-09-10T21:59:00.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc892a00640 1 -- 172.21.15.171:0/2114594530 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc894005020 con 0x7fc89c109130 2024-09-10T21:59:00.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c1a1770 con 0x7fc89c109130 2024-09-10T21:59:00.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc892a00640 1 -- 172.21.15.171:0/2114594530 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8940116d0 con 0x7fc89c109130 2024-09-10T21:59:00.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.518+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c1a7e30 con 0x7fc89c109130 2024-09-10T21:59:00.522 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.520+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89c103180 con 0x7fc89c109130 2024-09-10T21:59:00.523 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.522+0000 7fc892a00640 1 -- 172.21.15.171:0/2114594530 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fc89400f040 con 0x7fc89c109130 2024-09-10T21:59:00.525 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.523+0000 7fc892a00640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc8800b53b0 0x7fc8800b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:00.525 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.524+0000 7fc892a00640 1 -- 172.21.15.171:0/2114594530 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6909+0+0 (secure 0 0 0) 0x7fc8940e5890 con 0x7fc89c109130 2024-09-10T21:59:00.525 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.524+0000 7fc899e00640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc8800b53b0 0x7fc8800b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:00.526 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.525+0000 7fc899e00640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc8800b53b0 0x7fc8800b7870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00bbe0 tx=0x7fc88c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:00.527 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.525+0000 7fc892a00640 1 -- 172.21.15.171:0/2114594530 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc894016030 con 0x7fc89c109130 2024-09-10T21:59:00.723 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:00.721+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc89c063590 con 0x7fc8800b53b0 2024-09-10T21:59:00.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: osdmap e66: 11 total, 11 up, 11 in 2024-09-10T21:59:00.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi154 2024-09-10T21:59:00.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi154 to 4272M 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:59:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: osdmap e66: 11 total, 11 up, 11 in 2024-09-10T21:59:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi154 2024-09-10T21:59:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi154 to 4272M 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:59:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: osdmap e66: 11 total, 11 up, 11 in 2024-09-10T21:59:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi154 2024-09-10T21:59:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi154 to 4272M 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi171:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:59:01.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:01.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:01 smithi171 ceph-mon[34138]: osdmap e67: 11 total, 11 up, 11 in 2024-09-10T21:59:01.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:01 smithi171 ceph-mon[34138]: pgmap v249: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:01 smithi090 ceph-mon[30350]: osdmap e67: 11 total, 11 up, 11 in 2024-09-10T21:59:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:01 smithi090 ceph-mon[30350]: pgmap v249: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:02.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:01 smithi154 ceph-mon[34495]: osdmap e67: 11 total, 11 up, 11 in 2024-09-10T21:59:02.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:01 smithi154 ceph-mon[34495]: pgmap v249: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:04.832 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:04 smithi171 ceph-mon[34138]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:04.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:04 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/3236994641' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "599b64e5-d2f1-42c7-9532-67156d42d99a"} : dispatch 2024-09-10T21:59:04.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:04 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "599b64e5-d2f1-42c7-9532-67156d42d99a"} : dispatch 2024-09-10T21:59:04.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:04 smithi171 ceph-mon[34138]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "599b64e5-d2f1-42c7-9532-67156d42d99a"}]': finished 2024-09-10T21:59:04.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:04 smithi171 ceph-mon[34138]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T21:59:04.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:04 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:04.833 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:04 smithi171 ceph-mon[34138]: from='client.? 172.21.15.171:0/1477573719' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:59:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:04 smithi154 ceph-mon[34495]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:04 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/3236994641' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "599b64e5-d2f1-42c7-9532-67156d42d99a"} : dispatch 2024-09-10T21:59:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:04 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "599b64e5-d2f1-42c7-9532-67156d42d99a"} : dispatch 2024-09-10T21:59:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:04 smithi154 ceph-mon[34495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "599b64e5-d2f1-42c7-9532-67156d42d99a"}]': finished 2024-09-10T21:59:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:04 smithi154 ceph-mon[34495]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T21:59:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:04 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:04.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:04 smithi154 ceph-mon[34495]: from='client.? 172.21.15.171:0/1477573719' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:59:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:04 smithi090 ceph-mon[30350]: pgmap v250: 1 pgs: 1 remapped+peering; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:04 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/3236994641' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "599b64e5-d2f1-42c7-9532-67156d42d99a"} : dispatch 2024-09-10T21:59:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:04 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "599b64e5-d2f1-42c7-9532-67156d42d99a"} : dispatch 2024-09-10T21:59:05.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:04 smithi090 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "599b64e5-d2f1-42c7-9532-67156d42d99a"}]': finished 2024-09-10T21:59:05.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:04 smithi090 ceph-mon[30350]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T21:59:05.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:04 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:05.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:04 smithi090 ceph-mon[30350]: from='client.? 172.21.15.171:0/1477573719' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:59:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:06 smithi090 ceph-mon[30350]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:59:06.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:06 smithi154 ceph-mon[34495]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:59:06.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:06 smithi171 ceph-mon[34138]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:59:08.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:08 smithi171 ceph-mon[34138]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-10T21:59:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:08 smithi154 ceph-mon[34495]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-10T21:59:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:08 smithi090 ceph-mon[30350]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-10T21:59:10.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:10 smithi154 ceph-mon[34495]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T21:59:10.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:10 smithi171 ceph-mon[34138]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T21:59:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:10 smithi090 ceph-mon[30350]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T21:59:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:12 smithi154 ceph-mon[34495]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:59:12.963 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:12 smithi171 ceph-mon[34138]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:59:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:12 smithi090 ceph-mon[30350]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:59:13.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:13 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T21:59:13.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:13 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:13.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:13 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T21:59:13.922 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:13 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:14.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:13 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T21:59:14.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:13 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:14.879 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:14 smithi171 ceph-mon[34138]: Deploying daemon osd.11 on smithi171 2024-09-10T21:59:14.879 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:14 smithi171 ceph-mon[34138]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:59:14.917 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:14 smithi154 ceph-mon[34495]: Deploying daemon osd.11 on smithi171 2024-09-10T21:59:14.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:14 smithi154 ceph-mon[34495]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:59:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:14 smithi090 ceph-mon[30350]: Deploying daemon osd.11 on smithi171 2024-09-10T21:59:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:14 smithi090 ceph-mon[30350]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:59:16.792 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:16 smithi171 ceph-mon[34138]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T21:59:16.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:16 smithi154 ceph-mon[34495]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T21:59:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:16 smithi090 ceph-mon[30350]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T21:59:18.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:18 smithi154 ceph-mon[34495]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:18.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:18 smithi171 ceph-mon[34138]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:18 smithi090 ceph-mon[30350]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:20 smithi154 ceph-mon[34495]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:20 smithi171 ceph-mon[34138]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:20 smithi090 ceph-mon[30350]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:21.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:21 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:59:21.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:21 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:21.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:21 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:21.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:21 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:59:21.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:21 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:21.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:21 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:21 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:59:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:21 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:21 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:22.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:22 smithi154 ceph-mon[34495]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:22 smithi171 ceph-mon[34138]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:22 smithi090 ceph-mon[30350]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:24 smithi154 ceph-mon[34495]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:24.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:24 smithi171 ceph-mon[34138]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:24 smithi090 ceph-mon[30350]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:25.218 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.216+0000 7fc892a00640 1 -- 172.21.15.171:0/2114594530 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc89c063590 con 0x7fc8800b53b0 2024-09-10T21:59:25.218 INFO:teuthology.orchestra.run.smithi171.stdout:Created osd(s) 11 on host 'smithi171' 2024-09-10T21:59:25.220 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc8800b53b0 msgr2=0x7fc8800b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:25.220 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc8800b53b0 0x7fc8800b7870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00bbe0 tx=0x7fc88c00b390 comp rx=0 tx=0).stop 2024-09-10T21:59:25.220 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c109130 msgr2=0x7fc89c1a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:25.220 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c109130 0x7fc89c1a0cb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc89400eaa0 tx=0x7fc89400ef60 comp rx=0 tx=0).stop 2024-09-10T21:59:25.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 shutdown_connections 2024-09-10T21:59:25.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc8800b53b0 0x7fc8800b7870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:25.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc89c109130 0x7fc89c1a0cb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:25.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc89c1087b0 0x7fc89c19b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:25.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 --2- 172.21.15.171:0/2114594530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc89c102da0 0x7fc89c19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:25.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.218+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 >> 172.21.15.171:0/2114594530 conn(0x7fc89c0fe810 msgr2=0x7fc89c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:25.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.219+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 shutdown_connections 2024-09-10T21:59:25.221 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:25.219+0000 7fc8a0c00640 1 -- 172.21.15.171:0/2114594530 wait complete. 2024-09-10T21:59:25.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:25 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:59:25.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:25 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:25.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:25 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:25.979 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:25 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:59:25.980 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:25 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:25.980 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:25 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:26.016 DEBUG:teuthology.orchestra.run.smithi171:osd.11> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T21:59:26.021 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T21:59:26.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T21:59:26.076 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:25 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:59:26.076 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:25 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:26.076 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:25 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:26.399 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:59:26.601 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 21:59:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T21:59:26.541+0000 7fec2f024640 -1 osd.11 0 log_to_monitors true 2024-09-10T21:59:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:26 smithi090 ceph-mon[30350]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:26 smithi090 ceph-mon[30350]: from='osd.11 [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:59:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:26 smithi090 ceph-mon[30350]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:59:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:26 smithi154 ceph-mon[34495]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:26 smithi154 ceph-mon[34495]: from='osd.11 [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:59:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:26 smithi154 ceph-mon[34495]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:59:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:26 smithi171 ceph-mon[34138]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:26 smithi171 ceph-mon[34138]: from='osd.11 [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:59:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:26 smithi171 ceph-mon[34138]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:59:27.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.009+0000 7f04e3e00640 1 -- 172.21.15.90:0/3649466856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04dc10b350 msgr2=0x7f04dc111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:27.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.009+0000 7f04e3e00640 1 --2- 172.21.15.90:0/3649466856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04dc10b350 0x7f04dc111b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04d4009ca0 tx=0x7f04d402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.009+0000 7f04e3e00640 1 -- 172.21.15.90:0/3649466856 shutdown_connections 2024-09-10T21:59:27.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.009+0000 7f04e3e00640 1 --2- 172.21.15.90:0/3649466856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04dc10b350 0x7f04dc111b50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.009+0000 7f04e3e00640 1 --2- 172.21.15.90:0/3649466856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04dc10a9d0 0x7f04dc10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.009+0000 7f04e3e00640 1 --2- 172.21.15.90:0/3649466856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04dc104fc0 0x7f04dc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.009+0000 7f04e3e00640 1 -- 172.21.15.90:0/3649466856 >> 172.21.15.90:0/3649466856 conn(0x7f04dc1009f0 msgr2=0x7f04dc102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:27.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.009+0000 7f04e3e00640 1 -- 172.21.15.90:0/3649466856 shutdown_connections 2024-09-10T21:59:27.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.009+0000 7f04e3e00640 1 -- 172.21.15.90:0/3649466856 wait complete. 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.010+0000 7f04e3e00640 1 Processor -- start 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.010+0000 7f04e3e00640 1 -- start start 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04dc104fc0 0x7f04dc19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04dc10a9d0 0x7f04dc19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04dc10b350 0x7f04dc1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e3e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc114e80 con 0x7f04dc10b350 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e3e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc114d00 con 0x7f04dc10a9d0 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e3e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc115000 con 0x7f04dc104fc0 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04dc10b350 0x7f04dc1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04dc104fc0 0x7f04dc19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04dc10b350 0x7f04dc1a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40608/0 (socket says 172.21.15.90:40608) 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e1e00640 1 -- 172.21.15.90:0/1171567537 learned_addr learned my addr 172.21.15.90:0/1171567537 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e0a00640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04dc10a9d0 0x7f04dc19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e1400640 1 -- 172.21.15.90:0/1171567537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04dc10a9d0 msgr2=0x7f04dc19b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e1400640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04dc10a9d0 0x7f04dc19b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e1400640 1 -- 172.21.15.90:0/1171567537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04dc10b350 msgr2=0x7f04dc1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.011+0000 7f04e1400640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04dc10b350 0x7f04dc1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.012+0000 7f04e1400640 1 -- 172.21.15.90:0/1171567537 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc1a13c0 con 0x7f04dc104fc0 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.012+0000 7f04e1400640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04dc104fc0 0x7f04dc19b310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c970 tx=0x7f04d000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.012+0000 7f04da000640 1 -- 172.21.15.90:0/1171567537 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d0007bb0 con 0x7f04dc104fc0 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.012+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc1a16b0 con 0x7f04dc104fc0 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.012+0000 7f04da000640 1 -- 172.21.15.90:0/1171567537 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04d0007d50 con 0x7f04dc104fc0 2024-09-10T21:59:27.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.012+0000 7f04da000640 1 -- 172.21.15.90:0/1171567537 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d00186f0 con 0x7f04dc104fc0 2024-09-10T21:59:27.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.012+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc1a7f40 con 0x7f04dc104fc0 2024-09-10T21:59:27.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.013+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04dc106270 con 0x7f04dc104fc0 2024-09-10T21:59:27.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.016+0000 7f04da000640 1 -- 172.21.15.90:0/1171567537 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f04d000f450 con 0x7f04dc104fc0 2024-09-10T21:59:27.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.017+0000 7f04da000640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04c40b5330 0x7f04c40b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:27.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.017+0000 7f04da000640 1 -- 172.21.15.90:0/1171567537 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7024+0+0 (secure 0 0 0) 0x7f04d00e5a00 con 0x7f04dc104fc0 2024-09-10T21:59:27.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.017+0000 7f04e0a00640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04c40b5330 0x7f04c40b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:27.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.018+0000 7f04e0a00640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04c40b5330 0x7f04c40b77f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04dc107150 tx=0x7f04c8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:27.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.019+0000 7f04da000640 1 -- 172.21.15.90:0/1171567537 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f04d0014030 con 0x7f04dc104fc0 2024-09-10T21:59:27.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.203+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f04dc1053a0 con 0x7f04dc104fc0 2024-09-10T21:59:27.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.204+0000 7f04da000640 1 -- 172.21.15.90:0/1171567537 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7f04dc1053a0 con 0x7f04dc104fc0 2024-09-10T21:59:27.206 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:59:27.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04c40b5330 msgr2=0x7f04c40b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:27.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04c40b5330 0x7f04c40b77f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f04dc107150 tx=0x7f04c8005f90 comp rx=0 tx=0).stop 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04dc104fc0 msgr2=0x7f04dc19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04dc104fc0 0x7f04dc19b310 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f04d000c970 tx=0x7f04d000ce30 comp rx=0 tx=0).stop 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 shutdown_connections 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04c40b5330 0x7f04c40b77f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04dc10b350 0x7f04dc1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04dc10a9d0 0x7f04dc19b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 --2- 172.21.15.90:0/1171567537 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04dc104fc0 0x7f04dc19b310 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 >> 172.21.15.90:0/1171567537 conn(0x7f04dc1009f0 msgr2=0x7f04dc102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 shutdown_connections 2024-09-10T21:59:27.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:27.207+0000 7f04e3e00640 1 -- 172.21.15.90:0/1171567537 wait complete. 2024-09-10T21:59:27.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:27 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1171567537' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:59:27.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:27 smithi171 ceph-mon[34138]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T21:59:27.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:27 smithi171 ceph-mon[34138]: osdmap e69: 12 total, 11 up, 12 in 2024-09-10T21:59:27.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:27 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:27.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:27 smithi171 ceph-mon[34138]: from='osd.11 [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:59:27.798 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:27 smithi171 ceph-mon[34138]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:59:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:27 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1171567537' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:59:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:27 smithi090 ceph-mon[30350]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T21:59:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:27 smithi090 ceph-mon[30350]: osdmap e69: 12 total, 11 up, 12 in 2024-09-10T21:59:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:27 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:27 smithi090 ceph-mon[30350]: from='osd.11 [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:59:27.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:27 smithi090 ceph-mon[30350]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:59:28.108 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726005538,"num_in_osds":12,"osd_in_since":1726005543,"num_remapped_pgs":0} 2024-09-10T21:59:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:27 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1171567537' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:59:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:27 smithi154 ceph-mon[34495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T21:59:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:27 smithi154 ceph-mon[34495]: osdmap e69: 12 total, 11 up, 12 in 2024-09-10T21:59:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:27 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:27 smithi154 ceph-mon[34495]: from='osd.11 [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:59:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:27 smithi154 ceph-mon[34495]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]} : dispatch 2024-09-10T21:59:28.497 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 21:59:28 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T21:59:28.241+0000 7fec29c00640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T21:59:28.498 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 21:59:28 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T21:59:28.257+0000 7fec24c00640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:59:28.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:28 smithi171 ceph-mon[34138]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:28 smithi171 ceph-mon[34138]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:59:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:28 smithi171 ceph-mon[34138]: osdmap e70: 12 total, 11 up, 12 in 2024-09-10T21:59:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:28 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:28 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:28 smithi090 ceph-mon[30350]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:28 smithi090 ceph-mon[30350]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:59:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:28 smithi090 ceph-mon[30350]: osdmap e70: 12 total, 11 up, 12 in 2024-09-10T21:59:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:28 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:28 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:29.110 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T21:59:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:28 smithi154 ceph-mon[34495]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:59:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:28 smithi154 ceph-mon[34495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi171", "root=default"]}]': finished 2024-09-10T21:59:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:28 smithi154 ceph-mon[34495]: osdmap e70: 12 total, 11 up, 12 in 2024-09-10T21:59:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:28 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:28 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:29.497 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:59:30.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.125+0000 7f27d5200640 1 -- 172.21.15.90:0/37972409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27d0102170 msgr2=0x7f27d010eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:30.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.125+0000 7f27d5200640 1 --2- 172.21.15.90:0/37972409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27d0102170 0x7f27d010eec0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009900 tx=0x7f27bc02a480 comp rx=0 tx=0).stop 2024-09-10T21:59:30.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.125+0000 7f27d5200640 1 -- 172.21.15.90:0/37972409 shutdown_connections 2024-09-10T21:59:30.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.125+0000 7f27d5200640 1 --2- 172.21.15.90:0/37972409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d010f400 0x7f27d01117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:30.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.125+0000 7f27d5200640 1 --2- 172.21.15.90:0/37972409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27d0102170 0x7f27d010eec0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:30.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.125+0000 7f27d5200640 1 --2- 172.21.15.90:0/37972409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27d0101850 0x7f27d0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:30.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.125+0000 7f27d5200640 1 -- 172.21.15.90:0/37972409 >> 172.21.15.90:0/37972409 conn(0x7f27d00fd700 msgr2=0x7f27d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:30.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.125+0000 7f27d5200640 1 -- 172.21.15.90:0/37972409 shutdown_connections 2024-09-10T21:59:30.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.125+0000 7f27d5200640 1 -- 172.21.15.90:0/37972409 wait complete. 2024-09-10T21:59:30.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.126+0000 7f27d5200640 1 Processor -- start 2024-09-10T21:59:30.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.126+0000 7f27d5200640 1 -- start start 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27d5200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27d0101850 0x7f27d019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27d5200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27d0102170 0x7f27d019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27d5200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d010f400 0x7f27d01a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27d5200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0114e40 con 0x7f27d0101850 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27d5200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0114cc0 con 0x7f27d0102170 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27d5200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0114fc0 con 0x7f27d010f400 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27cde00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27d0101850 0x7f27d019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27cde00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27d0101850 0x7f27d019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54916/0 (socket says 172.21.15.90:54916) 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27cde00640 1 -- 172.21.15.90:0/1104015754 learned_addr learned my addr 172.21.15.90:0/1104015754 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27cd400640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27d0102170 0x7f27d019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:30.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.127+0000 7f27ce800640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d010f400 0x7f27d01a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:30.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.128+0000 7f27cd400640 1 -- 172.21.15.90:0/1104015754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d010f400 msgr2=0x7f27d01a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:30.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.128+0000 7f27cd400640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d010f400 0x7f27d01a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:30.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.128+0000 7f27cd400640 1 -- 172.21.15.90:0/1104015754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27d0101850 msgr2=0x7f27d019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:30.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.128+0000 7f27cd400640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27d0101850 0x7f27d019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:30.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.128+0000 7f27cd400640 1 -- 172.21.15.90:0/1104015754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d01a14e0 con 0x7f27d0102170 2024-09-10T21:59:30.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.128+0000 7f27ce800640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d010f400 0x7f27d01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:59:30.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.128+0000 7f27cd400640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27d0102170 0x7f27d019b970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27bc02a9a0 tx=0x7f27bc003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:30.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.129+0000 7f27c6a00640 1 -- 172.21.15.90:0/1104015754 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc03f070 con 0x7f27d0102170 2024-09-10T21:59:30.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.129+0000 7f27c6a00640 1 -- 172.21.15.90:0/1104015754 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27bc03baf0 con 0x7f27d0102170 2024-09-10T21:59:30.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.129+0000 7f27c6a00640 1 -- 172.21.15.90:0/1104015754 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27bc035030 con 0x7f27d0102170 2024-09-10T21:59:30.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.129+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d01a1770 con 0x7f27d0102170 2024-09-10T21:59:30.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.129+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d01a7e90 con 0x7f27d0102170 2024-09-10T21:59:30.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.130+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d0101c30 con 0x7f27d0102170 2024-09-10T21:59:30.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.132+0000 7f27c6a00640 1 -- 172.21.15.90:0/1104015754 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f27bc03b660 con 0x7f27d0102170 2024-09-10T21:59:30.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.133+0000 7f27c6a00640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b40b53b0 0x7f27b40b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:30.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.134+0000 7f27c6a00640 1 -- 172.21.15.90:0/1104015754 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7342+0+0 (secure 0 0 0) 0x7f27bc106200 con 0x7f27d0102170 2024-09-10T21:59:30.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.134+0000 7f27cde00640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b40b53b0 0x7f27b40b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:30.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.135+0000 7f27cde00640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b40b53b0 0x7f27b40b7870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009c90 tx=0x7f27c0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:30.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.136+0000 7f27c6a00640 1 -- 172.21.15.90:0/1104015754 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f27bc0ccda0 con 0x7f27d0102170 2024-09-10T21:59:30.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.328+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f27d01a8220 con 0x7f27d0102170 2024-09-10T21:59:30.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.329+0000 7f27c6a00640 1 -- 172.21.15.90:0/1104015754 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f27bc0d17d0 con 0x7f27d0102170 2024-09-10T21:59:30.331 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:59:30.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.331+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b40b53b0 msgr2=0x7f27b40b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:30.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.331+0000 7f27d5200640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b40b53b0 0x7f27b40b7870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009c90 tx=0x7f27c0005f90 comp rx=0 tx=0).stop 2024-09-10T21:59:30.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.331+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27d0102170 msgr2=0x7f27d019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:30.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.331+0000 7f27d5200640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27d0102170 0x7f27d019b970 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27bc02a9a0 tx=0x7f27bc003b90 comp rx=0 tx=0).stop 2024-09-10T21:59:30.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.332+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 shutdown_connections 2024-09-10T21:59:30.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.332+0000 7f27d5200640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b40b53b0 0x7f27b40b7870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:30.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.332+0000 7f27d5200640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27d010f400 0x7f27d01a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:30.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.332+0000 7f27d5200640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27d0102170 0x7f27d019b970 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:30.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.332+0000 7f27d5200640 1 --2- 172.21.15.90:0/1104015754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27d0101850 0x7f27d019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:30.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.332+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 >> 172.21.15.90:0/1104015754 conn(0x7f27d00fd700 msgr2=0x7f27d00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:30.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.332+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 shutdown_connections 2024-09-10T21:59:30.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:30.332+0000 7f27d5200640 1 -- 172.21.15.90:0/1104015754 wait complete. 2024-09-10T21:59:30.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T21:59:30.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T21:59:30.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: osd.11 [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] boot 2024-09-10T21:59:30.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:30.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:30 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: purged_snaps scrub starts 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: purged_snaps scrub ok 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: osd.11 [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] boot 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:30.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:30.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:30 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: purged_snaps scrub starts 2024-09-10T21:59:30.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: purged_snaps scrub ok 2024-09-10T21:59:30.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: osd.11 [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] boot 2024-09-10T21:59:30.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T21:59:30.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:59:30.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:30.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:59:30.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:30.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:30.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:30.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:30 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:31.202 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1726005569,"num_in_osds":12,"osd_in_since":1726005543,"num_remapped_pgs":0} 2024-09-10T21:59:31.203 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:59:31.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:31 smithi171 ceph-mon[34138]: Detected new or changed devices on smithi171 2024-09-10T21:59:31.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:31 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi171 to 4272M 2024-09-10T21:59:31.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:31 smithi171 ceph-mon[34138]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T21:59:31.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:31 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1104015754' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:59:31.568 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:59:31.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:31 smithi090 ceph-mon[30350]: Detected new or changed devices on smithi171 2024-09-10T21:59:31.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:31 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi171 to 4272M 2024-09-10T21:59:31.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:31 smithi090 ceph-mon[30350]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T21:59:31.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:31 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1104015754' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:59:31.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:31 smithi154 ceph-mon[34495]: Detected new or changed devices on smithi171 2024-09-10T21:59:31.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:31 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi171 to 4272M 2024-09-10T21:59:31.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:31 smithi154 ceph-mon[34495]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T21:59:31.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:31 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1104015754' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:59:32.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.219+0000 7fc367200640 1 -- 172.21.15.90:0/1963045957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36810a800 msgr2=0x7fc36810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.219+0000 7fc367200640 1 --2- 172.21.15.90:0/1963045957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36810a800 0x7fc36810ac40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc354009900 tx=0x7fc35402a410 comp rx=0 tx=0).stop 2024-09-10T21:59:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.219+0000 7fc367200640 1 -- 172.21.15.90:0/1963045957 shutdown_connections 2024-09-10T21:59:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.219+0000 7fc367200640 1 --2- 172.21.15.90:0/1963045957 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc36810b180 0x7fc368111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.219+0000 7fc367200640 1 --2- 172.21.15.90:0/1963045957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36810a800 0x7fc36810ac40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.219+0000 7fc367200640 1 --2- 172.21.15.90:0/1963045957 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc368104df0 0x7fc3681051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.219+0000 7fc367200640 1 -- 172.21.15.90:0/1963045957 >> 172.21.15.90:0/1963045957 conn(0x7fc368100840 msgr2=0x7fc368102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.219+0000 7fc367200640 1 -- 172.21.15.90:0/1963045957 shutdown_connections 2024-09-10T21:59:32.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.219+0000 7fc367200640 1 -- 172.21.15.90:0/1963045957 wait complete. 2024-09-10T21:59:32.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.220+0000 7fc367200640 1 Processor -- start 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.220+0000 7fc367200640 1 -- start start 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.220+0000 7fc367200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc368104df0 0x7fc36819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.220+0000 7fc367200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc36810a800 0x7fc36819b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.220+0000 7fc367200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc36810b180 0x7fc3681a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.220+0000 7fc367200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc368114d00 con 0x7fc368104df0 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.220+0000 7fc367200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc368114b80 con 0x7fc36810b180 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.220+0000 7fc367200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc368114e80 con 0x7fc36810a800 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc368104df0 0x7fc36819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc368104df0 0x7fc36819b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54928/0 (socket says 172.21.15.90:54928) 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365e00640 1 -- 172.21.15.90:0/2719979434 learned_addr learned my addr 172.21.15.90:0/2719979434 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc366800640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc36810b180 0x7fc3681a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365400640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc36810a800 0x7fc36819b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:32.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365e00640 1 -- 172.21.15.90:0/2719979434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc36810a800 msgr2=0x7fc36819b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:32.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365e00640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc36810a800 0x7fc36819b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:32.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365e00640 1 -- 172.21.15.90:0/2719979434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc36810b180 msgr2=0x7fc3681a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:32.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365e00640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc36810b180 0x7fc3681a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:32.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365e00640 1 -- 172.21.15.90:0/2719979434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3681a11c0 con 0x7fc368104df0 2024-09-10T21:59:32.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.221+0000 7fc365e00640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc368104df0 0x7fc36819b110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc35800d770 tx=0x7fc35800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:32.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.222+0000 7fc35ea00640 1 -- 172.21.15.90:0/2719979434 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358017070 con 0x7fc368104df0 2024-09-10T21:59:32.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.222+0000 7fc35ea00640 1 -- 172.21.15.90:0/2719979434 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3580054b0 con 0x7fc368104df0 2024-09-10T21:59:32.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.222+0000 7fc35ea00640 1 -- 172.21.15.90:0/2719979434 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358004c60 con 0x7fc368104df0 2024-09-10T21:59:32.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.222+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3681a14b0 con 0x7fc368104df0 2024-09-10T21:59:32.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.222+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3681a7b70 con 0x7fc368104df0 2024-09-10T21:59:32.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.223+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3681051d0 con 0x7fc368104df0 2024-09-10T21:59:32.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.225+0000 7fc35ea00640 1 -- 172.21.15.90:0/2719979434 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fc358004e00 con 0x7fc368104df0 2024-09-10T21:59:32.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.226+0000 7fc35ea00640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc34c0b5460 0x7fc34c0b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:32.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.227+0000 7fc35ea00640 1 -- 172.21.15.90:0/2719979434 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fc3580e5310 con 0x7fc368104df0 2024-09-10T21:59:32.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.227+0000 7fc365400640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc34c0b5460 0x7fc34c0b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:32.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.227+0000 7fc365400640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc34c0b5460 0x7fc34c0b7920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc354006010 tx=0x7fc354005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:32.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.230+0000 7fc35ea00640 1 -- 172.21.15.90:0/2719979434 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc3580ac500 con 0x7fc368104df0 2024-09-10T21:59:32.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.417+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc36802e5c0 con 0x7fc368104df0 2024-09-10T21:59:32.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.418+0000 7fc35ea00640 1 -- 172.21.15.90:0/2719979434 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18796 (secure 0 0 0) 0x7fc3580c6e20 con 0x7fc368104df0 2024-09-10T21:59:32.420 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:59:32.420 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":72,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T21:59:30.225970+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:59:32.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc34c0b5460 msgr2=0x7fc34c0b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:32.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc34c0b5460 0x7fc34c0b7920 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc354006010 tx=0x7fc354005a20 comp rx=0 tx=0).stop 2024-09-10T21:59:32.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc368104df0 msgr2=0x7fc36819b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:32.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc368104df0 0x7fc36819b110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc35800d770 tx=0x7fc35800dc30 comp rx=0 tx=0).stop 2024-09-10T21:59:32.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 shutdown_connections 2024-09-10T21:59:32.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc34c0b5460 0x7fc34c0b7920 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:32.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc36810b180 0x7fc3681a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:32.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc36810a800 0x7fc36819b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:32.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 --2- 172.21.15.90:0/2719979434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc368104df0 0x7fc36819b110 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:32.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.420+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 >> 172.21.15.90:0/2719979434 conn(0x7fc368100840 msgr2=0x7fc368102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:32.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.421+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 shutdown_connections 2024-09-10T21:59:32.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:32.421+0000 7fc367200640 1 -- 172.21.15.90:0/2719979434 wait complete. 2024-09-10T21:59:32.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:32 smithi171 ceph-mon[34138]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:32.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:32 smithi090 ceph-mon[30350]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:32.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:32 smithi154 ceph-mon[34495]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:33.331 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:54:47.794295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:59:33.332 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T21:59:33.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:33 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2719979434' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:59:33.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:33 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2719979434' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:59:33.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:33 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2719979434' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:59:33.694 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.298+0000 7fac9e800640 1 -- 172.21.15.90:0/2165568184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac9810a800 msgr2=0x7fac9810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.298+0000 7fac9e800640 1 --2- 172.21.15.90:0/2165568184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac9810a800 0x7fac9810ac40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac84009900 tx=0x7fac8402a410 comp rx=0 tx=0).stop 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.299+0000 7fac9e800640 1 -- 172.21.15.90:0/2165568184 shutdown_connections 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.299+0000 7fac9e800640 1 --2- 172.21.15.90:0/2165568184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac9810b180 0x7fac98111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.299+0000 7fac9e800640 1 --2- 172.21.15.90:0/2165568184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac9810a800 0x7fac9810ac40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.299+0000 7fac9e800640 1 --2- 172.21.15.90:0/2165568184 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac98104df0 0x7fac981051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.299+0000 7fac9e800640 1 -- 172.21.15.90:0/2165568184 >> 172.21.15.90:0/2165568184 conn(0x7fac98100840 msgr2=0x7fac98102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.299+0000 7fac9e800640 1 -- 172.21.15.90:0/2165568184 shutdown_connections 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.299+0000 7fac9e800640 1 -- 172.21.15.90:0/2165568184 wait complete. 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9e800640 1 Processor -- start 2024-09-10T21:59:34.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9e800640 1 -- start start 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac98104df0 0x7fac9819b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac9810a800 0x7fac9819b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac9810b180 0x7fac981a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9e800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac98114d40 con 0x7fac98104df0 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9e800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fac98114bc0 con 0x7fac9810b180 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9e800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fac98114ec0 con 0x7fac9810a800 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac98104df0 0x7fac9819b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac98104df0 0x7fac9819b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54938/0 (socket says 172.21.15.90:54938) 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9d400640 1 -- 172.21.15.90:0/155806752 learned_addr learned my addr 172.21.15.90:0/155806752 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.300+0000 7fac9ca00640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac9810a800 0x7fac9819b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac9de00640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac9810b180 0x7fac981a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac9d400640 1 -- 172.21.15.90:0/155806752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac9810a800 msgr2=0x7fac9819b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:34.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac9d400640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac9810a800 0x7fac9819b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac9d400640 1 -- 172.21.15.90:0/155806752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac9810b180 msgr2=0x7fac981a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac9d400640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac9810b180 0x7fac981a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac9d400640 1 -- 172.21.15.90:0/155806752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac981a1260 con 0x7fac98104df0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac9d400640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac98104df0 0x7fac9819b1b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00d770 tx=0x7fac8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac96000640 1 -- 172.21.15.90:0/155806752 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c017070 con 0x7fac98104df0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac96000640 1 -- 172.21.15.90:0/155806752 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac8c0054b0 con 0x7fac98104df0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac96000640 1 -- 172.21.15.90:0/155806752 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8c004c60 con 0x7fac98104df0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac981a1550 con 0x7fac98104df0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.301+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac981a7c10 con 0x7fac98104df0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.302+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac981051d0 con 0x7fac98104df0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.304+0000 7fac96000640 1 -- 172.21.15.90:0/155806752 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fac8c004e00 con 0x7fac98104df0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.306+0000 7fac96000640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac800b5460 0x7fac800b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.306+0000 7fac96000640 1 -- 172.21.15.90:0/155806752 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fac8c0e5390 con 0x7fac98104df0 2024-09-10T21:59:34.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.306+0000 7fac9ca00640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac800b5460 0x7fac800b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:34.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.307+0000 7fac9ca00640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac800b5460 0x7fac800b7920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac84006010 tx=0x7fac84005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:34.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.309+0000 7fac96000640 1 -- 172.21.15.90:0/155806752 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac8c0ac580 con 0x7fac98104df0 2024-09-10T21:59:34.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.492+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac9802e5c0 con 0x7fac98104df0 2024-09-10T21:59:34.494 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:34 smithi090 ceph-mon[30350]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:34.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.493+0000 7fac96000640 1 -- 172.21.15.90:0/155806752 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7fac8c0abe80 con 0x7fac98104df0 2024-09-10T21:59:34.495 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-09-10T21:59:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac800b5460 msgr2=0x7fac800b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac800b5460 0x7fac800b7920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fac84006010 tx=0x7fac84005a20 comp rx=0 tx=0).stop 2024-09-10T21:59:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac98104df0 msgr2=0x7fac9819b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac98104df0 0x7fac9819b1b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00d770 tx=0x7fac8c00dc30 comp rx=0 tx=0).stop 2024-09-10T21:59:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 shutdown_connections 2024-09-10T21:59:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac800b5460 0x7fac800b7920 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac9810b180 0x7fac981a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac9810a800 0x7fac9819b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:34.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 --2- 172.21.15.90:0/155806752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac98104df0 0x7fac9819b1b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:34.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.495+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 >> 172.21.15.90:0/155806752 conn(0x7fac98100840 msgr2=0x7fac98106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:34.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.496+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 shutdown_connections 2024-09-10T21:59:34.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:34.496+0000 7fac9e800640 1 -- 172.21.15.90:0/155806752 wait complete. 2024-09-10T21:59:34.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:34 smithi154 ceph-mon[34495]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:34.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:34 smithi171 ceph-mon[34138]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:35.254 INFO:tasks.cephadm:Adding mds.a on smithi090 2024-09-10T21:59:35.254 INFO:tasks.cephadm:Adding mds.d on smithi090 2024-09-10T21:59:35.254 INFO:tasks.cephadm:Adding mds.g on smithi090 2024-09-10T21:59:35.255 INFO:tasks.cephadm:Adding mds.j on smithi090 2024-09-10T21:59:35.255 INFO:tasks.cephadm:Adding mds.b on smithi154 2024-09-10T21:59:35.255 INFO:tasks.cephadm:Adding mds.e on smithi154 2024-09-10T21:59:35.255 INFO:tasks.cephadm:Adding mds.h on smithi154 2024-09-10T21:59:35.255 INFO:tasks.cephadm:Adding mds.k on smithi154 2024-09-10T21:59:35.255 INFO:tasks.cephadm:Adding mds.c on smithi171 2024-09-10T21:59:35.255 INFO:tasks.cephadm:Adding mds.f on smithi171 2024-09-10T21:59:35.255 INFO:tasks.cephadm:Adding mds.i on smithi171 2024-09-10T21:59:35.255 INFO:tasks.cephadm:Adding mds.l on smithi171 2024-09-10T21:59:35.255 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l' 2024-09-10T21:59:35.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:35 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/155806752' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:59:35.636 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:59:35.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:35 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/155806752' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:59:35.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:35 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/155806752' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:59:36.310 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.309+0000 7f5723e00640 1 -- 172.21.15.171:0/2252673352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c10f3c0 msgr2=0x7f571c1117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:36.310 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.309+0000 7f5723e00640 1 --2- 172.21.15.171:0/2252673352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c10f3c0 0x7f571c1117b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5714009ca0 tx=0x7f571402a8b0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.310 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.309+0000 7f5723e00640 1 -- 172.21.15.171:0/2252673352 shutdown_connections 2024-09-10T21:59:36.310 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.309+0000 7f5723e00640 1 --2- 172.21.15.171:0/2252673352 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c10f3c0 0x7f571c1117b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.311 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.309+0000 7f5723e00640 1 --2- 172.21.15.171:0/2252673352 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571c075f40 0x7f571c10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.311 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.309+0000 7f5723e00640 1 --2- 172.21.15.171:0/2252673352 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f571c075540 0x7f571c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.311 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.309+0000 7f5723e00640 1 -- 172.21.15.171:0/2252673352 >> 172.21.15.171:0/2252673352 conn(0x7f571c0fe810 msgr2=0x7f571c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:36.311 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.309+0000 7f5723e00640 1 -- 172.21.15.171:0/2252673352 shutdown_connections 2024-09-10T21:59:36.311 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.309+0000 7f5723e00640 1 -- 172.21.15.171:0/2252673352 wait complete. 2024-09-10T21:59:36.312 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.310+0000 7f5723e00640 1 Processor -- start 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.310+0000 7f5723e00640 1 -- start start 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5723e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571c075540 0x7f571c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5723e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c075f40 0x7f571c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5723e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f571c10f3c0 0x7f571c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5723e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f571c114e40 con 0x7f571c075540 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5723e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f571c114cc0 con 0x7f571c10f3c0 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5723e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f571c114fc0 con 0x7f571c075f40 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5720a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c075f40 0x7f571c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5720a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c075f40 0x7f571c19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.171:51510/0 (socket says 172.21.15.171:51510) 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5720a00640 1 -- 172.21.15.171:0/2015948849 learned_addr learned my addr 172.21.15.171:0/2015948849 (peer_addr_for_me v2:172.21.15.171:0/0) 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5721400640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571c075540 0x7f571c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5721e00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f571c10f3c0 0x7f571c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5720a00640 1 -- 172.21.15.171:0/2015948849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f571c10f3c0 msgr2=0x7f571c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:36.313 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5720a00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f571c10f3c0 0x7f571c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5720a00640 1 -- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571c075540 msgr2=0x7f571c19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:36.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5720a00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571c075540 0x7f571c19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.311+0000 7f5720a00640 1 -- 172.21.15.171:0/2015948849 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c1a1420 con 0x7f571c075f40 2024-09-10T21:59:36.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.312+0000 7f5720a00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c075f40 0x7f571c19b8b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f570800d950 tx=0x7f570800dd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:36.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.312+0000 7f571a000640 1 -- 172.21.15.171:0/2015948849 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708017070 con 0x7f571c075f40 2024-09-10T21:59:36.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.312+0000 7f571a000640 1 -- 172.21.15.171:0/2015948849 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f570800de80 con 0x7f571c075f40 2024-09-10T21:59:36.314 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.312+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571c1a1710 con 0x7f571c075f40 2024-09-10T21:59:36.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.312+0000 7f571a000640 1 -- 172.21.15.171:0/2015948849 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570800f620 con 0x7f571c075f40 2024-09-10T21:59:36.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.312+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571c1a7dd0 con 0x7f571c075f40 2024-09-10T21:59:36.315 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.313+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571c075920 con 0x7f571c075f40 2024-09-10T21:59:36.316 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.315+0000 7f571a000640 1 -- 172.21.15.171:0/2015948849 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f5708010460 con 0x7f571c075f40 2024-09-10T21:59:36.319 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.319+0000 7f571a000640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57040b5330 0x7f57040b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:36.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.319+0000 7f571a000640 1 -- 172.21.15.171:0/2015948849 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f57080e5430 con 0x7f571c075f40 2024-09-10T21:59:36.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.319+0000 7f5721400640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57040b5330 0x7f57040b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:36.320 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.320+0000 7f5721400640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57040b5330 0x7f57040b77f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f571000abe0 tx=0x7f571000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:36.323 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.322+0000 7f571a000640 1 -- 172.21.15.171:0/2015948849 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f57080ac620 con 0x7f571c075f40 2024-09-10T21:59:36.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:36 smithi171 ceph-mon[34138]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:36.519 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.518+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}) -- 0x7f571c100520 con 0x7f57040b5330 2024-09-10T21:59:36.529 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.529+0000 7f571a000640 1 -- 172.21.15.171:0/2015948849 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f571c100520 con 0x7f57040b5330 2024-09-10T21:59:36.529 INFO:teuthology.orchestra.run.smithi171.stdout:Scheduled mds.all update... 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57040b5330 msgr2=0x7f57040b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57040b5330 0x7f57040b77f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f571000abe0 tx=0x7f571000a2d0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c075f40 msgr2=0x7f571c19b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c075f40 0x7f571c19b8b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f570800d950 tx=0x7f570800dd50 comp rx=0 tx=0).stop 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 shutdown_connections 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f57040b5330 0x7f57040b77f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f571c10f3c0 0x7f571c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f571c075f40 0x7f571c19b8b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 --2- 172.21.15.171:0/2015948849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f571c075540 0x7f571c19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:36.532 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 >> 172.21.15.171:0/2015948849 conn(0x7f571c0fe810 msgr2=0x7f571c111590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:36.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 shutdown_connections 2024-09-10T21:59:36.533 INFO:teuthology.orchestra.run.smithi171.stderr:2024-09-10T21:59:36.531+0000 7f5723e00640 1 -- 172.21.15.171:0/2015948849 wait complete. 2024-09-10T21:59:36.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:36 smithi090 ceph-mon[30350]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:36.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:36 smithi154 ceph-mon[34495]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:37.377 DEBUG:teuthology.orchestra.run.smithi090:mds.a> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T21:59:37.381 DEBUG:teuthology.orchestra.run.smithi090:mds.d> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T21:59:37.426 DEBUG:teuthology.orchestra.run.smithi090:mds.g> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T21:59:37.470 DEBUG:teuthology.orchestra.run.smithi090:mds.j> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T21:59:37.516 DEBUG:teuthology.orchestra.run.smithi154:mds.b> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T21:59:37.521 DEBUG:teuthology.orchestra.run.smithi154:mds.e> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T21:59:37.566 DEBUG:teuthology.orchestra.run.smithi154:mds.h> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T21:59:37.597 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:37.598 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:37 smithi171 ceph-mon[34138]: Deploying daemon mds.l on smithi171 2024-09-10T21:59:37.613 DEBUG:teuthology.orchestra.run.smithi154:mds.k> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T21:59:37.658 DEBUG:teuthology.orchestra.run.smithi171:mds.c> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T21:59:37.662 DEBUG:teuthology.orchestra.run.smithi171:mds.f> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T21:59:37.708 DEBUG:teuthology.orchestra.run.smithi171:mds.i> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T21:59:37.754 DEBUG:teuthology.orchestra.run.smithi171:mds.l> sudo journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T21:59:37.800 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T21:59:37.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:59:37.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:59:37.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-09-10T21:59:37.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:37.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:59:37.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:37.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:37.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:37.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:37.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:37.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:37.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:37 smithi090 ceph-mon[30350]: Deploying daemon mds.l on smithi171 2024-09-10T21:59:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:59:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-09-10T21:59:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:59:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:59:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:37.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:37.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:37 smithi154 ceph-mon[34495]: Deploying daemon mds.l on smithi171 2024-09-10T21:59:38.159 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:59:38.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:38 smithi090 ceph-mon[30350]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:38.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:38 smithi171 ceph-mon[34138]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:38.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.754+0000 7f0738200640 1 -- 172.21.15.90:0/818584000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 msgr2=0x7f0730075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:38.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.754+0000 7f0738200640 1 --2- 172.21.15.90:0/818584000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 0x7f0730075920 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0724009900 tx=0x7f072402a3f0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.755+0000 7f0738200640 1 -- 172.21.15.90:0/818584000 shutdown_connections 2024-09-10T21:59:38.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.755+0000 7f0738200640 1 --2- 172.21.15.90:0/818584000 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f073010f3c0 0x7f07301117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.755+0000 7f0738200640 1 --2- 172.21.15.90:0/818584000 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0730075f40 0x7f073010edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.755+0000 7f0738200640 1 --2- 172.21.15.90:0/818584000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 0x7f0730075920 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.755+0000 7f0738200640 1 -- 172.21.15.90:0/818584000 >> 172.21.15.90:0/818584000 conn(0x7f07300fe810 msgr2=0x7f0730100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:38.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.755+0000 7f0738200640 1 -- 172.21.15.90:0/818584000 shutdown_connections 2024-09-10T21:59:38.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.755+0000 7f0738200640 1 -- 172.21.15.90:0/818584000 wait complete. 2024-09-10T21:59:38.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0738200640 1 Processor -- start 2024-09-10T21:59:38.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0738200640 1 -- start start 2024-09-10T21:59:38.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0738200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 0x7f073019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:38.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0738200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0730075f40 0x7f073019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:38.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0738200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f073010f3c0 0x7f07301a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:38.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0738200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0730114ee0 con 0x7f0730075540 2024-09-10T21:59:38.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0738200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0730114d60 con 0x7f073010f3c0 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0738200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0730115060 con 0x7f0730075f40 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0735800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 0x7f073019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0735800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 0x7f073019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57158/0 (socket says 172.21.15.90:57158) 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0735800640 1 -- 172.21.15.90:0/3540927275 learned_addr learned my addr 172.21.15.90:0/3540927275 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.756+0000 7f0734e00640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0730075f40 0x7f073019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f0735800640 1 -- 172.21.15.90:0/3540927275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0730075f40 msgr2=0x7f073019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f0735800640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0730075f40 0x7f073019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f0735800640 1 -- 172.21.15.90:0/3540927275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f073010f3c0 msgr2=0x7f07301a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f0736200640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f073010f3c0 0x7f07301a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f0735800640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f073010f3c0 0x7f07301a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f0735800640 1 -- 172.21.15.90:0/3540927275 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07301a1400 con 0x7f0730075540 2024-09-10T21:59:38.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f0736200640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f073010f3c0 0x7f07301a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:59:38.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f0735800640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 0x7f073019b350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f072402a910 tx=0x7f0724003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:38.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f072e000640 1 -- 172.21.15.90:0/3540927275 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072403f070 con 0x7f0730075540 2024-09-10T21:59:38.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f072e000640 1 -- 172.21.15.90:0/3540927275 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f072403bd00 con 0x7f0730075540 2024-09-10T21:59:38.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07301a1690 con 0x7f0730075540 2024-09-10T21:59:38.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.757+0000 7f072e000640 1 -- 172.21.15.90:0/3540927275 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724035030 con 0x7f0730075540 2024-09-10T21:59:38.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.758+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07301a7db0 con 0x7f0730075540 2024-09-10T21:59:38.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.759+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0730075920 con 0x7f0730075540 2024-09-10T21:59:38.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.761+0000 7f072e000640 1 -- 172.21.15.90:0/3540927275 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f0724031400 con 0x7f0730075540 2024-09-10T21:59:38.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.762+0000 7f072e000640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07180b5430 0x7f07180b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:38.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.763+0000 7f072e000640 1 -- 172.21.15.90:0/3540927275 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f0724106af0 con 0x7f0730075540 2024-09-10T21:59:38.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.763+0000 7f0734e00640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07180b5430 0x7f07180b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:38.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.763+0000 7f0734e00640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07180b5430 0x7f07180b78f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0730101040 tx=0x7f071c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:38.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.764+0000 7f072e000640 1 -- 172.21.15.90:0/3540927275 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f07240cdce0 con 0x7f0730075540 2024-09-10T21:59:38.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:38 smithi154 ceph-mon[34495]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:38.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.981+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f073019c480 con 0x7f0730075540 2024-09-10T21:59:38.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.981+0000 7f072e000640 1 -- 172.21.15.90:0/3540927275 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f07240cd5e0 con 0x7f0730075540 2024-09-10T21:59:38.983 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:59:38.983 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":1,"btime":"2024-09-10T21:50:11:271199+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T21:59:38.983 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 1 2024-09-10T21:59:38.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07180b5430 msgr2=0x7f07180b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:38.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07180b5430 0x7f07180b78f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0730101040 tx=0x7f071c00a2d0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 msgr2=0x7f073019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 0x7f073019b350 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f072402a910 tx=0x7f0724003d60 comp rx=0 tx=0).stop 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 shutdown_connections 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07180b5430 0x7f07180b78f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f073010f3c0 0x7f07301a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0730075f40 0x7f073019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 --2- 172.21.15.90:0/3540927275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0730075540 0x7f073019b350 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 >> 172.21.15.90:0/3540927275 conn(0x7f07300fe810 msgr2=0x7f0730111560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 shutdown_connections 2024-09-10T21:59:38.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:38.984+0000 7f0738200640 1 -- 172.21.15.90:0/3540927275 wait complete. 2024-09-10T21:59:39.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:39 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3540927275' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:39 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3540927275' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:39 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3540927275' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:40.037 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 21:59:39 smithi171 podman[64564]: 2024-09-10 21:59:39.770205705 +0000 UTC m=+0.145730465 container create a4454f43cfa3b2b24247eef599880154f5df09abc3ee6532b74bd4a0e56e5200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T21:59:40.037 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 21:59:40 smithi171 podman[64564]: 2024-09-10 21:59:40.03640502 +0000 UTC m=+0.411929782 container init a4454f43cfa3b2b24247eef599880154f5df09abc3ee6532b74bd4a0e56e5200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T21:59:40.497 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 21:59:40 smithi171 podman[64564]: 2024-09-10 21:59:40.043284581 +0000 UTC m=+0.418809343 container start a4454f43cfa3b2b24247eef599880154f5df09abc3ee6532b74bd4a0e56e5200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T21:59:40.498 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mds[64583]: -- 172.21.15.171:0/3557417381 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2eebb5860 con 0x55c2eeb93400 2024-09-10T21:59:40.498 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l[64579]: starting mds.l at 2024-09-10T21:59:40.498 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 21:59:40 smithi171 bash[64564]: a4454f43cfa3b2b24247eef599880154f5df09abc3ee6532b74bd4a0e56e5200 2024-09-10T21:59:40.498 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 21:59:40 smithi171 systemd[1]: Started Ceph mds.l for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:59:40.826 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:40.826 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.171:6832/3906453887,v1:172.21.15.171:6833/3906453887] up:boot 2024-09-10T21:59:40.826 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: fsmap 1 up:standby 2024-09-10T21:59:40.826 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T21:59:40.827 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:40.827 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:40.827 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:40.827 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:40.827 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:40.827 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:40 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.171:6832/3906453887,v1:172.21.15.171:6833/3906453887] up:boot 2024-09-10T21:59:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: fsmap 1 up:standby 2024-09-10T21:59:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T21:59:40.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:40.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:40.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:40.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:40.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:40.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:40 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.171:6832/3906453887,v1:172.21.15.171:6833/3906453887] up:boot 2024-09-10T21:59:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: fsmap 1 up:standby 2024-09-10T21:59:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T21:59:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:40.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:40.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:40.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:40 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:41.785 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:59:41.840 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:41 smithi090 ceph-mon[30350]: Deploying daemon mds.k on smithi154 2024-09-10T21:59:41.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:41 smithi154 ceph-mon[34495]: Deploying daemon mds.k on smithi154 2024-09-10T21:59:41.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:41 smithi171 ceph-mon[34138]: Deploying daemon mds.k on smithi154 2024-09-10T21:59:42.160 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:59:42.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.745+0000 7fc542e00640 1 -- 172.21.15.90:0/3045292427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c100d30 msgr2=0x7fc53c101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:42.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.745+0000 7fc542e00640 1 --2- 172.21.15.90:0/3045292427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c100d30 0x7fc53c101110 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009ca0 tx=0x7fc52c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.745+0000 7fc542e00640 1 -- 172.21.15.90:0/3045292427 shutdown_connections 2024-09-10T21:59:42.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.745+0000 7fc542e00640 1 --2- 172.21.15.90:0/3045292427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53c108030 0x7fc53c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.745+0000 7fc542e00640 1 --2- 172.21.15.90:0/3045292427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c1076b0 0x7fc53c107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.745+0000 7fc542e00640 1 --2- 172.21.15.90:0/3045292427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c100d30 0x7fc53c101110 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.745+0000 7fc542e00640 1 -- 172.21.15.90:0/3045292427 >> 172.21.15.90:0/3045292427 conn(0x7fc53c0fca20 msgr2=0x7fc53c0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:42.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.745+0000 7fc542e00640 1 -- 172.21.15.90:0/3045292427 shutdown_connections 2024-09-10T21:59:42.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.745+0000 7fc542e00640 1 -- 172.21.15.90:0/3045292427 wait complete. 2024-09-10T21:59:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.746+0000 7fc542e00640 1 Processor -- start 2024-09-10T21:59:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.746+0000 7fc542e00640 1 -- start start 2024-09-10T21:59:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.746+0000 7fc542e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c100d30 0x7fc53c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.746+0000 7fc542e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c1076b0 0x7fc53c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.746+0000 7fc542e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53c108030 0x7fc53c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.746+0000 7fc542e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c112ca0 con 0x7fc53c1076b0 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.746+0000 7fc542e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c112b20 con 0x7fc53c108030 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.746+0000 7fc542e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c112e20 con 0x7fc53c100d30 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c1076b0 0x7fc53c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc540e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53c108030 0x7fc53c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c1076b0 0x7fc53c19b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57186/0 (socket says 172.21.15.90:57186) 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53b400640 1 -- 172.21.15.90:0/2608363144 learned_addr learned my addr 172.21.15.90:0/2608363144 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc540e00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53c108030 0x7fc53c1a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:56046/0 (socket says 172.21.15.90:56046) 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53be00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c100d30 0x7fc53c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53b400640 1 -- 172.21.15.90:0/2608363144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c100d30 msgr2=0x7fc53c19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53b400640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c100d30 0x7fc53c19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53b400640 1 -- 172.21.15.90:0/2608363144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53c108030 msgr2=0x7fc53c1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53b400640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53c108030 0x7fc53c1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53b400640 1 -- 172.21.15.90:0/2608363144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c1a14d0 con 0x7fc53c1076b0 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53be00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c100d30 0x7fc53c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc540e00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53c108030 0x7fc53c1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.747+0000 7fc53b400640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c1076b0 0x7fc53c19b960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc52400d950 tx=0x7fc52400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.748+0000 7fc538c00640 1 -- 172.21.15.90:0/2608363144 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524017070 con 0x7fc53c1076b0 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.748+0000 7fc538c00640 1 -- 172.21.15.90:0/2608363144 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc524009c30 con 0x7fc53c1076b0 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.748+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c1a17c0 con 0x7fc53c1076b0 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.748+0000 7fc538c00640 1 -- 172.21.15.90:0/2608363144 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52400f620 con 0x7fc53c1076b0 2024-09-10T21:59:42.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.748+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c1a7e80 con 0x7fc53c1076b0 2024-09-10T21:59:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.749+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c101110 con 0x7fc53c1076b0 2024-09-10T21:59:42.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.751+0000 7fc538c00640 1 -- 172.21.15.90:0/2608363144 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fc524010460 con 0x7fc53c1076b0 2024-09-10T21:59:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.752+0000 7fc538c00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc5200b5330 0x7fc5200b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.753+0000 7fc538c00640 1 -- 172.21.15.90:0/2608363144 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fc5240e5540 con 0x7fc53c1076b0 2024-09-10T21:59:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.753+0000 7fc53be00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc5200b5330 0x7fc5200b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.753+0000 7fc53be00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc5200b5330 0x7fc5200b77f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000f80 tx=0x7fc52c005fa0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.756+0000 7fc538c00640 1 -- 172.21.15.90:0/2608363144 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc5240ac730 con 0x7fc53c1076b0 2024-09-10T21:59:42.843 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:42 smithi154 ceph-mon[34495]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:42.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:42 smithi090 ceph-mon[30350]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:42.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.979+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc53c1a81a0 con 0x7fc53c1076b0 2024-09-10T21:59:42.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.979+0000 7fc538c00640 1 -- 172.21.15.90:0/2608363144 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fc524009de0 con 0x7fc53c1076b0 2024-09-10T21:59:42.981 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:59:42.981 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":2,"btime":"2024-09-10T21:59:40:089353+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T21:59:42.981 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 2 2024-09-10T21:59:42.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.981+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc5200b5330 msgr2=0x7fc5200b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:42.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.981+0000 7fc542e00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc5200b5330 0x7fc5200b77f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000f80 tx=0x7fc52c005fa0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.981+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c1076b0 msgr2=0x7fc53c19b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:42.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.981+0000 7fc542e00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c1076b0 0x7fc53c19b960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc52400d950 tx=0x7fc52400de10 comp rx=0 tx=0).stop 2024-09-10T21:59:42.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.982+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 shutdown_connections 2024-09-10T21:59:42.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.982+0000 7fc542e00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc5200b5330 0x7fc5200b77f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.982+0000 7fc542e00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc53c108030 0x7fc53c1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.982+0000 7fc542e00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc53c1076b0 0x7fc53c19b960 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.982+0000 7fc542e00640 1 --2- 172.21.15.90:0/2608363144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc53c100d30 0x7fc53c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:42.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.982+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 >> 172.21.15.90:0/2608363144 conn(0x7fc53c0fca20 msgr2=0x7fc53c10dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:42.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.982+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 shutdown_connections 2024-09-10T21:59:42.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:42.982+0000 7fc542e00640 1 -- 172.21.15.90:0/2608363144 wait complete. 2024-09-10T21:59:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:42 smithi171 ceph-mon[34138]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:43.157 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 21:59:42 smithi154 systemd[1]: Starting Ceph mds.k for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T21:59:43.418 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 21:59:43 smithi154 podman[64673]: 2024-09-10 21:59:43.060126428 +0000 UTC m=+0.027341567 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:59:43.418 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 21:59:43 smithi154 podman[64673]: 2024-09-10 21:59:43.185573644 +0000 UTC m=+0.152788778 container create 5f9d0d7e4467a15ac0bf5f423728811f531fdb8ba7586dc55f39571469e6cdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-10T21:59:43.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:43 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2608363144' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:43.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:43 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2608363144' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:43.918 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 21:59:43 smithi154 podman[64673]: 2024-09-10 21:59:43.447206283 +0000 UTC m=+0.414421443 container init 5f9d0d7e4467a15ac0bf5f423728811f531fdb8ba7586dc55f39571469e6cdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T21:59:43.918 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 21:59:43 smithi154 podman[64673]: 2024-09-10 21:59:43.453362418 +0000 UTC m=+0.420577553 container start 5f9d0d7e4467a15ac0bf5f423728811f531fdb8ba7586dc55f39571469e6cdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T21:59:43.918 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 21:59:43 smithi154 ceph-mds[64692]: -- 172.21.15.154:0/2103885187 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561e43259860 con 0x561e43237000 2024-09-10T21:59:43.918 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 21:59:43 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k[64688]: starting mds.k at 2024-09-10T21:59:43.919 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 21:59:43 smithi154 bash[64673]: 5f9d0d7e4467a15ac0bf5f423728811f531fdb8ba7586dc55f39571469e6cdc4 2024-09-10T21:59:43.919 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 21:59:43 smithi154 systemd[1]: Started Ceph mds.k for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:59:43.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:43 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2608363144' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] up:boot 2024-09-10T21:59:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: fsmap 2 up:standby 2024-09-10T21:59:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T21:59:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:44.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:44.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:44.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:44.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:44.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:44 smithi090 ceph-mon[30350]: Deploying daemon mds.j on smithi090 2024-09-10T21:59:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] up:boot 2024-09-10T21:59:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: fsmap 2 up:standby 2024-09-10T21:59:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T21:59:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:44.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:44.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:44.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:44.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:44.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:44 smithi154 ceph-mon[34495]: Deploying daemon mds.j on smithi090 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] up:boot 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: fsmap 2 up:standby 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:44.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:44 smithi171 ceph-mon[34138]: Deploying daemon mds.j on smithi090 2024-09-10T21:59:45.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:59:46.219 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:59:46.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:46 smithi090 ceph-mon[30350]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:46.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.815+0000 7fe09c800640 1 -- 172.21.15.90:0/1088172192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 msgr2=0x7fe09410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:46.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.815+0000 7fe09c800640 1 --2- 172.21.15.90:0/1088172192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 0x7fe09410f550 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe090009ca0 tx=0x7fe09002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:59:46.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.815+0000 7fe09c800640 1 -- 172.21.15.90:0/1088172192 shutdown_connections 2024-09-10T21:59:46.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.815+0000 7fe09c800640 1 --2- 172.21.15.90:0/1088172192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 0x7fe09410f550 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:46.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.815+0000 7fe09c800640 1 --2- 172.21.15.90:0/1088172192 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0941045b0 0x7fe09410caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:46.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.815+0000 7fe09c800640 1 --2- 172.21.15.90:0/1088172192 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe094103c90 0x7fe094104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:46.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.815+0000 7fe09c800640 1 -- 172.21.15.90:0/1088172192 >> 172.21.15.90:0/1088172192 conn(0x7fe0940fd710 msgr2=0x7fe0940ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:46.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.815+0000 7fe09c800640 1 -- 172.21.15.90:0/1088172192 shutdown_connections 2024-09-10T21:59:46.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.815+0000 7fe09c800640 1 -- 172.21.15.90:0/1088172192 wait complete. 2024-09-10T21:59:46.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.816+0000 7fe09c800640 1 Processor -- start 2024-09-10T21:59:46.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.816+0000 7fe09c800640 1 -- start start 2024-09-10T21:59:46.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe09c800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe094103c90 0x7fe09419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:46.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe09c800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0941045b0 0x7fe09419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:46.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe09c800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 0x7fe0941a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:46.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe09c800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe094112c60 con 0x7fe09410d0f0 2024-09-10T21:59:46.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe09c800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe094112ae0 con 0x7fe094103c90 2024-09-10T21:59:46.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe09c800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe094112de0 con 0x7fe0941045b0 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe09a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 0x7fe0941a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe09a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 0x7fe0941a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57202/0 (socket says 172.21.15.90:57202) 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe09a800640 1 -- 172.21.15.90:0/3018837447 learned_addr learned my addr 172.21.15.90:0/3018837447 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe099400640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0941045b0 0x7fe09419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.817+0000 7fe099e00640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe094103c90 0x7fe09419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe09a800640 1 -- 172.21.15.90:0/3018837447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0941045b0 msgr2=0x7fe09419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe09a800640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0941045b0 0x7fe09419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe09a800640 1 -- 172.21.15.90:0/3018837447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe094103c90 msgr2=0x7fe09419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe09a800640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe094103c90 0x7fe09419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe09a800640 1 -- 172.21.15.90:0/3018837447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0941a1420 con 0x7fe09410d0f0 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe099e00640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe094103c90 0x7fe09419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe099400640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0941045b0 0x7fe09419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe09a800640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 0x7fe0941a0c50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe090005e30 tx=0x7fe090003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:46.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe08ea00640 1 -- 172.21.15.90:0/3018837447 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090032c10 con 0x7fe09410d0f0 2024-09-10T21:59:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0941a16b0 con 0x7fe09410d0f0 2024-09-10T21:59:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe08ea00640 1 -- 172.21.15.90:0/3018837447 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe090032db0 con 0x7fe09410d0f0 2024-09-10T21:59:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe08ea00640 1 -- 172.21.15.90:0/3018837447 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090030ab0 con 0x7fe09410d0f0 2024-09-10T21:59:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.818+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0941a7dd0 con 0x7fe09410d0f0 2024-09-10T21:59:46.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.819+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe094104070 con 0x7fe09410d0f0 2024-09-10T21:59:46.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.822+0000 7fe08ea00640 1 -- 172.21.15.90:0/3018837447 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fe090030c50 con 0x7fe09410d0f0 2024-09-10T21:59:46.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.823+0000 7fe08ea00640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe07c0b5330 0x7fe07c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.823+0000 7fe08ea00640 1 -- 172.21.15.90:0/3018837447 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fe090106b20 con 0x7fe09410d0f0 2024-09-10T21:59:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.823+0000 7fe099e00640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe07c0b5330 0x7fe07c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.824+0000 7fe099e00640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe07c0b5330 0x7fe07c0b77f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe088009c20 tx=0x7fe088005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:46.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:46.825+0000 7fe08ea00640 1 -- 172.21.15.90:0/3018837447 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe0900cdda0 con 0x7fe09410d0f0 2024-09-10T21:59:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:46 smithi154 ceph-mon[34495]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:46.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:46 smithi171 ceph-mon[34138]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:47.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.057+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0941a80b0 con 0x7fe09410d0f0 2024-09-10T21:59:47.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.058+0000 7fe08ea00640 1 -- 172.21.15.90:0/3018837447 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7fe0900cd6a0 con 0x7fe09410d0f0 2024-09-10T21:59:47.059 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:59:47.059 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":3,"btime":"2024-09-10T21:59:43:550260+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T21:59:47.059 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 3 2024-09-10T21:59:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.060+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe07c0b5330 msgr2=0x7fe07c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.060+0000 7fe09c800640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe07c0b5330 0x7fe07c0b77f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe088009c20 tx=0x7fe088005f90 comp rx=0 tx=0).stop 2024-09-10T21:59:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.060+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 msgr2=0x7fe0941a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.060+0000 7fe09c800640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 0x7fe0941a0c50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe090005e30 tx=0x7fe090003f30 comp rx=0 tx=0).stop 2024-09-10T21:59:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.060+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 shutdown_connections 2024-09-10T21:59:47.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.061+0000 7fe09c800640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe07c0b5330 0x7fe07c0b77f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:47.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.061+0000 7fe09c800640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe09410d0f0 0x7fe0941a0c50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:47.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.061+0000 7fe09c800640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe0941045b0 0x7fe09419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:47.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.061+0000 7fe09c800640 1 --2- 172.21.15.90:0/3018837447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe094103c90 0x7fe09419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:47.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.061+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 >> 172.21.15.90:0/3018837447 conn(0x7fe0940fd710 msgr2=0x7fe09410f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:47.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.061+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 shutdown_connections 2024-09-10T21:59:47.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:47.061+0000 7fe09c800640 1 -- 172.21.15.90:0/3018837447 wait complete. 2024-09-10T21:59:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:47 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3018837447' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:47 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3018837447' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:48.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:47 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3018837447' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:48 smithi090 ceph-mon[30350]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:48 smithi154 ceph-mon[34495]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:48.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:48 smithi171 ceph-mon[34138]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:49 smithi090 ceph-mon[30350]: Deploying daemon mds.d on smithi090 2024-09-10T21:59:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.90:6834/1514545418,v1:172.21.15.90:6835/1514545418] up:boot 2024-09-10T21:59:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:49 smithi090 ceph-mon[30350]: fsmap 3 up:standby 2024-09-10T21:59:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T21:59:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:49 smithi154 ceph-mon[34495]: Deploying daemon mds.d on smithi090 2024-09-10T21:59:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.90:6834/1514545418,v1:172.21.15.90:6835/1514545418] up:boot 2024-09-10T21:59:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:49 smithi154 ceph-mon[34495]: fsmap 3 up:standby 2024-09-10T21:59:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T21:59:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:49 smithi171 ceph-mon[34138]: Deploying daemon mds.d on smithi090 2024-09-10T21:59:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.90:6834/1514545418,v1:172.21.15.90:6835/1514545418] up:boot 2024-09-10T21:59:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:49 smithi171 ceph-mon[34138]: fsmap 3 up:standby 2024-09-10T21:59:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T21:59:50.655 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:59:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:50 smithi154 ceph-mon[34495]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:50.972 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:50 smithi090 ceph-mon[30350]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:50.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:50 smithi171 ceph-mon[34138]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:51.042 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:59:51.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.798+0000 7fd8b5000640 1 -- 172.21.15.90:0/3028500194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b010ede0 msgr2=0x7fd8b010f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:51.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.798+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3028500194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b010ede0 0x7fd8b010f220 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009900 tx=0x7fd89c02a410 comp rx=0 tx=0).stop 2024-09-10T21:59:51.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.798+0000 7fd8b5000640 1 -- 172.21.15.90:0/3028500194 shutdown_connections 2024-09-10T21:59:51.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.798+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3028500194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8b010f760 0x7fd8b0115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.798+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3028500194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b010ede0 0x7fd8b010f220 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.798+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3028500194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8b01093d0 0x7fd8b01097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.798+0000 7fd8b5000640 1 -- 172.21.15.90:0/3028500194 >> 172.21.15.90:0/3028500194 conn(0x7fd8b0078a40 msgr2=0x7fd8b007ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.798+0000 7fd8b5000640 1 -- 172.21.15.90:0/3028500194 shutdown_connections 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.798+0000 7fd8b5000640 1 -- 172.21.15.90:0/3028500194 wait complete. 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.799+0000 7fd8b5000640 1 Processor -- start 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.799+0000 7fd8b5000640 1 -- start start 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.799+0000 7fd8b5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b01093d0 0x7fd8b019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.799+0000 7fd8b5000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8b010ede0 0x7fd8b019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.799+0000 7fd8b5000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8b010f760 0x7fd8b01a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.799+0000 7fd8b5000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b01192e0 con 0x7fd8b01093d0 2024-09-10T21:59:51.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.799+0000 7fd8b5000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0119160 con 0x7fd8b010ede0 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.799+0000 7fd8b5000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0119460 con 0x7fd8b010f760 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ade00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b01093d0 0x7fd8b019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ade00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b01093d0 0x7fd8b019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52554/0 (socket says 172.21.15.90:52554) 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ade00640 1 -- 172.21.15.90:0/3362644528 learned_addr learned my addr 172.21.15.90:0/3362644528 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ad400640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8b010ede0 0x7fd8b019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ae800640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8b010f760 0x7fd8b01a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ade00640 1 -- 172.21.15.90:0/3362644528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8b010f760 msgr2=0x7fd8b01a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ade00640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8b010f760 0x7fd8b01a51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ade00640 1 -- 172.21.15.90:0/3362644528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8b010ede0 msgr2=0x7fd8b019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ade00640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8b010ede0 0x7fd8b019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ade00640 1 -- 172.21.15.90:0/3362644528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b01a59a0 con 0x7fd8b01093d0 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ad400640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8b010ede0 0x7fd8b019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:59:51.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ae800640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8b010f760 0x7fd8b01a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.800+0000 7fd8ade00640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b01093d0 0x7fd8b019f8f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000ba50 tx=0x7fd8a000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.801+0000 7fd8a6a00640 1 -- 172.21.15.90:0/3362644528 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0019070 con 0x7fd8b01093d0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.801+0000 7fd8a6a00640 1 -- 172.21.15.90:0/3362644528 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd8a000cbe0 con 0x7fd8b01093d0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.801+0000 7fd8a6a00640 1 -- 172.21.15.90:0/3362644528 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00075d0 con 0x7fd8b01093d0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.801+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b01a5c90 con 0x7fd8b01093d0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.801+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b01ac350 con 0x7fd8b01093d0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.802+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b01097b0 con 0x7fd8b01093d0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.804+0000 7fd8a6a00640 1 -- 172.21.15.90:0/3362644528 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fd8a0014440 con 0x7fd8b01093d0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.805+0000 7fd8a6a00640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8940b53b0 0x7fd8940b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.806+0000 7fd8a6a00640 1 -- 172.21.15.90:0/3362644528 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fd8a00e5280 con 0x7fd8b01093d0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.806+0000 7fd8ad400640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8940b53b0 0x7fd8940b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.806+0000 7fd8ad400640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8940b53b0 0x7fd8940b7870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd89c006010 tx=0x7fd89c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:51.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:51.809+0000 7fd8a6a00640 1 -- 172.21.15.90:0/3362644528 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd8a0053220 con 0x7fd8b01093d0 2024-09-10T21:59:52.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.036+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8b01a0a20 con 0x7fd8b01093d0 2024-09-10T21:59:52.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.036+0000 7fd8a6a00640 1 -- 172.21.15.90:0/3362644528 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7fd8a00ac470 con 0x7fd8b01093d0 2024-09-10T21:59:52.038 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:59:52.038 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":4,"btime":"2024-09-10T21:59:48:629217+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T21:59:52.038 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 4 2024-09-10T21:59:52.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8940b53b0 msgr2=0x7fd8940b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:52.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8940b53b0 0x7fd8940b7870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd89c006010 tx=0x7fd89c005a20 comp rx=0 tx=0).stop 2024-09-10T21:59:52.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b01093d0 msgr2=0x7fd8b019f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:52.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b01093d0 0x7fd8b019f8f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000ba50 tx=0x7fd8a000bf10 comp rx=0 tx=0).stop 2024-09-10T21:59:52.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 shutdown_connections 2024-09-10T21:59:52.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8940b53b0 0x7fd8940b7870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:52.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd8b010f760 0x7fd8b01a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:52.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd8b010ede0 0x7fd8b019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:52.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 --2- 172.21.15.90:0/3362644528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b01093d0 0x7fd8b019f8f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:52.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 >> 172.21.15.90:0/3362644528 conn(0x7fd8b0078a40 msgr2=0x7fd8b007ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:52.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 shutdown_connections 2024-09-10T21:59:52.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:52.039+0000 7fd8b5000640 1 -- 172.21.15.90:0/3362644528 wait complete. 2024-09-10T21:59:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:52 smithi154 ceph-mon[34495]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:52 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3362644528' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:52 smithi171 ceph-mon[34138]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:52.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:52 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3362644528' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:52 smithi090 ceph-mon[30350]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:52 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3362644528' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:53.857 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 21:59:53 smithi090 podman[74544]: 2024-09-10 21:59:53.53233147 +0000 UTC m=+0.027575128 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:59:54.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 21:59:53 smithi090 podman[74544]: 2024-09-10 21:59:53.865193123 +0000 UTC m=+0.360436777 container create 1e770cf4225f6cc0a148029703288ba335b45d90c4296024981d89b94df99729 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:59:54.480 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 21:59:54 smithi090 podman[74544]: 2024-09-10 21:59:54.128464 +0000 UTC m=+0.623707655 container init 1e770cf4225f6cc0a148029703288ba335b45d90c4296024981d89b94df99729 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:59:54.480 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 21:59:54 smithi090 podman[74544]: 2024-09-10 21:59:54.135958348 +0000 UTC m=+0.631202012 container start 1e770cf4225f6cc0a148029703288ba335b45d90c4296024981d89b94df99729 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:59:54.481 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 21:59:54 smithi090 ceph-mds[74563]: -- 172.21.15.90:0/1451958058 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607fa9c7860 con 0x5607fa9a4800 2024-09-10T21:59:54.481 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 21:59:54 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d[74559]: starting mds.d at 2024-09-10T21:59:54.481 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 21:59:54 smithi090 bash[74544]: 1e770cf4225f6cc0a148029703288ba335b45d90c4296024981d89b94df99729 2024-09-10T21:59:54.481 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 21:59:54 smithi090 systemd[1]: Started Ceph mds.d for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:59:54.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:54 smithi090 ceph-mon[30350]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:54.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:54.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:54.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:54.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:54.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:54.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:54 smithi154 ceph-mon[34495]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:54.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:54.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:54.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:54.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:54.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:54 smithi171 ceph-mon[34138]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:55.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:55.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:55.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:55.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:55.001 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:55.435 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:59:55.815 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:59:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:55 smithi090 ceph-mon[30350]: Deploying daemon mds.e on smithi154 2024-09-10T21:59:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:55 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.90:6836/4068804869,v1:172.21.15.90:6837/4068804869] up:boot 2024-09-10T21:59:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:55 smithi090 ceph-mon[30350]: fsmap 4 up:standby 2024-09-10T21:59:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T21:59:55.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:55 smithi171 ceph-mon[34138]: Deploying daemon mds.e on smithi154 2024-09-10T21:59:55.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:55 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.90:6836/4068804869,v1:172.21.15.90:6837/4068804869] up:boot 2024-09-10T21:59:55.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:55 smithi171 ceph-mon[34138]: fsmap 4 up:standby 2024-09-10T21:59:55.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T21:59:56.035 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:55 smithi154 ceph-mon[34495]: Deploying daemon mds.e on smithi154 2024-09-10T21:59:56.036 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:55 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.90:6836/4068804869,v1:172.21.15.90:6837/4068804869] up:boot 2024-09-10T21:59:56.036 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:55 smithi154 ceph-mon[34495]: fsmap 4 up:standby 2024-09-10T21:59:56.036 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T21:59:56.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.619+0000 7ff409000640 1 -- 172.21.15.90:0/2219914046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff404108fa0 msgr2=0x7ff40410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:56.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.619+0000 7ff409000640 1 --2- 172.21.15.90:0/2219914046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff404108fa0 0x7ff40410f7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009ca0 tx=0x7ff3fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.619+0000 7ff409000640 1 -- 172.21.15.90:0/2219914046 shutdown_connections 2024-09-10T21:59:56.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.619+0000 7ff409000640 1 --2- 172.21.15.90:0/2219914046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff404108fa0 0x7ff40410f7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.619+0000 7ff409000640 1 --2- 172.21.15.90:0/2219914046 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff404108620 0x7ff404108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.619+0000 7ff409000640 1 --2- 172.21.15.90:0/2219914046 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404102c10 0x7ff404102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.619+0000 7ff409000640 1 -- 172.21.15.90:0/2219914046 >> 172.21.15.90:0/2219914046 conn(0x7ff4040fe660 msgr2=0x7ff404100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.619+0000 7ff409000640 1 -- 172.21.15.90:0/2219914046 shutdown_connections 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.619+0000 7ff409000640 1 -- 172.21.15.90:0/2219914046 wait complete. 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.620+0000 7ff409000640 1 Processor -- start 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.620+0000 7ff409000640 1 -- start start 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.620+0000 7ff409000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff404102c10 0x7ff40419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.620+0000 7ff409000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404108620 0x7ff40419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.620+0000 7ff409000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff404108fa0 0x7ff4041a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.620+0000 7ff409000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff404112b20 con 0x7ff404108fa0 2024-09-10T21:59:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.620+0000 7ff409000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041129a0 con 0x7ff404102c10 2024-09-10T21:59:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff409000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff404112ca0 con 0x7ff404108620 2024-09-10T21:59:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff404102c10 0x7ff40419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff404108fa0 0x7ff4041a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff404102c10 0x7ff40419b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:33856/0 (socket says 172.21.15.90:33856) 2024-09-10T21:59:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403400640 1 -- 172.21.15.90:0/847472867 learned_addr learned my addr 172.21.15.90:0/847472867 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T21:59:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff402a00640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404108620 0x7ff40419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403400640 1 -- 172.21.15.90:0/847472867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404108620 msgr2=0x7ff40419b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403400640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404108620 0x7ff40419b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403400640 1 -- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff404108fa0 msgr2=0x7ff4041a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403400640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff404108fa0 0x7ff4041a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403400640 1 -- 172.21.15.90:0/847472867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4041a1180 con 0x7ff404102c10 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.621+0000 7ff403e00640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff404108fa0 0x7ff4041a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.622+0000 7ff403400640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff404102c10 0x7ff40419b0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d950 tx=0x7ff3f400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.622+0000 7ff3fbe00640 1 -- 172.21.15.90:0/847472867 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4017070 con 0x7ff404102c10 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.622+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4041a1470 con 0x7ff404102c10 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.622+0000 7ff3fbe00640 1 -- 172.21.15.90:0/847472867 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff3f4009c30 con 0x7ff404102c10 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.622+0000 7ff3fbe00640 1 -- 172.21.15.90:0/847472867 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f400f620 con 0x7ff404102c10 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.622+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041a7d00 con 0x7ff404102c10 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.623+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff404102ff0 con 0x7ff404102c10 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.625+0000 7ff3fbe00640 1 -- 172.21.15.90:0/847472867 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7ff3f4010460 con 0x7ff404102c10 2024-09-10T21:59:56.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.627+0000 7ff3fbe00640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff3e80b53e0 0x7ff3e80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:59:56.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.627+0000 7ff3fbe00640 1 -- 172.21.15.90:0/847472867 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7ff3f40e5490 con 0x7ff404102c10 2024-09-10T21:59:56.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.627+0000 7ff402a00640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff3e80b53e0 0x7ff3e80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:59:56.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.628+0000 7ff402a00640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff3e80b53e0 0x7ff3e80b78a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff404077450 tx=0x7ff3f0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:59:56.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.633+0000 7ff3fbe00640 1 -- 172.21.15.90:0/847472867 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff3f4009de0 con 0x7ff404102c10 2024-09-10T21:59:56.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.853+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4041a7ed0 con 0x7ff404102c10 2024-09-10T21:59:56.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.854+0000 7ff3fbe00640 1 -- 172.21.15.90:0/847472867 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3916 (secure 0 0 0) 0x7ff3f40ac030 con 0x7ff404102c10 2024-09-10T21:59:56.856 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T21:59:56.856 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":5,"btime":"2024-09-10T21:59:54:658550+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T21:59:56.857 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 5 2024-09-10T21:59:56.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff3e80b53e0 msgr2=0x7ff3e80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:56.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff3e80b53e0 0x7ff3e80b78a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff404077450 tx=0x7ff3f0005f90 comp rx=0 tx=0).stop 2024-09-10T21:59:56.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff404102c10 msgr2=0x7ff40419b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:59:56.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff404102c10 0x7ff40419b0d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d950 tx=0x7ff3f400de10 comp rx=0 tx=0).stop 2024-09-10T21:59:56.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 shutdown_connections 2024-09-10T21:59:56.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff3e80b53e0 0x7ff3e80b78a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff404108fa0 0x7ff4041a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff404108620 0x7ff40419b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 --2- 172.21.15.90:0/847472867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff404102c10 0x7ff40419b0d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:59:56.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.856+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 >> 172.21.15.90:0/847472867 conn(0x7ff4040fe660 msgr2=0x7ff404075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:59:56.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.857+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 shutdown_connections 2024-09-10T21:59:56.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T21:59:56.857+0000 7ff409000640 1 -- 172.21.15.90:0/847472867 wait complete. 2024-09-10T21:59:56.860 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:56 smithi154 ceph-mon[34495]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:56.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:56 smithi090 ceph-mon[30350]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:56.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:56 smithi171 ceph-mon[34138]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:57.168 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 21:59:56 smithi154 systemd[1]: Starting Ceph mds.e for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T21:59:57.486 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 21:59:57 smithi154 podman[64974]: 2024-09-10 21:59:57.079492796 +0000 UTC m=+0.027791434 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:59:57.486 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 21:59:57 smithi154 podman[64974]: 2024-09-10 21:59:57.215804033 +0000 UTC m=+0.164102658 container create a78c6981b572579b67c94b054e965d2d88b2b6d9801f5022ed66ba20e84b80b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T21:59:57.487 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 21:59:57 smithi154 podman[64974]: 2024-09-10 21:59:57.485892483 +0000 UTC m=+0.434191118 container init a78c6981b572579b67c94b054e965d2d88b2b6d9801f5022ed66ba20e84b80b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2) 2024-09-10T21:59:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:57 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/847472867' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:57.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:57 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:57.918 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 21:59:57 smithi154 podman[64974]: 2024-09-10 21:59:57.492424157 +0000 UTC m=+0.440722784 container start a78c6981b572579b67c94b054e965d2d88b2b6d9801f5022ed66ba20e84b80b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T21:59:57.918 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 21:59:57 smithi154 ceph-mds[64994]: -- 172.21.15.154:0/2890717384 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fac0ce9860 con 0x55fac0cc7000 2024-09-10T21:59:57.919 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 21:59:57 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e[64990]: starting mds.e at 2024-09-10T21:59:57.919 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 21:59:57 smithi154 bash[64974]: a78c6981b572579b67c94b054e965d2d88b2b6d9801f5022ed66ba20e84b80b1 2024-09-10T21:59:57.919 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 21:59:57 smithi154 systemd[1]: Started Ceph mds.e for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T21:59:57.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:57 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/847472867' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:57.967 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:57.967 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:57 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:57.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:57 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/847472867' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:59:57.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:57.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:57 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:58.982 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:58 smithi090 ceph-mon[30350]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:58.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:58.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:58 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] up:boot 2024-09-10T21:59:58.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:58 smithi090 ceph-mon[30350]: fsmap 5 up:standby 2024-09-10T21:59:58.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T21:59:58.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:58.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:58.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:58.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 21:59:58 smithi090 ceph-mon[30350]: Deploying daemon mds.f on smithi171 2024-09-10T21:59:58.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:58 smithi171 ceph-mon[34138]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:58.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:58.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:58 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] up:boot 2024-09-10T21:59:58.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:58 smithi171 ceph-mon[34138]: fsmap 5 up:standby 2024-09-10T21:59:58.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T21:59:58.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:58.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:58.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:58.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 21:59:58 smithi171 ceph-mon[34138]: Deploying daemon mds.f on smithi171 2024-09-10T21:59:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:58 smithi154 ceph-mon[34495]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:59:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T21:59:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:58 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] up:boot 2024-09-10T21:59:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:58 smithi154 ceph-mon[34495]: fsmap 5 up:standby 2024-09-10T21:59:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T21:59:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:59:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:59:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:59:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 21:59:58 smithi154 ceph-mon[34495]: Deploying daemon mds.f on smithi171 2024-09-10T21:59:59.970 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:00:00.349 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:00.839 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:00 smithi090 ceph-mon[30350]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:00.840 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:00 smithi090 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T22:00:00.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:00 smithi171 ceph-mon[34138]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:00.867 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:00 smithi171 ceph-mon[34138]: overall HEALTH_OK 2024-09-10T22:00:01.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.949+0000 7fca28400640 1 -- 172.21.15.90:0/3606405138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 msgr2=0x7fca2010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:01.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.949+0000 7fca28400640 1 --2- 172.21.15.90:0/3606405138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 0x7fca2010ae10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009900 tx=0x7fca0c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.949+0000 7fca28400640 1 -- 172.21.15.90:0/3606405138 shutdown_connections 2024-09-10T22:00:01.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.949+0000 7fca28400640 1 --2- 172.21.15.90:0/3606405138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2010b350 0x7fca20111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.949+0000 7fca28400640 1 --2- 172.21.15.90:0/3606405138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 0x7fca2010ae10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.949+0000 7fca28400640 1 --2- 172.21.15.90:0/3606405138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca20104fc0 0x7fca201053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.949+0000 7fca28400640 1 -- 172.21.15.90:0/3606405138 >> 172.21.15.90:0/3606405138 conn(0x7fca201009f0 msgr2=0x7fca20102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.949+0000 7fca28400640 1 -- 172.21.15.90:0/3606405138 shutdown_connections 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.949+0000 7fca28400640 1 -- 172.21.15.90:0/3606405138 wait complete. 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.950+0000 7fca28400640 1 Processor -- start 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.950+0000 7fca28400640 1 -- start start 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.950+0000 7fca28400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca20104fc0 0x7fca2019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca28400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 0x7fca2019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca28400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2010b350 0x7fca201a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca28400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca20114e70 con 0x7fca2010a9d0 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca28400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca20114cf0 con 0x7fca2010b350 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca28400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca20114ff0 con 0x7fca20104fc0 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca25000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 0x7fca2019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:01.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca25000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 0x7fca2019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48494/0 (socket says 172.21.15.90:48494) 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca25a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca20104fc0 0x7fca2019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca25000640 1 -- 172.21.15.90:0/1357085638 learned_addr learned my addr 172.21.15.90:0/1357085638 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca26400640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2010b350 0x7fca201a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca25000640 1 -- 172.21.15.90:0/1357085638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca20104fc0 msgr2=0x7fca2019b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca25000640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca20104fc0 0x7fca2019b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca25000640 1 -- 172.21.15.90:0/1357085638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2010b350 msgr2=0x7fca201a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca25000640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2010b350 0x7fca201a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca25000640 1 -- 172.21.15.90:0/1357085638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca201a1470 con 0x7fca2010a9d0 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.951+0000 7fca26400640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2010b350 0x7fca201a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.952+0000 7fca25000640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 0x7fca2019b900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fca0c02a910 tx=0x7fca0c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.952+0000 7fca1e000640 1 -- 172.21.15.90:0/1357085638 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c03f070 con 0x7fca2010a9d0 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.952+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca201a1700 con 0x7fca2010a9d0 2024-09-10T22:00:01.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.952+0000 7fca1e000640 1 -- 172.21.15.90:0/1357085638 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca0c003e40 con 0x7fca2010a9d0 2024-09-10T22:00:01.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.952+0000 7fca1e000640 1 -- 172.21.15.90:0/1357085638 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c035030 con 0x7fca2010a9d0 2024-09-10T22:00:01.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.952+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca201a7e20 con 0x7fca2010a9d0 2024-09-10T22:00:01.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.953+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca201053a0 con 0x7fca2010a9d0 2024-09-10T22:00:01.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.955+0000 7fca1e000640 1 -- 172.21.15.90:0/1357085638 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fca0c031400 con 0x7fca2010a9d0 2024-09-10T22:00:01.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.956+0000 7fca1e000640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca080b5330 0x7fca080b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:01.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.957+0000 7fca1e000640 1 -- 172.21.15.90:0/1357085638 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fca0c1067a0 con 0x7fca2010a9d0 2024-09-10T22:00:01.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.957+0000 7fca25a00640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca080b5330 0x7fca080b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:01.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.957+0000 7fca25a00640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca080b5330 0x7fca080b77f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fca14005fd0 tx=0x7fca1400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:01.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:00.959+0000 7fca1e000640 1 -- 172.21.15.90:0/1357085638 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fca0c0cd990 con 0x7fca2010a9d0 2024-09-10T22:00:01.609 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:00:00 smithi171 systemd[1]: Started Ceph mds.f for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T22:00:01.609 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:00 smithi154 ceph-mon[34495]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:01.609 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:00 smithi154 ceph-mon[34495]: overall HEALTH_OK 2024-09-10T22:00:01.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.185+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca2019c4f0 con 0x7fca2010a9d0 2024-09-10T22:00:01.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.186+0000 7fca1e000640 1 -- 172.21.15.90:0/1357085638 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7fca2019c4f0 con 0x7fca2010a9d0 2024-09-10T22:00:01.610 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:01.610 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":6,"btime":"2024-09-10T21:59:57:676387+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34640,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T22:00:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 6 2024-09-10T22:00:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca080b5330 msgr2=0x7fca080b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca080b5330 0x7fca080b77f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fca14005fd0 tx=0x7fca1400b390 comp rx=0 tx=0).stop 2024-09-10T22:00:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 msgr2=0x7fca2019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 0x7fca2019b900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fca0c02a910 tx=0x7fca0c003c20 comp rx=0 tx=0).stop 2024-09-10T22:00:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 shutdown_connections 2024-09-10T22:00:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca080b5330 0x7fca080b77f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca2010b350 0x7fca201a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca2010a9d0 0x7fca2019b900 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 --2- 172.21.15.90:0/1357085638 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca20104fc0 0x7fca2019b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 >> 172.21.15.90:0/1357085638 conn(0x7fca201009f0 msgr2=0x7fca20102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 shutdown_connections 2024-09-10T22:00:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:01.188+0000 7fca28400640 1 -- 172.21.15.90:0/1357085638 wait complete. 2024-09-10T22:00:02.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: Deploying daemon mds.c on smithi171 2024-09-10T22:00:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1357085638' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:02.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] up:boot 2024-09-10T22:00:02.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: fsmap 6 up:standby 2024-09-10T22:00:02.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:00:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: Deploying daemon mds.c on smithi171 2024-09-10T22:00:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1357085638' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] up:boot 2024-09-10T22:00:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: fsmap 6 up:standby 2024-09-10T22:00:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:00:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: Deploying daemon mds.c on smithi171 2024-09-10T22:00:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1357085638' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:02.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:02.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] up:boot 2024-09-10T22:00:02.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: fsmap 6 up:standby 2024-09-10T22:00:02.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:00:03.819 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:00:03 smithi171 systemd[1]: Starting Ceph mds.c for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:00:04.138 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:00:03 smithi171 podman[65175]: 2024-09-10 22:00:03.721957409 +0000 UTC m=+0.027502456 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:00:04.138 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:00:03 smithi171 podman[65175]: 2024-09-10 22:00:03.850978996 +0000 UTC m=+0.156524040 container create 9a12e81784aee4aa27a91d478c54be1050f9dd2a5792b32b819eaf3f4ee3f27a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T22:00:04.138 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:00:04 smithi171 podman[65175]: 2024-09-10 22:00:04.137370105 +0000 UTC m=+0.442915152 container init 9a12e81784aee4aa27a91d478c54be1050f9dd2a5792b32b819eaf3f4ee3f27a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:00:04.200 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:00:04.497 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:00:04 smithi171 podman[65175]: 2024-09-10 22:00:04.144554947 +0000 UTC m=+0.450099997 container start 9a12e81784aee4aa27a91d478c54be1050f9dd2a5792b32b819eaf3f4ee3f27a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:00:04.498 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:00:04 smithi171 ceph-mds[65193]: -- 172.21.15.171:0/1027448017 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56143c65f860 con 0x56143c63c800 2024-09-10T22:00:04.498 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:00:04 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c[65189]: starting mds.c at 2024-09-10T22:00:04.498 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:00:04 smithi171 bash[65175]: 9a12e81784aee4aa27a91d478c54be1050f9dd2a5792b32b819eaf3f4ee3f27a 2024-09-10T22:00:04.498 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:00:04 smithi171 systemd[1]: Started Ceph mds.c for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T22:00:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:04 smithi154 ceph-mon[34495]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:04 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:04 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:04 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:04 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:04 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:04 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:04.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:04 smithi090 ceph-mon[30350]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:04.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:04 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:04.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:04 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:04.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:04 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:04.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:04 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:04.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:04 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:04.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:04 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:04 smithi171 ceph-mon[34138]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:04 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:04 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:04 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:04 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:04 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:04 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.181+0000 7fa0db000640 1 -- 172.21.15.90:0/3678236535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 msgr2=0x7fa0d410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.181+0000 7fa0db000640 1 --2- 172.21.15.90:0/3678236535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 0x7fa0d410f950 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009ca0 tx=0x7fa0cc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.181+0000 7fa0db000640 1 -- 172.21.15.90:0/3678236535 shutdown_connections 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.181+0000 7fa0db000640 1 --2- 172.21.15.90:0/3678236535 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 0x7fa0d410f950 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.181+0000 7fa0db000640 1 --2- 172.21.15.90:0/3678236535 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0d41087d0 0x7fa0d4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.181+0000 7fa0db000640 1 --2- 172.21.15.90:0/3678236535 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d4102dc0 0x7fa0d41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.181+0000 7fa0db000640 1 -- 172.21.15.90:0/3678236535 >> 172.21.15.90:0/3678236535 conn(0x7fa0d40fe830 msgr2=0x7fa0d4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.181+0000 7fa0db000640 1 -- 172.21.15.90:0/3678236535 shutdown_connections 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.182+0000 7fa0db000640 1 -- 172.21.15.90:0/3678236535 wait complete. 2024-09-10T22:00:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.182+0000 7fa0db000640 1 Processor -- start 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.182+0000 7fa0db000640 1 -- start start 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0db000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d4102dc0 0x7fa0d4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0db000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0d41087d0 0x7fa0d41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0db000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 0x7fa0d419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0db000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4112d10 con 0x7fa0d4109150 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0db000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4112b90 con 0x7fa0d41087d0 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0db000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4112e90 con 0x7fa0d4102dc0 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0d9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 0x7fa0d419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0d9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 0x7fa0d419ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48512/0 (socket says 172.21.15.90:48512) 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0d9000640 1 -- 172.21.15.90:0/200521372 learned_addr learned my addr 172.21.15.90:0/200521372 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0cbe00640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0d41087d0 0x7fa0d41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0d3e00640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d4102dc0 0x7fa0d4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:05.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0d9000640 1 -- 172.21.15.90:0/200521372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d4102dc0 msgr2=0x7fa0d4199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0d9000640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d4102dc0 0x7fa0d4199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0d9000640 1 -- 172.21.15.90:0/200521372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0d41087d0 msgr2=0x7fa0d41996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0d9000640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0d41087d0 0x7fa0d41996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.183+0000 7fa0d9000640 1 -- 172.21.15.90:0/200521372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d419f210 con 0x7fa0d4109150 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.184+0000 7fa0d9000640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 0x7fa0d419ea40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc031040 tx=0x7fa0cc032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.184+0000 7fa0d1600640 1 -- 172.21.15.90:0/200521372 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc004040 con 0x7fa0d4109150 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.184+0000 7fa0d1600640 1 -- 172.21.15.90:0/200521372 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0cc004670 con 0x7fa0d4109150 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.184+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d419f4a0 con 0x7fa0d4109150 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.184+0000 7fa0d1600640 1 -- 172.21.15.90:0/200521372 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc030bb0 con 0x7fa0d4109150 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.184+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d41a5bc0 con 0x7fa0d4109150 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.185+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d41031a0 con 0x7fa0d4109150 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.187+0000 7fa0d1600640 1 -- 172.21.15.90:0/200521372 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fa0cc0041e0 con 0x7fa0d4109150 2024-09-10T22:00:05.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.189+0000 7fa0d1600640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0b80b53b0 0x7fa0b80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:05.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.189+0000 7fa0d1600640 1 -- 172.21.15.90:0/200521372 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fa0cc106d00 con 0x7fa0d4109150 2024-09-10T22:00:05.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.189+0000 7fa0d3e00640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0b80b53b0 0x7fa0b80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:05.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.189+0000 7fa0d3e00640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0b80b53b0 0x7fa0b80b7870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4005fd0 tx=0x7fa0c4005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:05.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.190+0000 7fa0d1600640 1 -- 172.21.15.90:0/200521372 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa0cc0cdef0 con 0x7fa0d4109150 2024-09-10T22:00:05.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.410+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0d402e5c0 con 0x7fa0d4109150 2024-09-10T22:00:05.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.410+0000 7fa0d1600640 1 -- 172.21.15.90:0/200521372 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6397 (secure 0 0 0) 0x7fa0cc0cd7f0 con 0x7fa0d4109150 2024-09-10T22:00:05.412 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:05.412 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":8,"btime":"2024-09-10T22:00:04:625102+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34640,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-10T22:00:05.412 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 8 2024-09-10T22:00:05.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.412+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0b80b53b0 msgr2=0x7fa0b80b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:05.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.412+0000 7fa0db000640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0b80b53b0 0x7fa0b80b7870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4005fd0 tx=0x7fa0c4005c70 comp rx=0 tx=0).stop 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.412+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 msgr2=0x7fa0d419ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.412+0000 7fa0db000640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 0x7fa0d419ea40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc031040 tx=0x7fa0cc032bc0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.413+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 shutdown_connections 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.413+0000 7fa0db000640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0b80b53b0 0x7fa0b80b7870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.413+0000 7fa0db000640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0d4109150 0x7fa0d419ea40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.413+0000 7fa0db000640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0d41087d0 0x7fa0d41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.413+0000 7fa0db000640 1 --2- 172.21.15.90:0/200521372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0d4102dc0 0x7fa0d4199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.413+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 >> 172.21.15.90:0/200521372 conn(0x7fa0d40fe830 msgr2=0x7fa0d4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.413+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 shutdown_connections 2024-09-10T22:00:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:05.413+0000 7fa0db000640 1 -- 172.21.15.90:0/200521372 wait complete. 2024-09-10T22:00:05.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:05 smithi090 ceph-mon[30350]: Deploying daemon mds.h on smithi154 2024-09-10T22:00:05.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:05 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.171:6836/2889350064,v1:172.21.15.171:6837/2889350064] up:boot 2024-09-10T22:00:05.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:05 smithi090 ceph-mon[30350]: fsmap 7 up:standby 2024-09-10T22:00:05.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:00:05.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:05 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/200521372' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:05.891 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:05 smithi154 ceph-mon[34495]: Deploying daemon mds.h on smithi154 2024-09-10T22:00:05.892 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:05 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.171:6836/2889350064,v1:172.21.15.171:6837/2889350064] up:boot 2024-09-10T22:00:05.892 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:05 smithi154 ceph-mon[34495]: fsmap 7 up:standby 2024-09-10T22:00:05.892 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:00:05.892 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:05 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/200521372' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:05.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:05 smithi171 ceph-mon[34138]: Deploying daemon mds.h on smithi154 2024-09-10T22:00:05.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:05 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.171:6836/2889350064,v1:172.21.15.171:6837/2889350064] up:boot 2024-09-10T22:00:05.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:05 smithi171 ceph-mon[34138]: fsmap 7 up:standby 2024-09-10T22:00:05.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:00:05.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:05 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/200521372' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:06.905 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:06 smithi154 ceph-mon[34495]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:06.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:06 smithi171 ceph-mon[34138]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:06 smithi090 ceph-mon[30350]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:07.168 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:00:06 smithi154 systemd[1]: Starting Ceph mds.h for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:00:07.543 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:00:07 smithi154 podman[65277]: 2024-09-10 22:00:07.124423184 +0000 UTC m=+0.028123635 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:00:07.543 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:00:07 smithi154 podman[65277]: 2024-09-10 22:00:07.255844084 +0000 UTC m=+0.159544521 container create 54d88274eab6f43ccb4bbf27fe413112d79f9b6bde9113536116f9ffff2ed905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T22:00:07.544 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:00:07 smithi154 podman[65277]: 2024-09-10 22:00:07.542353449 +0000 UTC m=+0.446053896 container init 54d88274eab6f43ccb4bbf27fe413112d79f9b6bde9113536116f9ffff2ed905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T22:00:07.918 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:00:07 smithi154 podman[65277]: 2024-09-10 22:00:07.548719206 +0000 UTC m=+0.452419642 container start 54d88274eab6f43ccb4bbf27fe413112d79f9b6bde9113536116f9ffff2ed905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T22:00:07.918 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:00:07 smithi154 ceph-mds[65296]: -- 172.21.15.154:0/2142166264 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55558ff35860 con 0x55558ff12800 2024-09-10T22:00:07.918 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:00:07 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h[65292]: starting mds.h at 2024-09-10T22:00:07.918 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:00:07 smithi154 bash[65277]: 54d88274eab6f43ccb4bbf27fe413112d79f9b6bde9113536116f9ffff2ed905 2024-09-10T22:00:07.918 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:00:07 smithi154 systemd[1]: Started Ceph mds.h for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T22:00:08.347 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:00:08.721 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:08.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] up:boot 2024-09-10T22:00:08.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: fsmap 8 up:standby 2024-09-10T22:00:08.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:00:08.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:08.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:08.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:08.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:08.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:08.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:08.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:08 smithi154 ceph-mon[34495]: Deploying daemon mds.i on smithi171 2024-09-10T22:00:08.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:08.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] up:boot 2024-09-10T22:00:08.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: fsmap 8 up:standby 2024-09-10T22:00:08.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:00:08.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:08.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:08.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:08.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:08.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:08.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:08.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:08 smithi090 ceph-mon[30350]: Deploying daemon mds.i on smithi171 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] up:boot 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: fsmap 8 up:standby 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:08 smithi171 ceph-mon[34138]: Deploying daemon mds.i on smithi171 2024-09-10T22:00:09.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.378+0000 7f4a7b200640 1 -- 172.21.15.90:0/3730832114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74101220 msgr2=0x7f4a74101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.378+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3730832114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74101220 0x7f4a74101600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009ca0 tx=0x7f4a6402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.378+0000 7f4a7b200640 1 -- 172.21.15.90:0/3730832114 shutdown_connections 2024-09-10T22:00:09.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.378+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3730832114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a74105ef0 0x7f4a7410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.378+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3730832114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a74105570 0x7f4a741059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.378+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3730832114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74101220 0x7f4a74101600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.378+0000 7f4a7b200640 1 -- 172.21.15.90:0/3730832114 >> 172.21.15.90:0/3730832114 conn(0x7f4a74078100 msgr2=0x7f4a740ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:09.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.378+0000 7f4a7b200640 1 -- 172.21.15.90:0/3730832114 shutdown_connections 2024-09-10T22:00:09.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.378+0000 7f4a7b200640 1 -- 172.21.15.90:0/3730832114 wait complete. 2024-09-10T22:00:09.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.379+0000 7f4a7b200640 1 Processor -- start 2024-09-10T22:00:09.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.379+0000 7f4a7b200640 1 -- start start 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a7b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a74101220 0x7f4a7419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a7b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74105570 0x7f4a7419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a7b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a74105ef0 0x7f4a741a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a7b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74112cf0 con 0x7f4a74105570 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a7b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74112b70 con 0x7f4a74101220 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a7b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4a74112e70 con 0x7f4a74105ef0 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a73400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74105570 0x7f4a7419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a73400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74105570 0x7f4a7419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49876/0 (socket says 172.21.15.90:49876) 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a73400640 1 -- 172.21.15.90:0/3788039981 learned_addr learned my addr 172.21.15.90:0/3788039981 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a79200640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a74105ef0 0x7f4a741a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.380+0000 7f4a73e00640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a74101220 0x7f4a7419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a73400640 1 -- 172.21.15.90:0/3788039981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a74105ef0 msgr2=0x7f4a741a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a73400640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a74105ef0 0x7f4a741a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a73400640 1 -- 172.21.15.90:0/3788039981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a74101220 msgr2=0x7f4a7419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a73400640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a74101220 0x7f4a7419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a73400640 1 -- 172.21.15.90:0/3788039981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a741a1470 con 0x7f4a74105570 2024-09-10T22:00:09.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a73e00640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a74101220 0x7f4a7419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:09.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a73400640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74105570 0x7f4a7419b900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000d950 tx=0x7f4a6000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:09.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a70c00640 1 -- 172.21.15.90:0/3788039981 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60017030 con 0x7f4a74105570 2024-09-10T22:00:09.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a70c00640 1 -- 172.21.15.90:0/3788039981 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4a60004020 con 0x7f4a74105570 2024-09-10T22:00:09.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a70c00640 1 -- 172.21.15.90:0/3788039981 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a600055d0 con 0x7f4a74105570 2024-09-10T22:00:09.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a741a1760 con 0x7f4a74105570 2024-09-10T22:00:09.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.381+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a7e20 con 0x7f4a74105570 2024-09-10T22:00:09.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.382+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a74101600 con 0x7f4a74105570 2024-09-10T22:00:09.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.385+0000 7f4a70c00640 1 -- 172.21.15.90:0/3788039981 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f4a60009a30 con 0x7f4a74105570 2024-09-10T22:00:09.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.387+0000 7f4a70c00640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4a580b5330 0x7f4a580b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:09.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.387+0000 7f4a70c00640 1 -- 172.21.15.90:0/3788039981 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f4a600e5390 con 0x7f4a74105570 2024-09-10T22:00:09.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.387+0000 7f4a73e00640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4a580b5330 0x7f4a580b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:09.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.388+0000 7f4a73e00640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4a580b5330 0x7f4a580b77f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005fa0 tx=0x7f4a64009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:09.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.390+0000 7f4a70c00640 1 -- 172.21.15.90:0/3788039981 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4a600ac580 con 0x7f4a74105570 2024-09-10T22:00:09.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.610+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a741a8210 con 0x7f4a74105570 2024-09-10T22:00:09.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.610+0000 7f4a70c00640 1 -- 172.21.15.90:0/3788039981 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7f4a600abe80 con 0x7f4a74105570 2024-09-10T22:00:09.612 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:09.612 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":9,"btime":"2024-09-10T22:00:07:641079+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34640,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34670,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T22:00:09.613 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 9 2024-09-10T22:00:09.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4a580b5330 msgr2=0x7f4a580b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4a580b5330 0x7f4a580b77f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005fa0 tx=0x7f4a64009360 comp rx=0 tx=0).stop 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74105570 msgr2=0x7f4a7419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74105570 0x7f4a7419b900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000d950 tx=0x7f4a6000de10 comp rx=0 tx=0).stop 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 shutdown_connections 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4a580b5330 0x7f4a580b77f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4a74105ef0 0x7f4a741a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a74105570 0x7f4a7419b900 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 --2- 172.21.15.90:0/3788039981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4a74101220 0x7f4a7419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 >> 172.21.15.90:0/3788039981 conn(0x7f4a74078100 msgr2=0x7f4a74108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 shutdown_connections 2024-09-10T22:00:09.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:09.613+0000 7f4a7b200640 1 -- 172.21.15.90:0/3788039981 wait complete. 2024-09-10T22:00:09.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:09 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3788039981' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:09.949 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:09 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3788039981' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:10.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:09 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3788039981' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:10.498 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:00:10 smithi171 systemd[1]: Starting Ceph mds.i for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:00:10.844 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:10 smithi171 ceph-mon[34138]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:10.844 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:00:10 smithi171 podman[65474]: 2024-09-10 22:00:10.430707969 +0000 UTC m=+0.027368287 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:00:10.844 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:00:10 smithi171 podman[65474]: 2024-09-10 22:00:10.555842494 +0000 UTC m=+0.152502812 container create 6a13ab59feeb787efe45618f9d1d5c5ef2595806bdaefb6bfdb368e5d1fe1b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T22:00:10.844 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:00:10 smithi171 podman[65474]: 2024-09-10 22:00:10.843339501 +0000 UTC m=+0.439999831 container init 6a13ab59feeb787efe45618f9d1d5c5ef2595806bdaefb6bfdb368e5d1fe1b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T22:00:10.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:10 smithi090 ceph-mon[30350]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:10.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:10 smithi154 ceph-mon[34495]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:11.247 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:00:10 smithi171 podman[65474]: 2024-09-10 22:00:10.849902078 +0000 UTC m=+0.446562396 container start 6a13ab59feeb787efe45618f9d1d5c5ef2595806bdaefb6bfdb368e5d1fe1b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:00:11.248 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:00:10 smithi171 ceph-mds[65492]: -- 172.21.15.171:0/1360542241 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4c089fc20 con 0x55c4c087d400 2024-09-10T22:00:11.248 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:00:10 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i[65488]: starting mds.i at 2024-09-10T22:00:11.248 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:00:10 smithi171 bash[65474]: 6a13ab59feeb787efe45618f9d1d5c5ef2595806bdaefb6bfdb368e5d1fe1b47 2024-09-10T22:00:11.248 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:00:10 smithi171 systemd[1]: Started Ceph mds.i for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T22:00:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: Deploying daemon mds.g on smithi090 2024-09-10T22:00:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.171:6838/3372214298,v1:172.21.15.171:6839/3372214298] up:boot 2024-09-10T22:00:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: fsmap 9 up:standby 2024-09-10T22:00:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:00:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:12.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:12.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:12.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: Deploying daemon mds.g on smithi090 2024-09-10T22:00:12.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:12.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.171:6838/3372214298,v1:172.21.15.171:6839/3372214298] up:boot 2024-09-10T22:00:12.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: fsmap 9 up:standby 2024-09-10T22:00:12.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: Deploying daemon mds.g on smithi090 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.171:6838/3372214298,v1:172.21.15.171:6839/3372214298] up:boot 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: fsmap 9 up:standby 2024-09-10T22:00:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:00:12.564 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:00:13.170 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:14.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.031+0000 7efc44400640 1 -- 172.21.15.90:0/2864742709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c0776d0 msgr2=0x7efc3c077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:14.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.031+0000 7efc44400640 1 --2- 172.21.15.90:0/2864742709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c0776d0 0x7efc3c077ab0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efc30009ca0 tx=0x7efc3002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.031+0000 7efc44400640 1 -- 172.21.15.90:0/2864742709 shutdown_connections 2024-09-10T22:00:14.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.031+0000 7efc44400640 1 --2- 172.21.15.90:0/2864742709 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc3c1139b0 0x7efc3c115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.031+0000 7efc44400640 1 --2- 172.21.15.90:0/2864742709 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc3c077ff0 0x7efc3c113470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.031+0000 7efc44400640 1 --2- 172.21.15.90:0/2864742709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c0776d0 0x7efc3c077ab0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.031+0000 7efc44400640 1 -- 172.21.15.90:0/2864742709 >> 172.21.15.90:0/2864742709 conn(0x7efc3c1009f0 msgr2=0x7efc3c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.031+0000 7efc44400640 1 -- 172.21.15.90:0/2864742709 shutdown_connections 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.031+0000 7efc44400640 1 -- 172.21.15.90:0/2864742709 wait complete. 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.032+0000 7efc44400640 1 Processor -- start 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.032+0000 7efc44400640 1 -- start start 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc44400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc3c0776d0 0x7efc3c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc44400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc3c077ff0 0x7efc3c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc44400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c1139b0 0x7efc3c1a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc44400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c1192b0 con 0x7efc3c1139b0 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc44400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c119130 con 0x7efc3c0776d0 2024-09-10T22:00:14.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc44400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c119430 con 0x7efc3c077ff0 2024-09-10T22:00:14.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc42400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c1139b0 0x7efc3c1a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc42400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c1139b0 0x7efc3c1a50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49900/0 (socket says 172.21.15.90:49900) 2024-09-10T22:00:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc42400640 1 -- 172.21.15.90:0/1229549179 learned_addr learned my addr 172.21.15.90:0/1229549179 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc41a00640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc3c0776d0 0x7efc3c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc41000640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc3c077ff0 0x7efc3c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc42400640 1 -- 172.21.15.90:0/1229549179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc3c077ff0 msgr2=0x7efc3c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc42400640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc3c077ff0 0x7efc3c19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc42400640 1 -- 172.21.15.90:0/1229549179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc3c0776d0 msgr2=0x7efc3c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.033+0000 7efc42400640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc3c0776d0 0x7efc3c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.034+0000 7efc42400640 1 -- 172.21.15.90:0/1229549179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c1a5890 con 0x7efc3c1139b0 2024-09-10T22:00:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.034+0000 7efc42400640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c1139b0 0x7efc3c1a50c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efc3400d720 tx=0x7efc3400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.034+0000 7efc3a000640 1 -- 172.21.15.90:0/1229549179 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34017070 con 0x7efc3c1139b0 2024-09-10T22:00:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.034+0000 7efc3a000640 1 -- 172.21.15.90:0/1229549179 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efc3400de30 con 0x7efc3c1139b0 2024-09-10T22:00:14.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.034+0000 7efc3a000640 1 -- 172.21.15.90:0/1229549179 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34004c60 con 0x7efc3c1139b0 2024-09-10T22:00:14.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.034+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c1a5b80 con 0x7efc3c1139b0 2024-09-10T22:00:14.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.034+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c1ac240 con 0x7efc3c1139b0 2024-09-10T22:00:14.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.036+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c077ab0 con 0x7efc3c1139b0 2024-09-10T22:00:14.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.037+0000 7efc3a000640 1 -- 172.21.15.90:0/1229549179 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7efc34004e00 con 0x7efc3c1139b0 2024-09-10T22:00:14.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.039+0000 7efc3a000640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efc240b5330 0x7efc240b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:14.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.039+0000 7efc3a000640 1 -- 172.21.15.90:0/1229549179 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7efc340e51a0 con 0x7efc3c1139b0 2024-09-10T22:00:14.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.039+0000 7efc41a00640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efc240b5330 0x7efc240b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:14.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.040+0000 7efc41a00640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efc240b5330 0x7efc240b77f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc30006010 tx=0x7efc30031000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:14.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.043+0000 7efc3a000640 1 -- 172.21.15.90:0/1229549179 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efc340ac390 con 0x7efc3c1139b0 2024-09-10T22:00:14.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.281+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc3c02e5c0 con 0x7efc3c1139b0 2024-09-10T22:00:14.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.282+0000 7efc3a000640 1 -- 172.21.15.90:0/1229549179 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8055 (secure 0 0 0) 0x7efc340abc90 con 0x7efc3c1139b0 2024-09-10T22:00:14.284 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:14.284 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":10,"btime":"2024-09-10T22:00:11:661829+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34640,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34670,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T22:00:14.284 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 10 2024-09-10T22:00:14.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efc240b5330 msgr2=0x7efc240b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:14.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efc240b5330 0x7efc240b77f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc30006010 tx=0x7efc30031000 comp rx=0 tx=0).stop 2024-09-10T22:00:14.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c1139b0 msgr2=0x7efc3c1a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:14.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c1139b0 0x7efc3c1a50c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efc3400d720 tx=0x7efc3400dbe0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 shutdown_connections 2024-09-10T22:00:14.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efc240b5330 0x7efc240b77f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc3c1139b0 0x7efc3c1a50c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efc3c077ff0 0x7efc3c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 --2- 172.21.15.90:0/1229549179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efc3c0776d0 0x7efc3c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:14.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 >> 172.21.15.90:0/1229549179 conn(0x7efc3c1009f0 msgr2=0x7efc3c1145b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:14.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 shutdown_connections 2024-09-10T22:00:14.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:14.285+0000 7efc44400640 1 -- 172.21.15.90:0/1229549179 wait complete. 2024-09-10T22:00:14.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:14 smithi154 ceph-mon[34495]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:14.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:14 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1229549179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:14 smithi171 ceph-mon[34138]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:14.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:14 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1229549179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:14.998 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:14 smithi090 ceph-mon[30350]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:14.998 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:14 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1229549179' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:15.959 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:00:15 smithi090 podman[75401]: 2024-09-10 22:00:15.920269975 +0000 UTC m=+1.711985263 container start 98f264980d681bf8737990158f9ca54f2ecb4dd8f8a3cc9fab28bbf9e52ff3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T22:00:16.356 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:00:15 smithi090 ceph-mds[75436]: -- 172.21.15.90:0/1839130443 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55686b831860 con 0x55686b80e800 2024-09-10T22:00:16.356 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:00:15 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g[75428]: starting mds.g at 2024-09-10T22:00:16.356 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:00:16 smithi090 bash[75401]: 98f264980d681bf8737990158f9ca54f2ecb4dd8f8a3cc9fab28bbf9e52ff3fb 2024-09-10T22:00:16.356 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:00:16 smithi090 systemd[1]: Started Ceph mds.g for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T22:00:16.901 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:16 smithi090 ceph-mon[30350]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:16.901 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:16.901 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:16.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:16.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:16.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:16.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:16.921 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:16 smithi154 ceph-mon[34495]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:16.921 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:16.921 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:16.921 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:16.921 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:16.922 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:16.922 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:16 smithi171 ceph-mon[34138]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:16.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:17.734 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:00:17.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:17 smithi090 ceph-mon[30350]: Deploying daemon mds.a on smithi090 2024-09-10T22:00:17.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:17 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.90:6838/3596805819,v1:172.21.15.90:6839/3596805819] up:boot 2024-09-10T22:00:17.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:17 smithi090 ceph-mon[30350]: fsmap 10 up:standby 2024-09-10T22:00:17.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:00:17.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:17 smithi154 ceph-mon[34495]: Deploying daemon mds.a on smithi090 2024-09-10T22:00:17.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:17 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.90:6838/3596805819,v1:172.21.15.90:6839/3596805819] up:boot 2024-09-10T22:00:17.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:17 smithi154 ceph-mon[34495]: fsmap 10 up:standby 2024-09-10T22:00:17.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:00:17.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:17 smithi171 ceph-mon[34138]: Deploying daemon mds.a on smithi090 2024-09-10T22:00:17.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:17 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.90:6838/3596805819,v1:172.21.15.90:6839/3596805819] up:boot 2024-09-10T22:00:17.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:17 smithi171 ceph-mon[34138]: fsmap 10 up:standby 2024-09-10T22:00:17.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:00:18.141 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:18.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:18 smithi154 ceph-mon[34495]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:18.984 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:18 smithi090 ceph-mon[30350]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.984+0000 7fa0fce00640 1 -- 172.21.15.90:0/24879483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810a9b0 msgr2=0x7fa0f810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.984+0000 7fa0fce00640 1 --2- 172.21.15.90:0/24879483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810a9b0 0x7fa0f810adf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009900 tx=0x7fa0e402a410 comp rx=0 tx=0).stop 2024-09-10T22:00:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.984+0000 7fa0fce00640 1 -- 172.21.15.90:0/24879483 shutdown_connections 2024-09-10T22:00:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.984+0000 7fa0fce00640 1 --2- 172.21.15.90:0/24879483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f810b330 0x7fa0f8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.984+0000 7fa0fce00640 1 --2- 172.21.15.90:0/24879483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810a9b0 0x7fa0f810adf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.984+0000 7fa0fce00640 1 --2- 172.21.15.90:0/24879483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f8104fa0 0x7fa0f8105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.984+0000 7fa0fce00640 1 -- 172.21.15.90:0/24879483 >> 172.21.15.90:0/24879483 conn(0x7fa0f8100a50 msgr2=0x7fa0f8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.985+0000 7fa0fce00640 1 -- 172.21.15.90:0/24879483 shutdown_connections 2024-09-10T22:00:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.985+0000 7fa0fce00640 1 -- 172.21.15.90:0/24879483 wait complete. 2024-09-10T22:00:18.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.985+0000 7fa0fce00640 1 Processor -- start 2024-09-10T22:00:18.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0fce00640 1 -- start start 2024-09-10T22:00:18.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0fce00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f8104fa0 0x7fa0f819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:18.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0fce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f810a9b0 0x7fa0f819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:18.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0fce00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810b330 0x7fa0f81a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0fce00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8114e00 con 0x7fa0f810b330 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0fce00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8114c80 con 0x7fa0f8104fa0 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0fce00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f8114f80 con 0x7fa0f810a9b0 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0f6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810b330 0x7fa0f81a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0f5400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f810a9b0 0x7fa0f819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0f5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f8104fa0 0x7fa0f819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0f6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810b330 0x7fa0f81a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40350/0 (socket says 172.21.15.90:40350) 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.986+0000 7fa0f6800640 1 -- 172.21.15.90:0/2078362502 learned_addr learned my addr 172.21.15.90:0/2078362502 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0f6800640 1 -- 172.21.15.90:0/2078362502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f810a9b0 msgr2=0x7fa0f819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0f6800640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f810a9b0 0x7fa0f819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0f6800640 1 -- 172.21.15.90:0/2078362502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f8104fa0 msgr2=0x7fa0f819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0f6800640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f8104fa0 0x7fa0f819b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0f6800640 1 -- 172.21.15.90:0/2078362502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f81a1440 con 0x7fa0f810b330 2024-09-10T22:00:18.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0f5e00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f8104fa0 0x7fa0f819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:18.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0f6800640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810b330 0x7fa0f81a0c70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000eaa0 tx=0x7fa0f000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:18.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0eea00640 1 -- 172.21.15.90:0/2078362502 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000cdb0 con 0x7fa0f810b330 2024-09-10T22:00:18.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f81a1730 con 0x7fa0f810b330 2024-09-10T22:00:18.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0eea00640 1 -- 172.21.15.90:0/2078362502 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0f0005020 con 0x7fa0f810b330 2024-09-10T22:00:18.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0eea00640 1 -- 172.21.15.90:0/2078362502 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00116d0 con 0x7fa0f810b330 2024-09-10T22:00:18.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.987+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f81a7df0 con 0x7fa0f810b330 2024-09-10T22:00:18.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.989+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f8106250 con 0x7fa0f810b330 2024-09-10T22:00:18.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.991+0000 7fa0eea00640 1 -- 172.21.15.90:0/2078362502 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fa0f000f040 con 0x7fa0f810b330 2024-09-10T22:00:18.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.992+0000 7fa0eea00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0dc0b5330 0x7fa0dc0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:18.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.992+0000 7fa0eea00640 1 -- 172.21.15.90:0/2078362502 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fa0f00e56c0 con 0x7fa0f810b330 2024-09-10T22:00:18.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.992+0000 7fa0f5e00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0dc0b5330 0x7fa0dc0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:18.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.993+0000 7fa0f5e00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0dc0b5330 0x7fa0dc0b77f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009fc0 tx=0x7fa0e8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:18.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:18.996+0000 7fa0eea00640 1 -- 172.21.15.90:0/2078362502 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa0f0016030 con 0x7fa0f810b330 2024-09-10T22:00:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:18 smithi171 ceph-mon[34138]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:19.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.268+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0f810d170 con 0x7fa0f810b330 2024-09-10T22:00:19.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.268+0000 7fa0eea00640 1 -- 172.21.15.90:0/2078362502 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8881 (secure 0 0 0) 0x7fa0f00ac260 con 0x7fa0f810b330 2024-09-10T22:00:19.270 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:19.270 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":11,"btime":"2024-09-10T22:00:16:632159+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14841,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34640,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34670,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T22:00:19.271 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 11 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0dc0b5330 msgr2=0x7fa0dc0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0dc0b5330 0x7fa0dc0b77f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009fc0 tx=0x7fa0e8005f90 comp rx=0 tx=0).stop 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810b330 msgr2=0x7fa0f81a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810b330 0x7fa0f81a0c70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000eaa0 tx=0x7fa0f000ef60 comp rx=0 tx=0).stop 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 shutdown_connections 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0dc0b5330 0x7fa0dc0b77f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0f810b330 0x7fa0f81a0c70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0f810a9b0 0x7fa0f819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 --2- 172.21.15.90:0/2078362502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0f8104fa0 0x7fa0f819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:19.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 >> 172.21.15.90:0/2078362502 conn(0x7fa0f8100a50 msgr2=0x7fa0f8102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:19.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 shutdown_connections 2024-09-10T22:00:19.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:19.271+0000 7fa0fce00640 1 -- 172.21.15.90:0/2078362502 wait complete. 2024-09-10T22:00:19.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:19 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2078362502' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:19.982 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:19 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2078362502' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:19 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2078362502' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:20.356 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:00:20 smithi090 systemd[1]: Started Ceph mds.a for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T22:00:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:20 smithi090 ceph-mon[30350]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:20.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:20.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:20.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:20.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:20.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:20.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:20 smithi154 ceph-mon[34495]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:20.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:20.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:20 smithi171 ceph-mon[34138]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T22:00:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T22:00:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:21.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:21 smithi154 ceph-mon[34495]: Deploying daemon mds.b on smithi154 2024-09-10T22:00:21.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:21 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.90:6840/3169114114,v1:172.21.15.90:6841/3169114114] up:boot 2024-09-10T22:00:21.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:21 smithi154 ceph-mon[34495]: fsmap 11 up:standby 2024-09-10T22:00:21.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:21 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:00:21.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:21 smithi171 ceph-mon[34138]: Deploying daemon mds.b on smithi154 2024-09-10T22:00:21.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:21 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.90:6840/3169114114,v1:172.21.15.90:6841/3169114114] up:boot 2024-09-10T22:00:21.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:21 smithi171 ceph-mon[34138]: fsmap 11 up:standby 2024-09-10T22:00:21.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:21 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:00:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:21 smithi090 ceph-mon[30350]: Deploying daemon mds.b on smithi154 2024-09-10T22:00:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:21 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.90:6840/3169114114,v1:172.21.15.90:6841/3169114114] up:boot 2024-09-10T22:00:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:21 smithi090 ceph-mon[30350]: fsmap 11 up:standby 2024-09-10T22:00:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:21 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:00:22.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:00:22.832 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:22 smithi154 ceph-mon[34495]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:22 smithi171 ceph-mon[34138]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:23.018 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:22 smithi090 ceph-mon[30350]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:23.146 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:22 smithi154 systemd[1]: Starting Ceph mds.b for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:00:23.418 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:23 smithi154 podman[65593]: 2024-09-10 22:00:23.049397163 +0000 UTC m=+0.027422430 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:00:23.418 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:23 smithi154 podman[65593]: 2024-09-10 22:00:23.188924652 +0000 UTC m=+0.166949919 container create 60d5877916da954419638508f2989752cdd7dee9c5e1285c1291f483a80956c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.639+0000 7f2773800640 1 -- 172.21.15.90:0/3992656335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c102dc0 msgr2=0x7f276c1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.639+0000 7f2773800640 1 --2- 172.21.15.90:0/3992656335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c102dc0 0x7f276c1031a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2760009900 tx=0x7f276002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.640+0000 7f2773800640 1 -- 172.21.15.90:0/3992656335 shutdown_connections 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.640+0000 7f2773800640 1 --2- 172.21.15.90:0/3992656335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f276c109150 0x7f276c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.640+0000 7f2773800640 1 --2- 172.21.15.90:0/3992656335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f276c1087d0 0x7f276c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.640+0000 7f2773800640 1 --2- 172.21.15.90:0/3992656335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c102dc0 0x7f276c1031a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.640+0000 7f2773800640 1 -- 172.21.15.90:0/3992656335 >> 172.21.15.90:0/3992656335 conn(0x7f276c0fe830 msgr2=0x7f276c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.640+0000 7f2773800640 1 -- 172.21.15.90:0/3992656335 shutdown_connections 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.640+0000 7f2773800640 1 -- 172.21.15.90:0/3992656335 wait complete. 2024-09-10T22:00:23.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2773800640 1 Processor -- start 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2773800640 1 -- start start 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2773800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f276c102dc0 0x7f276c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2773800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f276c1087d0 0x7f276c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2773800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c109150 0x7f276c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2773800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f276c112cd0 con 0x7f276c109150 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2773800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f276c112b50 con 0x7f276c1087d0 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2773800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f276c112e50 con 0x7f276c102dc0 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2771800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c109150 0x7f276c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2770e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f276c102dc0 0x7f276c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2771800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c109150 0x7f276c1a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40402/0 (socket says 172.21.15.90:40402) 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2770e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f276c102dc0 0x7f276c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:53604/0 (socket says 172.21.15.90:53604) 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f2771800640 1 -- 172.21.15.90:0/2335679859 learned_addr learned my addr 172.21.15.90:0/2335679859 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:23.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.641+0000 7f276be00640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f276c1087d0 0x7f276c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.642+0000 7f2771800640 1 -- 172.21.15.90:0/2335679859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f276c102dc0 msgr2=0x7f276c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.642+0000 7f2771800640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f276c102dc0 0x7f276c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.642+0000 7f2771800640 1 -- 172.21.15.90:0/2335679859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f276c1087d0 msgr2=0x7f276c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.642+0000 7f2771800640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f276c1087d0 0x7f276c19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.642+0000 7f2771800640 1 -- 172.21.15.90:0/2335679859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c1a1460 con 0x7f276c109150 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.642+0000 7f2771800640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c109150 0x7f276c1a0c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f276400eaa0 tx=0x7f276400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.642+0000 7f2769600640 1 -- 172.21.15.90:0/2335679859 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764004d20 con 0x7f276c109150 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.642+0000 7f2769600640 1 -- 172.21.15.90:0/2335679859 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2764005020 con 0x7f276c109150 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.642+0000 7f2769600640 1 -- 172.21.15.90:0/2335679859 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27640116d0 con 0x7f276c109150 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.643+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276c1a1750 con 0x7f276c109150 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.643+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276c1a7e10 con 0x7f276c109150 2024-09-10T22:00:23.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.644+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276c1031a0 con 0x7f276c109150 2024-09-10T22:00:23.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.646+0000 7f2769600640 1 -- 172.21.15.90:0/2335679859 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f276400f040 con 0x7f276c109150 2024-09-10T22:00:23.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.648+0000 7f2769600640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27540b53b0 0x7f27540b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:23.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.648+0000 7f2769600640 1 -- 172.21.15.90:0/2335679859 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f27640e5670 con 0x7f276c109150 2024-09-10T22:00:23.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.648+0000 7f2770e00640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27540b53b0 0x7f27540b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:23.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.649+0000 7f2770e00640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27540b53b0 0x7f27540b7870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2760006010 tx=0x7f2760005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:23.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.652+0000 7f2769600640 1 -- 172.21.15.90:0/2335679859 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2764016030 con 0x7f276c109150 2024-09-10T22:00:23.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.883+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f276c02e5c0 con 0x7f276c109150 2024-09-10T22:00:23.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.884+0000 7f2769600640 1 -- 172.21.15.90:0/2335679859 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10533 (secure 0 0 0) 0x7f27640ac860 con 0x7f276c109150 2024-09-10T22:00:23.885 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:23.885 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":13,"btime":"2024-09-10T22:00:23:683289+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14841,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14853,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34640,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34670,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":34703,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T22:00:23.886 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 13 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27540b53b0 msgr2=0x7f27540b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27540b53b0 0x7f27540b7870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2760006010 tx=0x7f2760005e80 comp rx=0 tx=0).stop 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c109150 msgr2=0x7f276c1a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c109150 0x7f276c1a0c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f276400eaa0 tx=0x7f276400eea0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 shutdown_connections 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27540b53b0 0x7f27540b7870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f276c109150 0x7f276c1a0c90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f276c1087d0 0x7f276c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 --2- 172.21.15.90:0/2335679859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f276c102dc0 0x7f276c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 >> 172.21.15.90:0/2335679859 conn(0x7f276c0fe830 msgr2=0x7f276c104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:23.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 shutdown_connections 2024-09-10T22:00:23.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:23.886+0000 7f2773800640 1 -- 172.21.15.90:0/2335679859 wait complete. 2024-09-10T22:00:23.918 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:23 smithi154 podman[65593]: 2024-09-10 22:00:23.433964588 +0000 UTC m=+0.411989855 container init 60d5877916da954419638508f2989752cdd7dee9c5e1285c1291f483a80956c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T22:00:23.918 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:23 smithi154 podman[65593]: 2024-09-10 22:00:23.440254639 +0000 UTC m=+0.418279914 container start 60d5877916da954419638508f2989752cdd7dee9c5e1285c1291f483a80956c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-10T22:00:23.918 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:23 smithi154 ceph-mds[65612]: -- 172.21.15.154:0/1646739352 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6b1d09860 con 0x55b6b1ce7000 2024-09-10T22:00:23.918 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:23 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b[65608]: starting mds.b at 2024-09-10T22:00:23.918 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:23 smithi154 bash[65593]: 60d5877916da954419638508f2989752cdd7dee9c5e1285c1291f483a80956c9 2024-09-10T22:00:23.918 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:23 smithi154 systemd[1]: Started Ceph mds.b for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T22:00:24.701 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T22:00:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] up:boot 2024-09-10T22:00:24.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: fsmap 12 up:standby 2024-09-10T22:00:24.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:00:24.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2335679859' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:24.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:00:24.928 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] up:boot 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: fsmap 12 up:standby 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2335679859' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:24.929 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:00:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] up:boot 2024-09-10T22:00:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: fsmap 12 up:standby 2024-09-10T22:00:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:00:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2335679859' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:00:25.861 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.531+0000 7f1265e00640 1 -- 172.21.15.90:0/3364535416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010b350 msgr2=0x7f1260111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.531+0000 7f1265e00640 1 --2- 172.21.15.90:0/3364535416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010b350 0x7f1260111b50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1258009ca0 tx=0x7f125802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.531+0000 7f1265e00640 1 -- 172.21.15.90:0/3364535416 shutdown_connections 2024-09-10T22:00:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.531+0000 7f1265e00640 1 --2- 172.21.15.90:0/3364535416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010b350 0x7f1260111b50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.531+0000 7f1265e00640 1 --2- 172.21.15.90:0/3364535416 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f126010a9d0 0x7f126010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.531+0000 7f1265e00640 1 --2- 172.21.15.90:0/3364535416 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1260104fc0 0x7f12601053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.531+0000 7f1265e00640 1 -- 172.21.15.90:0/3364535416 >> 172.21.15.90:0/3364535416 conn(0x7f12601009f0 msgr2=0x7f1260102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.531+0000 7f1265e00640 1 -- 172.21.15.90:0/3364535416 shutdown_connections 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.531+0000 7f1265e00640 1 -- 172.21.15.90:0/3364535416 wait complete. 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.532+0000 7f1265e00640 1 Processor -- start 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.532+0000 7f1265e00640 1 -- start start 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.532+0000 7f1265e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1260104fc0 0x7f126019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.532+0000 7f1265e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010a9d0 0x7f126019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.532+0000 7f1265e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f126010b350 0x7f12601a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f1265e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1260114e80 con 0x7f126010a9d0 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f1265e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1260114d00 con 0x7f126010b350 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f1265e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1260115000 con 0x7f1260104fc0 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010a9d0 0x7f126019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010a9d0 0x7f126019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40422/0 (socket says 172.21.15.90:40422) 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125e000640 1 -- 172.21.15.90:0/2192277730 learned_addr learned my addr 172.21.15.90:0/2192277730 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125ea00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1260104fc0 0x7f126019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125f400640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f126010b350 0x7f12601a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125e000640 1 -- 172.21.15.90:0/2192277730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1260104fc0 msgr2=0x7f126019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125e000640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1260104fc0 0x7f126019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125e000640 1 -- 172.21.15.90:0/2192277730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f126010b350 msgr2=0x7f12601a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125e000640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f126010b350 0x7f12601a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.533+0000 7f125e000640 1 -- 172.21.15.90:0/2192277730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12601a1420 con 0x7f126010a9d0 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.534+0000 7f125e000640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010a9d0 0x7f126019b8b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9f0 tx=0x7f124c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.534+0000 7f1264a00640 1 -- 172.21.15.90:0/2192277730 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c00cd80 con 0x7f126010a9d0 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.534+0000 7f1264a00640 1 -- 172.21.15.90:0/2192277730 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f124c0054b0 con 0x7f126010a9d0 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.534+0000 7f1264a00640 1 -- 172.21.15.90:0/2192277730 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c010430 con 0x7f126010a9d0 2024-09-10T22:00:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.534+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12601a1710 con 0x7f126010a9d0 2024-09-10T22:00:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.534+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12601a7dd0 con 0x7f126010a9d0 2024-09-10T22:00:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.535+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12601053a0 con 0x7f126010a9d0 2024-09-10T22:00:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.538+0000 7f1264a00640 1 -- 172.21.15.90:0/2192277730 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f124c005020 con 0x7f126010a9d0 2024-09-10T22:00:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.540+0000 7f1264a00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f12440b53b0 0x7f12440b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.541+0000 7f1264a00640 1 -- 172.21.15.90:0/2192277730 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f124c0e5ca0 con 0x7f126010a9d0 2024-09-10T22:00:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.541+0000 7f125ea00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f12440b53b0 0x7f12440b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.541+0000 7f125ea00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f12440b53b0 0x7f12440b7870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1250009bd0 tx=0x7f1250005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.544+0000 7f1264a00640 1 -- 172.21.15.90:0/2192277730 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f124c0ace90 con 0x7f126010a9d0 2024-09-10T22:00:26.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.802+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f126010d130 con 0x7f126010a9d0 2024-09-10T22:00:26.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:26 smithi090 ceph-mon[30350]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:26.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:26 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.805 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:26.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.803+0000 7f1264a00640 1 -- 172.21.15.90:0/2192277730 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f124c0ac790 con 0x7f126010a9d0 2024-09-10T22:00:26.806 INFO:teuthology.orchestra.run.smithi090.stdout:[ 2024-09-10T22:00:26.806 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:00:26.806 INFO:teuthology.orchestra.run.smithi090.stdout: "poolnum": 1, 2024-09-10T22:00:26.806 INFO:teuthology.orchestra.run.smithi090.stdout: "poolname": ".mgr" 2024-09-10T22:00:26.806 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:00:26.806 INFO:teuthology.orchestra.run.smithi090.stdout:] 2024-09-10T22:00:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f12440b53b0 msgr2=0x7f12440b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f12440b53b0 0x7f12440b7870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1250009bd0 tx=0x7f1250005f90 comp rx=0 tx=0).stop 2024-09-10T22:00:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010a9d0 msgr2=0x7f126019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:26.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010a9d0 0x7f126019b8b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f124c00e9f0 tx=0x7f124c00edf0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 shutdown_connections 2024-09-10T22:00:26.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f12440b53b0 0x7f12440b7870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f126010b350 0x7f12601a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f126010a9d0 0x7f126019b8b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 --2- 172.21.15.90:0/2192277730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1260104fc0 0x7f126019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:26.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.806+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 >> 172.21.15.90:0/2192277730 conn(0x7f12601009f0 msgr2=0x7f1260102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:26.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.807+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 shutdown_connections 2024-09-10T22:00:26.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:26.807+0000 7f1265e00640 1 -- 172.21.15.90:0/2192277730 wait complete. 2024-09-10T22:00:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:26 smithi154 ceph-mon[34495]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:26 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:26 smithi171 ceph-mon[34138]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:26 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:27.665 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T22:00:27.665 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T22:00:27.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:27 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2192277730' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:00:27.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:27 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2192277730' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:00:28.065 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:28.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:27 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2192277730' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T22:00:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.709+0000 7f3e2e800640 1 -- 172.21.15.90:0/430521354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e28104430 msgr2=0x7f3e2810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.709+0000 7f3e2e800640 1 --2- 172.21.15.90:0/430521354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e28104430 0x7f3e2810c920 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3e14009900 tx=0x7f3e1402a410 comp rx=0 tx=0).stop 2024-09-10T22:00:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.709+0000 7f3e2e800640 1 -- 172.21.15.90:0/430521354 shutdown_connections 2024-09-10T22:00:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.709+0000 7f3e2e800640 1 --2- 172.21.15.90:0/430521354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e2810cf70 0x7f3e2810f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:28.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.709+0000 7f3e2e800640 1 --2- 172.21.15.90:0/430521354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e28104430 0x7f3e2810c920 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.709+0000 7f3e2e800640 1 --2- 172.21.15.90:0/430521354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e28103b10 0x7f3e28103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.709+0000 7f3e2e800640 1 -- 172.21.15.90:0/430521354 >> 172.21.15.90:0/430521354 conn(0x7f3e280fd5b0 msgr2=0x7f3e280ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.710+0000 7f3e2e800640 1 -- 172.21.15.90:0/430521354 shutdown_connections 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.710+0000 7f3e2e800640 1 -- 172.21.15.90:0/430521354 wait complete. 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.710+0000 7f3e2e800640 1 Processor -- start 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2e800640 1 -- start start 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e28103b10 0x7f3e2819b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e28104430 0x7f3e2819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2810cf70 0x7f3e281a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2e800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28112a90 con 0x7f3e2810cf70 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2e800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28112910 con 0x7f3e28103b10 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2e800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3e28112c10 con 0x7f3e28104430 2024-09-10T22:00:28.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2810cf70 0x7f3e281a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:28.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2810cf70 0x7f3e281a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56206/0 (socket says 172.21.15.90:56206) 2024-09-10T22:00:28.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2de00640 1 -- 172.21.15.90:0/1766385729 learned_addr learned my addr 172.21.15.90:0/1766385729 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:28.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:28 smithi090 ceph-mon[30350]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:28.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:28 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:28.720 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:28 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.711+0000 7f3e2d400640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e28103b10 0x7f3e2819b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.712+0000 7f3e2ca00640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e28104430 0x7f3e2819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.712+0000 7f3e2de00640 1 -- 172.21.15.90:0/1766385729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e28104430 msgr2=0x7f3e2819b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.712+0000 7f3e2de00640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e28104430 0x7f3e2819b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.712+0000 7f3e2de00640 1 -- 172.21.15.90:0/1766385729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e28103b10 msgr2=0x7f3e2819b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:28.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.712+0000 7f3e2de00640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e28103b10 0x7f3e2819b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.712+0000 7f3e2de00640 1 -- 172.21.15.90:0/1766385729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e281a1230 con 0x7f3e2810cf70 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.712+0000 7f3e2d400640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e28103b10 0x7f3e2819b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.712+0000 7f3e2de00640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2810cf70 0x7f3e281a0a60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000eaa0 tx=0x7f3e2000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.713+0000 7f3e26000640 1 -- 172.21.15.90:0/1766385729 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2000cdb0 con 0x7f3e2810cf70 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.713+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e281a1520 con 0x7f3e2810cf70 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.713+0000 7f3e26000640 1 -- 172.21.15.90:0/1766385729 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e20005020 con 0x7f3e2810cf70 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.713+0000 7f3e26000640 1 -- 172.21.15.90:0/1766385729 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e200116d0 con 0x7f3e2810cf70 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.713+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e281a7be0 con 0x7f3e2810cf70 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.714+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e28103ef0 con 0x7f3e2810cf70 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.716+0000 7f3e26000640 1 -- 172.21.15.90:0/1766385729 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f3e2000f040 con 0x7f3e2810cf70 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.718+0000 7f3e26000640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3e100b53e0 0x7f3e100b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.719+0000 7f3e26000640 1 -- 172.21.15.90:0/1766385729 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f3e200e56b0 con 0x7f3e2810cf70 2024-09-10T22:00:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.719+0000 7f3e2d400640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3e100b53e0 0x7f3e100b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.720+0000 7f3e2d400640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3e100b53e0 0x7f3e100b78a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c005d00 tx=0x7f3e1c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.722+0000 7f3e26000640 1 -- 172.21.15.90:0/1766385729 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3e20016030 con 0x7f3e2810cf70 2024-09-10T22:00:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:28 smithi154 ceph-mon[34495]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:28 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:28.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:28 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:28.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:28.918+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3e281a7ee0 con 0x7f3e2810cf70 2024-09-10T22:00:28.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:28 smithi171 ceph-mon[34138]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:28.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:28 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:28.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:28 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.800+0000 7f3e26000640 1 -- 172.21.15.90:0/1766385729 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f3e200ac250 con 0x7f3e2810cf70 2024-09-10T22:00:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' created 2024-09-10T22:00:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3e100b53e0 msgr2=0x7f3e100b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3e100b53e0 0x7f3e100b78a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c005d00 tx=0x7f3e1c00b410 comp rx=0 tx=0).stop 2024-09-10T22:00:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2810cf70 msgr2=0x7f3e281a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2810cf70 0x7f3e281a0a60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000eaa0 tx=0x7f3e2000ef60 comp rx=0 tx=0).stop 2024-09-10T22:00:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 shutdown_connections 2024-09-10T22:00:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3e100b53e0 0x7f3e100b78a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e2810cf70 0x7f3e281a0a60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3e28104430 0x7f3e2819b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 --2- 172.21.15.90:0/1766385729 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3e28103b10 0x7f3e2819b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.803+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 >> 172.21.15.90:0/1766385729 conn(0x7f3e280fd5b0 msgr2=0x7f3e2810f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.804+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 shutdown_connections 2024-09-10T22:00:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:29.804+0000 7f3e2e800640 1 -- 172.21.15.90:0/1766385729 wait complete. 2024-09-10T22:00:29.943 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:29.943 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:00:29.943 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:29.944 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:29 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1766385729' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:00:29.944 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:00:29.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:29.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:00:29.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:29.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:29 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1766385729' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:00:29.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:00:30.053 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:30.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:00:30.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:30.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:29 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1766385729' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T22:00:30.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:00:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:30 smithi171 ceph-mon[34138]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:30 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1766385729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:00:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:30 smithi171 ceph-mon[34138]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:00:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:30 smithi090 ceph-mon[30350]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:30 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1766385729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:00:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:30 smithi090 ceph-mon[30350]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:00:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:30 smithi154 ceph-mon[34495]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:30 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1766385729' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T22:00:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:30 smithi154 ceph-mon[34495]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T22:00:31.265 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T22:00:31.675 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:32.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:31 smithi090 ceph-mon[30350]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:00:32.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:31 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:31 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:31 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:31 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:31 smithi090 ceph-mon[30350]: pgmap v301: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:31 smithi154 ceph-mon[34495]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:00:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:31 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:31 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:31 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:31 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:31 smithi154 ceph-mon[34495]: pgmap v301: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:32.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:31 smithi171 ceph-mon[34138]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T22:00:32.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:31 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:31 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:31 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:31 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:32.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:31 smithi171 ceph-mon[34138]: pgmap v301: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:32.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.303+0000 7ff649a00640 1 -- 172.21.15.90:0/2066654093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 msgr2=0x7ff644115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:32.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.303+0000 7ff649a00640 1 --2- 172.21.15.90:0/2066654093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 0x7ff644115f10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009900 tx=0x7ff63c02a410 comp rx=0 tx=0).stop 2024-09-10T22:00:32.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.303+0000 7ff649a00640 1 -- 172.21.15.90:0/2066654093 shutdown_connections 2024-09-10T22:00:32.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.303+0000 7ff649a00640 1 --2- 172.21.15.90:0/2066654093 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 0x7ff644115f10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.303+0000 7ff649a00640 1 --2- 172.21.15.90:0/2066654093 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644075e60 0x7ff6441135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.303+0000 7ff649a00640 1 --2- 172.21.15.90:0/2066654093 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff644075540 0x7ff644075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.303+0000 7ff649a00640 1 -- 172.21.15.90:0/2066654093 >> 172.21.15.90:0/2066654093 conn(0x7ff6440fe830 msgr2=0x7ff644100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:32.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.303+0000 7ff649a00640 1 -- 172.21.15.90:0/2066654093 shutdown_connections 2024-09-10T22:00:32.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.303+0000 7ff649a00640 1 -- 172.21.15.90:0/2066654093 wait complete. 2024-09-10T22:00:32.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.304+0000 7ff649a00640 1 Processor -- start 2024-09-10T22:00:32.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.304+0000 7ff649a00640 1 -- start start 2024-09-10T22:00:32.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff649a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff644075540 0x7ff64419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:32.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff649a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644075e60 0x7ff64419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:32.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff649a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 0x7ff6441a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:32.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff649a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff644119230 con 0x7ff644113b20 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff649a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441190b0 con 0x7ff644075e60 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff649a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff6441193b0 con 0x7ff644075540 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff643400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 0x7ff6441a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff642000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644075e60 0x7ff64419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff643400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 0x7ff6441a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56218/0 (socket says 172.21.15.90:56218) 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff642a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff644075540 0x7ff64419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.305+0000 7ff643400640 1 -- 172.21.15.90:0/490645590 learned_addr learned my addr 172.21.15.90:0/490645590 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff643400640 1 -- 172.21.15.90:0/490645590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff644075540 msgr2=0x7ff64419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff643400640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff644075540 0x7ff64419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff643400640 1 -- 172.21.15.90:0/490645590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644075e60 msgr2=0x7ff64419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff643400640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644075e60 0x7ff64419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff643400640 1 -- 172.21.15.90:0/490645590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6441a5870 con 0x7ff644113b20 2024-09-10T22:00:32.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff643400640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 0x7ff6441a50a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0057c0 tx=0x7ff63c0318f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:32.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff63b400640 1 -- 172.21.15.90:0/490645590 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c03f030 con 0x7ff644113b20 2024-09-10T22:00:32.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff63b400640 1 -- 172.21.15.90:0/490645590 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff63c004350 con 0x7ff644113b20 2024-09-10T22:00:32.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6441a5b00 con 0x7ff644113b20 2024-09-10T22:00:32.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff63b400640 1 -- 172.21.15.90:0/490645590 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c035070 con 0x7ff644113b20 2024-09-10T22:00:32.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.306+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6441ac220 con 0x7ff644113b20 2024-09-10T22:00:32.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.307+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff644075920 con 0x7ff644113b20 2024-09-10T22:00:32.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.310+0000 7ff63b400640 1 -- 172.21.15.90:0/490645590 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7ff63c003b10 con 0x7ff644113b20 2024-09-10T22:00:32.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.311+0000 7ff63b400640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6280b5330 0x7ff6280b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.311+0000 7ff63b400640 1 -- 172.21.15.90:0/490645590 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7712+0+0 (secure 0 0 0) 0x7ff63c106480 con 0x7ff644113b20 2024-09-10T22:00:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.311+0000 7ff642a00640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6280b5330 0x7ff6280b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:32.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.312+0000 7ff642a00640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6280b5330 0x7ff6280b77f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff634005fd0 tx=0x7ff634009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:32.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.313+0000 7ff63b400640 1 -- 172.21.15.90:0/490645590 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff63c0cced0 con 0x7ff644113b20 2024-09-10T22:00:32.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:32.519+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7ff6441ac480 con 0x7ff644113b20 2024-09-10T22:00:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:32 smithi090 ceph-mon[30350]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:00:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:32 smithi090 ceph-mon[30350]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:00:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:32 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:32 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:32 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:32 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:00:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:32 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:33.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:32 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/490645590' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-10T22:00:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:32 smithi154 ceph-mon[34495]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:00:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:32 smithi154 ceph-mon[34495]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:00:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:32 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:32 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:32 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:33.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:32 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:00:33.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:32 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:33.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:32 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/490645590' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-10T22:00:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:32 smithi171 ceph-mon[34138]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T22:00:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:32 smithi171 ceph-mon[34138]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:00:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:32 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:32 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:32 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:00:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:32 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:00:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:32 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:00:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:32 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/490645590' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-09-10T22:00:33.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.465+0000 7ff63b400640 1 -- 172.21.15.90:0/490645590 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7ff63c0d1900 con 0x7ff644113b20 2024-09-10T22:00:33.467 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' created 2024-09-10T22:00:33.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6280b5330 msgr2=0x7ff6280b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6280b5330 0x7ff6280b77f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff634005fd0 tx=0x7ff634009500 comp rx=0 tx=0).stop 2024-09-10T22:00:33.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 msgr2=0x7ff6441a50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:33.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 0x7ff6441a50a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0057c0 tx=0x7ff63c0318f0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 shutdown_connections 2024-09-10T22:00:33.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff6280b5330 0x7ff6280b77f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff644113b20 0x7ff6441a50a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff644075e60 0x7ff64419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 --2- 172.21.15.90:0/490645590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff644075540 0x7ff64419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:33.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 >> 172.21.15.90:0/490645590 conn(0x7ff6440fe830 msgr2=0x7ff6440ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:33.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 shutdown_connections 2024-09-10T22:00:33.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:33.468+0000 7ff649a00640 1 -- 172.21.15.90:0/490645590 wait complete. 2024-09-10T22:00:34.360 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T22:00:34.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:34 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/490645590' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T22:00:34.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:34 smithi090 ceph-mon[30350]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:00:34.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:34 smithi090 ceph-mon[30350]: pgmap v304: 97 pgs: 10 creating+peering, 75 unknown, 12 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:34.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:34 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/490645590' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T22:00:34.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:34 smithi171 ceph-mon[34138]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:00:34.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:34 smithi171 ceph-mon[34138]: pgmap v304: 97 pgs: 10 creating+peering, 75 unknown, 12 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:34.752 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:34.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:34 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/490645590' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T22:00:34.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:34 smithi154 ceph-mon[34495]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T22:00:34.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:34 smithi154 ceph-mon[34495]: pgmap v304: 97 pgs: 10 creating+peering, 75 unknown, 12 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.359+0000 7fe5b7400640 1 -- 172.21.15.90:0/1177353697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5b01087d0 msgr2=0x7fe5b0108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.359+0000 7fe5b7400640 1 --2- 172.21.15.90:0/1177353697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5b01087d0 0x7fe5b0108c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009900 tx=0x7fe59c02a410 comp rx=0 tx=0).stop 2024-09-10T22:00:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.359+0000 7fe5b7400640 1 -- 172.21.15.90:0/1177353697 shutdown_connections 2024-09-10T22:00:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.359+0000 7fe5b7400640 1 --2- 172.21.15.90:0/1177353697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5b0109150 0x7fe5b010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.359+0000 7fe5b7400640 1 --2- 172.21.15.90:0/1177353697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5b01087d0 0x7fe5b0108c10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.359+0000 7fe5b7400640 1 --2- 172.21.15.90:0/1177353697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5b0102dc0 0x7fe5b01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.359+0000 7fe5b7400640 1 -- 172.21.15.90:0/1177353697 >> 172.21.15.90:0/1177353697 conn(0x7fe5b00fe830 msgr2=0x7fe5b0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.360+0000 7fe5b7400640 1 -- 172.21.15.90:0/1177353697 shutdown_connections 2024-09-10T22:00:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.360+0000 7fe5b7400640 1 -- 172.21.15.90:0/1177353697 wait complete. 2024-09-10T22:00:35.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b7400640 1 Processor -- start 2024-09-10T22:00:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b7400640 1 -- start start 2024-09-10T22:00:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b7400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5b0102dc0 0x7fe5b00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b7400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5b01087d0 0x7fe5b00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5b0109150 0x7fe5b0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b7400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0112cc0 con 0x7fe5b0109150 2024-09-10T22:00:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b7400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0112b40 con 0x7fe5b01087d0 2024-09-10T22:00:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b7400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0112e40 con 0x7fe5b0102dc0 2024-09-10T22:00:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5b0102dc0 0x7fe5b00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b5400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5b0109150 0x7fe5b0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5b0102dc0 0x7fe5b00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:52668/0 (socket says 172.21.15.90:52668) 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.361+0000 7fe5b4a00640 1 -- 172.21.15.90:0/2154238144 learned_addr learned my addr 172.21.15.90:0/2154238144 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5afe00640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5b01087d0 0x7fe5b00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5b4a00640 1 -- 172.21.15.90:0/2154238144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5b01087d0 msgr2=0x7fe5b00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5b4a00640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5b01087d0 0x7fe5b00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5b4a00640 1 -- 172.21.15.90:0/2154238144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5b0109150 msgr2=0x7fe5b0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5b4a00640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5b0109150 0x7fe5b0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5b4a00640 1 -- 172.21.15.90:0/2154238144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b01a75b0 con 0x7fe5b0102dc0 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5b4a00640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5b0102dc0 0x7fe5b00754a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000db80 tx=0x7fe5a000b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:35.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5ad600640 1 -- 172.21.15.90:0/2154238144 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0017070 con 0x7fe5b0102dc0 2024-09-10T22:00:35.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5ad600640 1 -- 172.21.15.90:0/2154238144 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5a0004020 con 0x7fe5b0102dc0 2024-09-10T22:00:35.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.362+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b01a78a0 con 0x7fe5b0102dc0 2024-09-10T22:00:35.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.363+0000 7fe5ad600640 1 -- 172.21.15.90:0/2154238144 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000be60 con 0x7fe5b0102dc0 2024-09-10T22:00:35.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.363+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01a7db0 con 0x7fe5b0102dc0 2024-09-10T22:00:35.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.364+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b01031a0 con 0x7fe5b0102dc0 2024-09-10T22:00:35.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.366+0000 7fe5ad600640 1 -- 172.21.15.90:0/2154238144 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fe5a0003b70 con 0x7fe5b0102dc0 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.367+0000 7fe5ad600640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe5940b5330 0x7fe5940b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.368+0000 7fe5ad600640 1 -- 172.21.15.90:0/2154238144 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8094+0+0 (secure 0 0 0) 0x7fe5a00e5dc0 con 0x7fe5b0102dc0 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.368+0000 7fe5afe00640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe5940b5330 0x7fe5940b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:35.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.368+0000 7fe5afe00640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe5940b5330 0x7fe5940b77f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe59c000c00 tx=0x7fe59c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:35.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.369+0000 7fe5ad600640 1 -- 172.21.15.90:0/2154238144 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe5a0020070 con 0x7fe5b0102dc0 2024-09-10T22:00:35.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:35.593+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe5b002e5c0 con 0x7fe5b0102dc0 2024-09-10T22:00:35.595 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:35 smithi090 ceph-mon[30350]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:00:35.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:35 smithi171 ceph-mon[34138]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:00:35.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:35 smithi154 ceph-mon[34495]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T22:00:36.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.495+0000 7fe5ad600640 1 -- 172.21.15.90:0/2154238144 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fe5a00accc0 con 0x7fe5b0102dc0 2024-09-10T22:00:36.497 INFO:teuthology.orchestra.run.smithi090.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T22:00:36.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe5940b5330 msgr2=0x7fe5940b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe5940b5330 0x7fe5940b77f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe59c000c00 tx=0x7fe59c005a20 comp rx=0 tx=0).stop 2024-09-10T22:00:36.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5b0102dc0 msgr2=0x7fe5b00754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:36.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5b0102dc0 0x7fe5b00754a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000db80 tx=0x7fe5a000b630 comp rx=0 tx=0).stop 2024-09-10T22:00:36.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 shutdown_connections 2024-09-10T22:00:36.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe5940b5330 0x7fe5940b77f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5b0109150 0x7fe5b0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe5b01087d0 0x7fe5b00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 --2- 172.21.15.90:0/2154238144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe5b0102dc0 0x7fe5b00754a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:36.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 >> 172.21.15.90:0/2154238144 conn(0x7fe5b00fe830 msgr2=0x7fe5b0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:36.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 shutdown_connections 2024-09-10T22:00:36.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:36.498+0000 7fe5b7400640 1 -- 172.21.15.90:0/2154238144 wait complete. 2024-09-10T22:00:36.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:36 smithi171 ceph-mon[34138]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T22:00:36.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:36 smithi171 ceph-mon[34138]: pgmap v307: 97 pgs: 33 creating+peering, 13 unknown, 51 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:36.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:36 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2154238144' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:00:36.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:36 smithi171 ceph-mon[34138]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:00:36.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:36 smithi171 ceph-mon[34138]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T22:00:36.802 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:36 smithi090 ceph-mon[30350]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T22:00:36.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:36 smithi090 ceph-mon[30350]: pgmap v307: 97 pgs: 33 creating+peering, 13 unknown, 51 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:36.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:36 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2154238144' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:00:36.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:36 smithi090 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:00:36.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:36 smithi090 ceph-mon[30350]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T22:00:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:36 smithi154 ceph-mon[34495]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T22:00:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:36 smithi154 ceph-mon[34495]: pgmap v307: 97 pgs: 33 creating+peering, 13 unknown, 51 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:36 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2154238144' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:00:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:36 smithi154 ceph-mon[34495]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T22:00:36.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:36 smithi154 ceph-mon[34495]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T22:00:37.372 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T22:00:37.372 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T22:00:37.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:37 smithi090 ceph-mon[30350]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T22:00:37.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:37 smithi090 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:00:37.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:37 smithi090 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T22:00:37.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:37 smithi090 ceph-mon[30350]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T22:00:37.751 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:37 smithi154 ceph-mon[34495]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T22:00:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:37 smithi154 ceph-mon[34495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:00:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:37 smithi154 ceph-mon[34495]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T22:00:37.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:37 smithi154 ceph-mon[34495]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T22:00:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:37 smithi171 ceph-mon[34138]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T22:00:37.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:37 smithi171 ceph-mon[34138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T22:00:37.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:37 smithi171 ceph-mon[34138]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T22:00:37.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:37 smithi171 ceph-mon[34138]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.318+0000 7f7003200640 1 -- 172.21.15.90:0/1799168621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 msgr2=0x7f6ffc0758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.318+0000 7f7003200640 1 --2- 172.21.15.90:0/1799168621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 0x7f6ffc0758a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009ca0 tx=0x7f6fec02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.318+0000 7f7003200640 1 -- 172.21.15.90:0/1799168621 shutdown_connections 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.318+0000 7f7003200640 1 --2- 172.21.15.90:0/1799168621 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ffc10f3d0 0x7f6ffc1117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.318+0000 7f7003200640 1 --2- 172.21.15.90:0/1799168621 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ffc075de0 0x7f6ffc10ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.318+0000 7f7003200640 1 --2- 172.21.15.90:0/1799168621 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 0x7f6ffc0758a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.318+0000 7f7003200640 1 -- 172.21.15.90:0/1799168621 >> 172.21.15.90:0/1799168621 conn(0x7f6ffc0fe830 msgr2=0x7f6ffc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.318+0000 7f7003200640 1 -- 172.21.15.90:0/1799168621 shutdown_connections 2024-09-10T22:00:38.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.318+0000 7f7003200640 1 -- 172.21.15.90:0/1799168621 wait complete. 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.319+0000 7f7003200640 1 Processor -- start 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.319+0000 7f7003200640 1 -- start start 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.319+0000 7f7003200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 0x7f6ffc19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.319+0000 7f7003200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ffc075de0 0x7f6ffc19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.319+0000 7f7003200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ffc10f3d0 0x7f6ffc1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.319+0000 7f7003200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc114ec0 con 0x7f6ffc0754c0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.319+0000 7f7003200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc114d40 con 0x7f6ffc075de0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.319+0000 7f7003200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc115040 con 0x7f6ffc10f3d0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.319+0000 7f6ffbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 0x7f6ffc19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 0x7f6ffc19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55186/0 (socket says 172.21.15.90:55186) 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffbe00640 1 -- 172.21.15.90:0/837713588 learned_addr learned my addr 172.21.15.90:0/837713588 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f7001200640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ffc10f3d0 0x7f6ffc1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffb400640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ffc075de0 0x7f6ffc19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffbe00640 1 -- 172.21.15.90:0/837713588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ffc10f3d0 msgr2=0x7f6ffc1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffbe00640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ffc10f3d0 0x7f6ffc1a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffbe00640 1 -- 172.21.15.90:0/837713588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ffc075de0 msgr2=0x7f6ffc19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffbe00640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ffc075de0 0x7f6ffc19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffbe00640 1 -- 172.21.15.90:0/837713588 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc1a1440 con 0x7f6ffc0754c0 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f7001200640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ffc10f3d0 0x7f6ffc1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:38.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffb400640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ffc075de0 0x7f6ffc19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.320+0000 7f6ffbe00640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 0x7f6ffc19b390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009dc0 tx=0x7f6fec032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.321+0000 7f6ff8c00640 1 -- 172.21.15.90:0/837713588 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec004040 con 0x7f6ffc0754c0 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.321+0000 7f6ff8c00640 1 -- 172.21.15.90:0/837713588 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6fec004670 con 0x7f6ffc0754c0 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.321+0000 7f6ff8c00640 1 -- 172.21.15.90:0/837713588 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fec030c80 con 0x7f6ffc0754c0 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.321+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc1a16d0 con 0x7f6ffc0754c0 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.321+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc1a7df0 con 0x7f6ffc0754c0 2024-09-10T22:00:38.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.322+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ffc0758a0 con 0x7f6ffc0754c0 2024-09-10T22:00:38.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.324+0000 7f6ff8c00640 1 -- 172.21.15.90:0/837713588 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f6fec0041e0 con 0x7f6ffc0754c0 2024-09-10T22:00:38.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.326+0000 7f6ff8c00640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6fe00b5430 0x7f6fe00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:38.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.326+0000 7f6ff8c00640 1 -- 172.21.15.90:0/837713588 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7f6fec1070f0 con 0x7f6ffc0754c0 2024-09-10T22:00:38.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.326+0000 7f6ffb400640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6fe00b5430 0x7f6fe00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:38.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.326+0000 7f6ffb400640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6fe00b5430 0x7f6fe00b78f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc101070 tx=0x7f6fe8005d90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:38.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.328+0000 7f6ff8c00640 1 -- 172.21.15.90:0/837713588 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6fec0cdf70 con 0x7f6ffc0754c0 2024-09-10T22:00:38.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:38.523+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f6ffc19c4c0 con 0x7f6ffc0754c0 2024-09-10T22:00:38.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:38 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] up:active 2024-09-10T22:00:38.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:38 smithi090 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:00:38.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:38 smithi090 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:00:38.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:38 smithi090 ceph-mon[30350]: pgmap v309: 97 pgs: 27 creating+peering, 9 unknown, 61 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:38.918 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:00:38 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b[65608]: 2024-09-10T22:00:38.699+0000 7f3f85c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T22:00:38.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:38 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] up:active 2024-09-10T22:00:38.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:38 smithi154 ceph-mon[34495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:00:38.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:38 smithi154 ceph-mon[34495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:00:38.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:38 smithi154 ceph-mon[34495]: pgmap v309: 97 pgs: 27 creating+peering, 9 unknown, 61 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:38 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] up:active 2024-09-10T22:00:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:38 smithi171 ceph-mon[34138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:00:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:38 smithi171 ceph-mon[34138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:00:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:38 smithi171 ceph-mon[34138]: pgmap v309: 97 pgs: 27 creating+peering, 9 unknown, 61 active+clean; 577 KiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.509+0000 7f6ff8c00640 1 -- 172.21.15.90:0/837713588 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7f6fec004820 con 0x7f6ffc0754c0 2024-09-10T22:00:39.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6fe00b5430 msgr2=0x7f6fe00b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:39.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6fe00b5430 0x7f6fe00b78f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc101070 tx=0x7f6fe8005d90 comp rx=0 tx=0).stop 2024-09-10T22:00:39.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 msgr2=0x7f6ffc19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:39.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 0x7f6ffc19b390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6fec009dc0 tx=0x7f6fec032bc0 comp rx=0 tx=0).stop 2024-09-10T22:00:39.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 shutdown_connections 2024-09-10T22:00:39.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6fe00b5430 0x7f6fe00b78f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:39.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6ffc10f3d0 0x7f6ffc1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:39.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6ffc075de0 0x7f6ffc19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:39.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 --2- 172.21.15.90:0/837713588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ffc0754c0 0x7f6ffc19b390 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:39.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.512+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 >> 172.21.15.90:0/837713588 conn(0x7f6ffc0fe830 msgr2=0x7f6ffc1115b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.513+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 shutdown_connections 2024-09-10T22:00:39.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:39.513+0000 7f7003200640 1 -- 172.21.15.90:0/837713588 wait complete. 2024-09-10T22:00:39.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:39 smithi090 ceph-mon[30350]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:00:39.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:39 smithi090 ceph-mon[30350]: Cluster is now healthy 2024-09-10T22:00:39.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:39 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/837713588' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-10T22:00:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:39 smithi154 ceph-mon[34495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:00:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:39 smithi154 ceph-mon[34495]: Cluster is now healthy 2024-09-10T22:00:39.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:39 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/837713588' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-10T22:00:39.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:39 smithi171 ceph-mon[34138]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T22:00:39.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:39 smithi171 ceph-mon[34138]: Cluster is now healthy 2024-09-10T22:00:39.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:39 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/837713588' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-09-10T22:00:40.473 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T22:00:40.844 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:40 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/837713588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T22:00:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:40 smithi090 ceph-mon[30350]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T22:00:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:40 smithi090 ceph-mon[30350]: pgmap v311: 97 pgs: 97 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-09-10T22:00:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:40 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/837713588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T22:00:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:40 smithi154 ceph-mon[34495]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T22:00:40.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:40 smithi154 ceph-mon[34495]: pgmap v311: 97 pgs: 97 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-09-10T22:00:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:40 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/837713588' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T22:00:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:40 smithi171 ceph-mon[34138]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T22:00:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:40 smithi171 ceph-mon[34138]: pgmap v311: 97 pgs: 97 active+clean; 577 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 3 op/s 2024-09-10T22:00:41.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.432+0000 7fb517200640 1 -- 172.21.15.90:0/521192441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb51810b1a0 msgr2=0x7fb5181119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:41.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.432+0000 7fb517200640 1 --2- 172.21.15.90:0/521192441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb51810b1a0 0x7fb5181119a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb510009ca0 tx=0x7fb51002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:41.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.433+0000 7fb517200640 1 -- 172.21.15.90:0/521192441 shutdown_connections 2024-09-10T22:00:41.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.433+0000 7fb517200640 1 --2- 172.21.15.90:0/521192441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb51810b1a0 0x7fb5181119a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:41.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.433+0000 7fb517200640 1 --2- 172.21.15.90:0/521192441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb51810a820 0x7fb51810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:41.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.433+0000 7fb517200640 1 --2- 172.21.15.90:0/521192441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb518104e10 0x7fb5181051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:41.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.433+0000 7fb517200640 1 -- 172.21.15.90:0/521192441 >> 172.21.15.90:0/521192441 conn(0x7fb518100840 msgr2=0x7fb518102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.433+0000 7fb517200640 1 -- 172.21.15.90:0/521192441 shutdown_connections 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.433+0000 7fb517200640 1 -- 172.21.15.90:0/521192441 wait complete. 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb517200640 1 Processor -- start 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb517200640 1 -- start start 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb517200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb518104e10 0x7fb51819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb517200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb51810a820 0x7fb51819b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb517200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb51810b1a0 0x7fb5181a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb517200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb518114d20 con 0x7fb518104e10 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb517200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb518114ba0 con 0x7fb51810a820 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb517200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb518114ea0 con 0x7fb51810b1a0 2024-09-10T22:00:41.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb515e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb518104e10 0x7fb51819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb515e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb518104e10 0x7fb51819b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55208/0 (socket says 172.21.15.90:55208) 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.434+0000 7fb515e00640 1 -- 172.21.15.90:0/1709848397 learned_addr learned my addr 172.21.15.90:0/1709848397 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb516800640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb51810b1a0 0x7fb5181a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb50fe00640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb51810a820 0x7fb51819b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb515e00640 1 -- 172.21.15.90:0/1709848397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb51810b1a0 msgr2=0x7fb5181a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb515e00640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb51810b1a0 0x7fb5181a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb515e00640 1 -- 172.21.15.90:0/1709848397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb51810a820 msgr2=0x7fb51819b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb515e00640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb51810a820 0x7fb51819b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb515e00640 1 -- 172.21.15.90:0/1709848397 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5181a11c0 con 0x7fb518104e10 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb515e00640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb518104e10 0x7fb51819b110 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c920 tx=0x7fb50800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb50ea00640 1 -- 172.21.15.90:0/1709848397 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508007c70 con 0x7fb518104e10 2024-09-10T22:00:41.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb50ea00640 1 -- 172.21.15.90:0/1709848397 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb508007e10 con 0x7fb518104e10 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5181a14b0 con 0x7fb518104e10 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.435+0000 7fb50ea00640 1 -- 172.21.15.90:0/1709848397 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508018710 con 0x7fb518104e10 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.436+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5181a7d40 con 0x7fb518104e10 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.437+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5181051f0 con 0x7fb518104e10 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.439+0000 7fb50ea00640 1 -- 172.21.15.90:0/1709848397 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fb50800f450 con 0x7fb518104e10 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.440+0000 7fb50ea00640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4fc0b5460 0x7fb4fc0b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.440+0000 7fb50ea00640 1 -- 172.21.15.90:0/1709848397 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8492+0+0 (secure 0 0 0) 0x7fb5080e5f80 con 0x7fb518104e10 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.440+0000 7fb50fe00640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4fc0b5460 0x7fb4fc0b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.441+0000 7fb50fe00640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4fc0b5460 0x7fb4fc0b7920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb518103230 tx=0x7fb500005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:41.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.444+0000 7fb50ea00640 1 -- 172.21.15.90:0/1709848397 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb508014030 con 0x7fb518104e10 2024-09-10T22:00:41.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:41.630+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb51819c240 con 0x7fb518104e10 2024-09-10T22:00:41.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:41 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1709848397' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:00:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:41 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1709848397' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:00:42.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:41 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1709848397' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T22:00:42.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.686+0000 7fb50ea00640 1 -- 172.21.15.90:0/1709848397 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7fb5080acce0 con 0x7fb518104e10 2024-09-10T22:00:42.688 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data_ec' created 2024-09-10T22:00:42.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.688+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4fc0b5460 msgr2=0x7fb4fc0b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.688+0000 7fb517200640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4fc0b5460 0x7fb4fc0b7920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb518103230 tx=0x7fb500005f90 comp rx=0 tx=0).stop 2024-09-10T22:00:42.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.688+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb518104e10 msgr2=0x7fb51819b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:42.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.688+0000 7fb517200640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb518104e10 0x7fb51819b110 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb50800c920 tx=0x7fb50800cde0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.689+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 shutdown_connections 2024-09-10T22:00:42.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.689+0000 7fb517200640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4fc0b5460 0x7fb4fc0b7920 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.689+0000 7fb517200640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb51810b1a0 0x7fb5181a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.689+0000 7fb517200640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb51810a820 0x7fb51819b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.689+0000 7fb517200640 1 --2- 172.21.15.90:0/1709848397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb518104e10 0x7fb51819b110 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:42.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.689+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 >> 172.21.15.90:0/1709848397 conn(0x7fb518100840 msgr2=0x7fb518103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:42.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.689+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 shutdown_connections 2024-09-10T22:00:42.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:42.690+0000 7fb517200640 1 -- 172.21.15.90:0/1709848397 wait complete. 2024-09-10T22:00:43.007 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:42 smithi171 ceph-mon[34138]: pgmap v312: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:00:43.007 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:42 smithi171 ceph-mon[34138]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T22:00:43.007 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:42 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1709848397' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:00:43.007 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:42 smithi171 ceph-mon[34138]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T22:00:43.072 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:42 smithi154 ceph-mon[34495]: pgmap v312: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:00:43.072 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:42 smithi154 ceph-mon[34495]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T22:00:43.072 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:42 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1709848397' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:00:43.072 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:42 smithi154 ceph-mon[34495]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T22:00:43.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:42 smithi090 ceph-mon[30350]: pgmap v312: 97 pgs: 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:00:43.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:42 smithi090 ceph-mon[30350]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T22:00:43.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:42 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1709848397' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T22:00:43.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:42 smithi090 ceph-mon[30350]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T22:00:43.621 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T22:00:44.014 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:44.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.632+0000 7fee52600640 1 -- 172.21.15.90:0/3255108604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c104fc0 msgr2=0x7fee4c1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.632+0000 7fee52600640 1 --2- 172.21.15.90:0/3255108604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c104fc0 0x7fee4c1053a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009ca0 tx=0x7fee3c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.632+0000 7fee52600640 1 -- 172.21.15.90:0/3255108604 shutdown_connections 2024-09-10T22:00:44.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.632+0000 7fee52600640 1 --2- 172.21.15.90:0/3255108604 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee4c10b350 0x7fee4c111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.632+0000 7fee52600640 1 --2- 172.21.15.90:0/3255108604 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee4c10a9d0 0x7fee4c10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.632+0000 7fee52600640 1 --2- 172.21.15.90:0/3255108604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c104fc0 0x7fee4c1053a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.632+0000 7fee52600640 1 -- 172.21.15.90:0/3255108604 >> 172.21.15.90:0/3255108604 conn(0x7fee4c1009f0 msgr2=0x7fee4c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:44.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.632+0000 7fee52600640 1 -- 172.21.15.90:0/3255108604 shutdown_connections 2024-09-10T22:00:44.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.632+0000 7fee52600640 1 -- 172.21.15.90:0/3255108604 wait complete. 2024-09-10T22:00:44.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.633+0000 7fee52600640 1 Processor -- start 2024-09-10T22:00:44.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.633+0000 7fee52600640 1 -- start start 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.633+0000 7fee52600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee4c104fc0 0x7fee4c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.633+0000 7fee52600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee4c10a9d0 0x7fee4c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.633+0000 7fee52600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c10b350 0x7fee4c1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.633+0000 7fee52600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c114e80 con 0x7fee4c10b350 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.633+0000 7fee52600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c114d00 con 0x7fee4c104fc0 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.633+0000 7fee52600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c115000 con 0x7fee4c10a9d0 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c10b350 0x7fee4c1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee4c104fc0 0x7fee4c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c10b350 0x7fee4c1a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55232/0 (socket says 172.21.15.90:55232) 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4be00640 1 -- 172.21.15.90:0/641225253 learned_addr learned my addr 172.21.15.90:0/641225253 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:44.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee42a00640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee4c10a9d0 0x7fee4c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4be00640 1 -- 172.21.15.90:0/641225253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee4c10a9d0 msgr2=0x7fee4c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4be00640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee4c10a9d0 0x7fee4c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4be00640 1 -- 172.21.15.90:0/641225253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee4c104fc0 msgr2=0x7fee4c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4be00640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee4c104fc0 0x7fee4c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4be00640 1 -- 172.21.15.90:0/641225253 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c1a14c0 con 0x7fee4c10b350 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.634+0000 7fee4b400640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee4c104fc0 0x7fee4c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.635+0000 7fee4be00640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c10b350 0x7fee4c1a0cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fee4400d750 tx=0x7fee4400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.635+0000 7fee48c00640 1 -- 172.21.15.90:0/641225253 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44017070 con 0x7fee4c10b350 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.635+0000 7fee48c00640 1 -- 172.21.15.90:0/641225253 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee440054b0 con 0x7fee4c10b350 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.635+0000 7fee48c00640 1 -- 172.21.15.90:0/641225253 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44004b90 con 0x7fee4c10b350 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.635+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c1a17b0 con 0x7fee4c10b350 2024-09-10T22:00:44.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.635+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c1a7e70 con 0x7fee4c10b350 2024-09-10T22:00:44.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.636+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee4c1053a0 con 0x7fee4c10b350 2024-09-10T22:00:44.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.639+0000 7fee48c00640 1 -- 172.21.15.90:0/641225253 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fee44004d30 con 0x7fee4c10b350 2024-09-10T22:00:44.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.640+0000 7fee48c00640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fee300b53b0 0x7fee300b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:44.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.640+0000 7fee48c00640 1 -- 172.21.15.90:0/641225253 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8985+0+0 (secure 0 0 0) 0x7fee440e5be0 con 0x7fee4c10b350 2024-09-10T22:00:44.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.640+0000 7fee4b400640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fee300b53b0 0x7fee300b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:44.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.641+0000 7fee4b400640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fee300b53b0 0x7fee300b7870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee3c006010 tx=0x7fee3c031000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:44.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.642+0000 7fee48c00640 1 -- 172.21.15.90:0/641225253 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fee440ac180 con 0x7fee4c10b350 2024-09-10T22:00:44.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:44.832+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fee4c02e5c0 con 0x7fee4c10b350 2024-09-10T22:00:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:44 smithi154 ceph-mon[34495]: pgmap v315: 129 pgs: 8 creating+peering, 24 unknown, 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:00:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:44 smithi154 ceph-mon[34495]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T22:00:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:44 smithi154 ceph-mon[34495]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:00:44.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:44 smithi171 ceph-mon[34138]: pgmap v315: 129 pgs: 8 creating+peering, 24 unknown, 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:00:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:44 smithi171 ceph-mon[34138]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T22:00:44.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:44 smithi171 ceph-mon[34138]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:00:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:44 smithi090 ceph-mon[30350]: pgmap v315: 129 pgs: 8 creating+peering, 24 unknown, 97 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T22:00:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:44 smithi090 ceph-mon[30350]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T22:00:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:44 smithi090 ceph-mon[30350]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T22:00:45.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.718+0000 7fee48c00640 1 -- 172.21.15.90:0/641225253 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7fee440b0bb0 con 0x7fee4c10b350 2024-09-10T22:00:45.720 INFO:teuthology.orchestra.run.smithi090.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T22:00:45.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fee300b53b0 msgr2=0x7fee300b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fee300b53b0 0x7fee300b7870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee3c006010 tx=0x7fee3c031000 comp rx=0 tx=0).stop 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c10b350 msgr2=0x7fee4c1a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c10b350 0x7fee4c1a0cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fee4400d750 tx=0x7fee4400dc10 comp rx=0 tx=0).stop 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 shutdown_connections 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fee300b53b0 0x7fee300b7870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee4c10b350 0x7fee4c1a0cf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fee4c10a9d0 0x7fee4c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 --2- 172.21.15.90:0/641225253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fee4c104fc0 0x7fee4c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 >> 172.21.15.90:0/641225253 conn(0x7fee4c1009f0 msgr2=0x7fee4c1012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 shutdown_connections 2024-09-10T22:00:45.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:45.721+0000 7fee52600640 1 -- 172.21.15.90:0/641225253 wait complete. 2024-09-10T22:00:45.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:45 smithi171 ceph-mon[34138]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T22:00:45.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:45 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/641225253' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-10T22:00:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:45 smithi090 ceph-mon[30350]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T22:00:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:45 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/641225253' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-10T22:00:46.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:45 smithi154 ceph-mon[34495]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T22:00:46.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:45 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/641225253' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-09-10T22:00:46.623 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T22:00:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:46 smithi090 ceph-mon[30350]: pgmap v318: 129 pgs: 14 creating+peering, 8 unknown, 107 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:46 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/641225253' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T22:00:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:46 smithi090 ceph-mon[30350]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T22:00:47.000 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:46 smithi171 ceph-mon[34138]: pgmap v318: 129 pgs: 14 creating+peering, 8 unknown, 107 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:47.000 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:46 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/641225253' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T22:00:47.000 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:46 smithi171 ceph-mon[34138]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T22:00:47.001 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:46 smithi154 ceph-mon[34495]: pgmap v318: 129 pgs: 14 creating+peering, 8 unknown, 107 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:46 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/641225253' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T22:00:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:46 smithi154 ceph-mon[34495]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T22:00:47.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.593+0000 7fd6b3600640 1 -- 172.21.15.90:0/2844376856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac1023d0 msgr2=0x7fd6ac1068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.593+0000 7fd6b3600640 1 --2- 172.21.15.90:0/2844376856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac1023d0 0x7fd6ac1068a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd698009900 tx=0x7fd69802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.593+0000 7fd6b3600640 1 -- 172.21.15.90:0/2844376856 shutdown_connections 2024-09-10T22:00:47.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.593+0000 7fd6b3600640 1 --2- 172.21.15.90:0/2844376856 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6ac106ef0 0x7fd6ac107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.593+0000 7fd6b3600640 1 --2- 172.21.15.90:0/2844376856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac1023d0 0x7fd6ac1068a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.593+0000 7fd6b3600640 1 --2- 172.21.15.90:0/2844376856 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6ac101ab0 0x7fd6ac101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.593+0000 7fd6b3600640 1 -- 172.21.15.90:0/2844376856 >> 172.21.15.90:0/2844376856 conn(0x7fd6ac0fd700 msgr2=0x7fd6ac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:47.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.593+0000 7fd6b3600640 1 -- 172.21.15.90:0/2844376856 shutdown_connections 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.593+0000 7fd6b3600640 1 -- 172.21.15.90:0/2844376856 wait complete. 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.594+0000 7fd6b3600640 1 Processor -- start 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.594+0000 7fd6b3600640 1 -- start start 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.594+0000 7fd6b3600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101ab0 0x7fd6ac19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.594+0000 7fd6b3600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6ac1023d0 0x7fd6ac19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.594+0000 7fd6b3600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6ac106ef0 0x7fd6ac1a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.594+0000 7fd6b3600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac1192d0 con 0x7fd6ac101ab0 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.594+0000 7fd6b3600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac119150 con 0x7fd6ac106ef0 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b3600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd6ac119450 con 0x7fd6ac1023d0 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b0c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101ab0 0x7fd6ac19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b0c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101ab0 0x7fd6ac19f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55252/0 (socket says 172.21.15.90:55252) 2024-09-10T22:00:47.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b0c00640 1 -- 172.21.15.90:0/977447934 learned_addr learned my addr 172.21.15.90:0/977447934 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b1600640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6ac106ef0 0x7fd6ac1a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b0c00640 1 -- 172.21.15.90:0/977447934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6ac1023d0 msgr2=0x7fd6ac19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b0c00640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6ac1023d0 0x7fd6ac19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b0c00640 1 -- 172.21.15.90:0/977447934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6ac106ef0 msgr2=0x7fd6ac1a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b0c00640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6ac106ef0 0x7fd6ac1a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b0c00640 1 -- 172.21.15.90:0/977447934 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6ac1a5930 con 0x7fd6ac101ab0 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b1600640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6ac106ef0 0x7fd6ac1a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.595+0000 7fd6b0c00640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101ab0 0x7fd6ac19f880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c890 tx=0x7fd6a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.596+0000 7fd6a9600640 1 -- 172.21.15.90:0/977447934 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004220 con 0x7fd6ac101ab0 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.596+0000 7fd6a9600640 1 -- 172.21.15.90:0/977447934 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6a0007e40 con 0x7fd6ac101ab0 2024-09-10T22:00:47.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.596+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6ac1a5c20 con 0x7fd6ac101ab0 2024-09-10T22:00:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.596+0000 7fd6a9600640 1 -- 172.21.15.90:0/977447934 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0004a00 con 0x7fd6ac101ab0 2024-09-10T22:00:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.596+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6ac1ac4b0 con 0x7fd6ac101ab0 2024-09-10T22:00:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.597+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6ac1068a0 con 0x7fd6ac101ab0 2024-09-10T22:00:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.599+0000 7fd6a9600640 1 -- 172.21.15.90:0/977447934 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fd6a00079b0 con 0x7fd6ac101ab0 2024-09-10T22:00:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.600+0000 7fd6a9600640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6940b53b0 0x7fd6940b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.600+0000 7fd6a9600640 1 -- 172.21.15.90:0/977447934 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8985+0+0 (secure 0 0 0) 0x7fd6a00e5cf0 con 0x7fd6ac101ab0 2024-09-10T22:00:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.601+0000 7fd6abe00640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6940b53b0 0x7fd6940b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.601+0000 7fd6abe00640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6940b53b0 0x7fd6940b7870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd698009230 tx=0x7fd698000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:47.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.604+0000 7fd6a9600640 1 -- 172.21.15.90:0/977447934 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd6a0014030 con 0x7fd6ac101ab0 2024-09-10T22:00:47.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:47.831+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fd6ac1ac790 con 0x7fd6ac101ab0 2024-09-10T22:00:48.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.724+0000 7fd6a9600640 1 -- 172.21.15.90:0/977447934 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fd6a00ac860 con 0x7fd6ac101ab0 2024-09-10T22:00:48.725 INFO:teuthology.orchestra.run.smithi090.stderr:added data pool 4 to fsmap 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6940b53b0 msgr2=0x7fd6940b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6940b53b0 0x7fd6940b7870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd698009230 tx=0x7fd698000bc0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101ab0 msgr2=0x7fd6ac19f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101ab0 0x7fd6ac19f880 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd6a000c890 tx=0x7fd6a000cd50 comp rx=0 tx=0).stop 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 shutdown_connections 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6940b53b0 0x7fd6940b7870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd6ac106ef0 0x7fd6ac1a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd6ac1023d0 0x7fd6ac19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 --2- 172.21.15.90:0/977447934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6ac101ab0 0x7fd6ac19f880 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 >> 172.21.15.90:0/977447934 conn(0x7fd6ac0fd700 msgr2=0x7fd6ac104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:48.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 shutdown_connections 2024-09-10T22:00:48.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:48.728+0000 7fd6b3600640 1 -- 172.21.15.90:0/977447934 wait complete. 2024-09-10T22:00:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:48 smithi171 ceph-mon[34138]: pgmap v320: 129 pgs: 14 creating+peering, 7 unknown, 108 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:48 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/977447934' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-10T22:00:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:48 smithi090 ceph-mon[30350]: pgmap v320: 129 pgs: 14 creating+peering, 7 unknown, 108 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:48 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/977447934' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-10T22:00:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:48 smithi154 ceph-mon[34495]: pgmap v320: 129 pgs: 14 creating+peering, 7 unknown, 108 active+clean; 579 KiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:48 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/977447934' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-09-10T22:00:49.604 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:00:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:49 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/977447934' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T22:00:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:49 smithi090 ceph-mon[30350]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T22:00:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:49 smithi090 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:00:49.977 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:49 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/977447934' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T22:00:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:49 smithi171 ceph-mon[34138]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T22:00:49.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:49 smithi171 ceph-mon[34138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:00:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:49 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/977447934' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T22:00:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:49 smithi154 ceph-mon[34495]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T22:00:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:49 smithi154 ceph-mon[34495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:00:50.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.558+0000 7f8425400640 1 -- 172.21.15.90:0/2290453077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f842010a800 msgr2=0x7f842010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.558+0000 7f8425400640 1 --2- 172.21.15.90:0/2290453077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f842010a800 0x7f842010ac40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f840c009900 tx=0x7f840c02a410 comp rx=0 tx=0).stop 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.558+0000 7f8425400640 1 -- 172.21.15.90:0/2290453077 shutdown_connections 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.558+0000 7f8425400640 1 --2- 172.21.15.90:0/2290453077 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f842010b180 0x7f8420111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.558+0000 7f8425400640 1 --2- 172.21.15.90:0/2290453077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f842010a800 0x7f842010ac40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.558+0000 7f8425400640 1 --2- 172.21.15.90:0/2290453077 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8420104df0 0x7f84201051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.558+0000 7f8425400640 1 -- 172.21.15.90:0/2290453077 >> 172.21.15.90:0/2290453077 conn(0x7f8420100840 msgr2=0x7f8420102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.558+0000 7f8425400640 1 -- 172.21.15.90:0/2290453077 shutdown_connections 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.558+0000 7f8425400640 1 -- 172.21.15.90:0/2290453077 wait complete. 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.559+0000 7f8425400640 1 Processor -- start 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.559+0000 7f8425400640 1 -- start start 2024-09-10T22:00:50.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f8425400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8420104df0 0x7f842019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f8425400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f842010a800 0x7f842019b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f8425400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f842010b180 0x7f84201a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f8425400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8420114cf0 con 0x7f8420104df0 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f8425400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8420114b70 con 0x7f842010b180 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f8425400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8420114e70 con 0x7f842010a800 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f841fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8420104df0 0x7f842019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f841fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8420104df0 0x7f842019b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52366/0 (socket says 172.21.15.90:52366) 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f841fe00640 1 -- 172.21.15.90:0/1411306493 learned_addr learned my addr 172.21.15.90:0/1411306493 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f841f400640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f842010a800 0x7f842019b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f8424a00640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f842010b180 0x7f84201a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f841fe00640 1 -- 172.21.15.90:0/1411306493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f842010a800 msgr2=0x7f842019b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:50.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f841fe00640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f842010a800 0x7f842019b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f841fe00640 1 -- 172.21.15.90:0/1411306493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f842010b180 msgr2=0x7f84201a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f841fe00640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f842010b180 0x7f84201a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f841fe00640 1 -- 172.21.15.90:0/1411306493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84201a1270 con 0x7f8420104df0 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.560+0000 7f8424a00640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f842010b180 0x7f84201a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.561+0000 7f841fe00640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8420104df0 0x7f842019b1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f841000e9f0 tx=0x7f841000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.561+0000 7f841cc00640 1 -- 172.21.15.90:0/1411306493 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841000cd80 con 0x7f8420104df0 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.561+0000 7f841cc00640 1 -- 172.21.15.90:0/1411306493 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84100054b0 con 0x7f8420104df0 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.561+0000 7f841cc00640 1 -- 172.21.15.90:0/1411306493 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410010430 con 0x7f8420104df0 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.561+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84201a1560 con 0x7f8420104df0 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.561+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201a7c20 con 0x7f8420104df0 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.562+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84201051d0 con 0x7f8420104df0 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.564+0000 7f841cc00640 1 -- 172.21.15.90:0/1411306493 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f8410005000 con 0x7f8420104df0 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.566+0000 7f841cc00640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f84040b5460 0x7f84040b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:50.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.566+0000 7f841cc00640 1 -- 172.21.15.90:0/1411306493 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f84100e6580 con 0x7f8420104df0 2024-09-10T22:00:50.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.566+0000 7f841f400640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f84040b5460 0x7f84040b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:50.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.567+0000 7f841f400640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f84040b5460 0x7f84040b7920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f840c0057c0 tx=0x7f840c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:50.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.567+0000 7f841cc00640 1 -- 172.21.15.90:0/1411306493 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8410056220 con 0x7f8420104df0 2024-09-10T22:00:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.788+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f842002e5c0 con 0x7f8420104df0 2024-09-10T22:00:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.789+0000 7f841cc00640 1 -- 172.21.15.90:0/1411306493 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7f84100ad0d0 con 0x7f8420104df0 2024-09-10T22:00:50.790 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:50.790 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"btime":"2024-09-10T22:00:48:722515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14841,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14853,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34640,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34670,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:00:47.832003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34703,"qdb_cluster":[34703]},"id":1}]} 2024-09-10T22:00:50.790 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-09-10T22:00:50.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.791+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f84040b5460 msgr2=0x7f84040b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:50.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.791+0000 7f8425400640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f84040b5460 0x7f84040b7920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f840c0057c0 tx=0x7f840c0058d0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.791+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8420104df0 msgr2=0x7f842019b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:50.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.791+0000 7f8425400640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8420104df0 0x7f842019b1c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f841000e9f0 tx=0x7f841000edf0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.792+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 shutdown_connections 2024-09-10T22:00:50.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.792+0000 7f8425400640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f84040b5460 0x7f84040b7920 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.792+0000 7f8425400640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f842010b180 0x7f84201a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.792+0000 7f8425400640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f842010a800 0x7f842019b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.792+0000 7f8425400640 1 --2- 172.21.15.90:0/1411306493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8420104df0 0x7f842019b1c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:50.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.792+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 >> 172.21.15.90:0/1411306493 conn(0x7f8420100840 msgr2=0x7f8420102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:50.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.792+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 shutdown_connections 2024-09-10T22:00:50.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:50.792+0000 7f8425400640 1 -- 172.21.15.90:0/1411306493 wait complete. 2024-09-10T22:00:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:50 smithi171 ceph-mon[34138]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:50.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:50 smithi171 ceph-mon[34138]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T22:00:50.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:50 smithi171 ceph-mon[34138]: Cluster is now healthy 2024-09-10T22:00:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:50 smithi090 ceph-mon[30350]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:50 smithi090 ceph-mon[30350]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T22:00:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:50 smithi090 ceph-mon[30350]: Cluster is now healthy 2024-09-10T22:00:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:50 smithi154 ceph-mon[34495]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:50 smithi154 ceph-mon[34495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T22:00:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:50 smithi154 ceph-mon[34495]: Cluster is now healthy 2024-09-10T22:00:51.693 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:00:51.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:51 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1411306493' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:52.062 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:51 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1411306493' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:51 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1411306493' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:52.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.711+0000 7f8ca6200640 1 -- 172.21.15.90:0/4016139663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0077ff0 msgr2=0x7f8ca0113450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.711+0000 7f8ca6200640 1 --2- 172.21.15.90:0/4016139663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0077ff0 0x7f8ca0113450 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009900 tx=0x7f8c8c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.711+0000 7f8ca6200640 1 -- 172.21.15.90:0/4016139663 shutdown_connections 2024-09-10T22:00:52.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.711+0000 7f8ca6200640 1 --2- 172.21.15.90:0/4016139663 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca0113990 0x7f8ca0115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.711+0000 7f8ca6200640 1 --2- 172.21.15.90:0/4016139663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0077ff0 0x7f8ca0113450 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.711+0000 7f8ca6200640 1 --2- 172.21.15.90:0/4016139663 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca00776d0 0x7f8ca0077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.711+0000 7f8ca6200640 1 -- 172.21.15.90:0/4016139663 >> 172.21.15.90:0/4016139663 conn(0x7f8ca0100a50 msgr2=0x7f8ca0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.711+0000 7f8ca6200640 1 -- 172.21.15.90:0/4016139663 shutdown_connections 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.711+0000 7f8ca6200640 1 -- 172.21.15.90:0/4016139663 wait complete. 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.712+0000 7f8ca6200640 1 Processor -- start 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.712+0000 7f8ca6200640 1 -- start start 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8ca6200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca00776d0 0x7f8ca019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8ca6200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca0077ff0 0x7f8ca019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8ca6200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0113990 0x7f8ca01a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8ca6200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca0119280 con 0x7f8ca0113990 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8ca6200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca0119100 con 0x7f8ca0077ff0 2024-09-10T22:00:52.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8ca6200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca0119400 con 0x7f8ca00776d0 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0113990 0x7f8ca01a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca00776d0 0x7f8ca019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0113990 0x7f8ca01a50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52380/0 (socket says 172.21.15.90:52380) 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9ea00640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca0077ff0 0x7f8ca019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9fe00640 1 -- 172.21.15.90:0/1663336039 learned_addr learned my addr 172.21.15.90:0/1663336039 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9fe00640 1 -- 172.21.15.90:0/1663336039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca00776d0 msgr2=0x7f8ca019f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9fe00640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca00776d0 0x7f8ca019f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9fe00640 1 -- 172.21.15.90:0/1663336039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca0077ff0 msgr2=0x7f8ca019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9fe00640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca0077ff0 0x7f8ca019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.713+0000 7f8c9fe00640 1 -- 172.21.15.90:0/1663336039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca01a58a0 con 0x7f8ca0113990 2024-09-10T22:00:52.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.714+0000 7f8c9fe00640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0113990 0x7f8ca01a50d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800e970 tx=0x7f8c9800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:52.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.714+0000 7f8c97e00640 1 -- 172.21.15.90:0/1663336039 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9800ccc0 con 0x7f8ca0113990 2024-09-10T22:00:52.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.714+0000 7f8c97e00640 1 -- 172.21.15.90:0/1663336039 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c980054b0 con 0x7f8ca0113990 2024-09-10T22:00:52.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.714+0000 7f8c97e00640 1 -- 172.21.15.90:0/1663336039 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c98010430 con 0x7f8ca0113990 2024-09-10T22:00:52.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.714+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca01a5b90 con 0x7f8ca0113990 2024-09-10T22:00:52.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.714+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca01ac420 con 0x7f8ca0113990 2024-09-10T22:00:52.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.716+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca0077ab0 con 0x7f8ca0113990 2024-09-10T22:00:52.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.718+0000 7f8c97e00640 1 -- 172.21.15.90:0/1663336039 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f8c98004f60 con 0x7f8ca0113990 2024-09-10T22:00:52.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.719+0000 7f8c97e00640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c840b53b0 0x7f8c840b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:52.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.719+0000 7f8c97e00640 1 -- 172.21.15.90:0/1663336039 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f8c980e6340 con 0x7f8ca0113990 2024-09-10T22:00:52.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.719+0000 7f8c9f400640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c840b53b0 0x7f8c840b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:52.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.720+0000 7f8c9f400640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c840b53b0 0x7f8c840b7870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8c90007ec0 tx=0x7f8c900075b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:52.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.723+0000 7f8c97e00640 1 -- 172.21.15.90:0/1663336039 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8c980ace90 con 0x7f8ca0113990 2024-09-10T22:00:52.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:52 smithi090 ceph-mon[30350]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:52.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.911+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ca010eb20 con 0x7f8ca0113990 2024-09-10T22:00:52.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.912+0000 7f8c97e00640 1 -- 172.21.15.90:0/1663336039 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24540 (secure 0 0 0) 0x7f8c980ac790 con 0x7f8ca0113990 2024-09-10T22:00:52.913 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:52.914 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:00:47.831865+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c840b53b0 msgr2=0x7f8c840b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c840b53b0 0x7f8c840b7870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8c90007ec0 tx=0x7f8c900075b0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0113990 msgr2=0x7f8ca01a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0113990 0x7f8ca01a50d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800e970 tx=0x7f8c9800ee30 comp rx=0 tx=0).stop 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 shutdown_connections 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c840b53b0 0x7f8c840b7870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ca0113990 0x7f8ca01a50d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8ca0077ff0 0x7f8ca019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 --2- 172.21.15.90:0/1663336039 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8ca00776d0 0x7f8ca019f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:52.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 >> 172.21.15.90:0/1663336039 conn(0x7f8ca0100a50 msgr2=0x7f8ca0102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:52.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 shutdown_connections 2024-09-10T22:00:52.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:52.915+0000 7f8ca6200640 1 -- 172.21.15.90:0/1663336039 wait complete. 2024-09-10T22:00:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:52 smithi154 ceph-mon[34495]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:53.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:52 smithi171 ceph-mon[34138]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:53.936 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:00:54.042 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1663336039' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:00:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1663336039' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:00:54.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1663336039' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:00:54.303 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:54.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:54 smithi154 ceph-mon[34495]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:54.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:54 smithi090 ceph-mon[30350]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:54.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.993+0000 7f4941c00640 1 -- 172.21.15.90:0/1882730382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c109130 msgr2=0x7f493c10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:54.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.993+0000 7f4941c00640 1 --2- 172.21.15.90:0/1882730382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c109130 0x7f493c10f930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4934009ca0 tx=0x7f493402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.993+0000 7f4941c00640 1 -- 172.21.15.90:0/1882730382 shutdown_connections 2024-09-10T22:00:54.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.993+0000 7f4941c00640 1 --2- 172.21.15.90:0/1882730382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c109130 0x7f493c10f930 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.993+0000 7f4941c00640 1 --2- 172.21.15.90:0/1882730382 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c1087b0 0x7f493c108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.993+0000 7f4941c00640 1 --2- 172.21.15.90:0/1882730382 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f493c102da0 0x7f493c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.993+0000 7f4941c00640 1 -- 172.21.15.90:0/1882730382 >> 172.21.15.90:0/1882730382 conn(0x7f493c0fe810 msgr2=0x7f493c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:54.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.993+0000 7f4941c00640 1 -- 172.21.15.90:0/1882730382 shutdown_connections 2024-09-10T22:00:54.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.993+0000 7f4941c00640 1 -- 172.21.15.90:0/1882730382 wait complete. 2024-09-10T22:00:54.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.995+0000 7f4941c00640 1 Processor -- start 2024-09-10T22:00:54.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.995+0000 7f4941c00640 1 -- start start 2024-09-10T22:00:54.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.995+0000 7f4941c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102da0 0x7f493c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:54.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.995+0000 7f4941c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c1087b0 0x7f493c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:54.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.995+0000 7f4941c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f493c109130 0x7f493c1a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:54.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.995+0000 7f4941c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f493c112cf0 con 0x7f493c1087b0 2024-09-10T22:00:54.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.995+0000 7f4941c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f493c112b70 con 0x7f493c102da0 2024-09-10T22:00:54.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.995+0000 7f4941c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f493c112e70 con 0x7f493c109130 2024-09-10T22:00:54.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c1087b0 0x7f493c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:54.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c1087b0 0x7f493c19b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52408/0 (socket says 172.21.15.90:52408) 2024-09-10T22:00:54.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493a000640 1 -- 172.21.15.90:0/1306208500 learned_addr learned my addr 172.21.15.90:0/1306208500 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:54.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493aa00640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102da0 0x7f493c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:54.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493b400640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f493c109130 0x7f493c1a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:54.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493a000640 1 -- 172.21.15.90:0/1306208500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f493c109130 msgr2=0x7f493c1a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:54.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493a000640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f493c109130 0x7f493c1a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493a000640 1 -- 172.21.15.90:0/1306208500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102da0 msgr2=0x7f493c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:54.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493a000640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102da0 0x7f493c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:54.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.996+0000 7f493a000640 1 -- 172.21.15.90:0/1306208500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c1a1400 con 0x7f493c1087b0 2024-09-10T22:00:54.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.997+0000 7f493a000640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c1087b0 0x7f493c19b890 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f492800d950 tx=0x7f492800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:54.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.997+0000 7f4933400640 1 -- 172.21.15.90:0/1306208500 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928017070 con 0x7f493c1087b0 2024-09-10T22:00:54.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.997+0000 7f4933400640 1 -- 172.21.15.90:0/1306208500 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4928009c30 con 0x7f493c1087b0 2024-09-10T22:00:55.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.997+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c1a16f0 con 0x7f493c1087b0 2024-09-10T22:00:55.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.997+0000 7f4933400640 1 -- 172.21.15.90:0/1306208500 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492800f620 con 0x7f493c1087b0 2024-09-10T22:00:55.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.997+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c1a7f80 con 0x7f493c1087b0 2024-09-10T22:00:55.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:54.998+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f493c104050 con 0x7f493c1087b0 2024-09-10T22:00:55.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.001+0000 7f4933400640 1 -- 172.21.15.90:0/1306208500 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f4928005020 con 0x7f493c1087b0 2024-09-10T22:00:55.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.003+0000 7f4933400640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f49200b53b0 0x7f49200b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:55.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.003+0000 7f4933400640 1 -- 172.21.15.90:0/1306208500 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f49280e5450 con 0x7f493c1087b0 2024-09-10T22:00:55.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.003+0000 7f493aa00640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f49200b53b0 0x7f49200b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:55.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.004+0000 7f493aa00640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f49200b53b0 0x7f49200b7870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f492c0058c0 tx=0x7f492c00a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:55.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.006+0000 7f4933400640 1 -- 172.21.15.90:0/1306208500 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4928009de0 con 0x7f493c1087b0 2024-09-10T22:00:55.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.200+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f493c10f320 con 0x7f493c1087b0 2024-09-10T22:00:55.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.202+0000 7f4933400640 1 -- 172.21.15.90:0/1306208500 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24540 (secure 0 0 0) 0x7f49280ab950 con 0x7f493c1087b0 2024-09-10T22:00:55.204 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:55.204 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:00:47.831865+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:55.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f49200b53b0 msgr2=0x7f49200b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:55.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f49200b53b0 0x7f49200b7870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f492c0058c0 tx=0x7f492c00a250 comp rx=0 tx=0).stop 2024-09-10T22:00:55.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c1087b0 msgr2=0x7f493c19b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:55.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c1087b0 0x7f493c19b890 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f492800d950 tx=0x7f492800de10 comp rx=0 tx=0).stop 2024-09-10T22:00:55.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 shutdown_connections 2024-09-10T22:00:55.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f49200b53b0 0x7f49200b7870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:55.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f493c109130 0x7f493c1a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:55.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f493c1087b0 0x7f493c19b890 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:55.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 --2- 172.21.15.90:0/1306208500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f493c102da0 0x7f493c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:55.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.204+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 >> 172.21.15.90:0/1306208500 conn(0x7f493c0fe810 msgr2=0x7f493c104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:55.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.205+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 shutdown_connections 2024-09-10T22:00:55.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:55.205+0000 7f4941c00640 1 -- 172.21.15.90:0/1306208500 wait complete. 2024-09-10T22:00:55.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:54 smithi171 ceph-mon[34138]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:56.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:55 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1306208500' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:00:56.185 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:00:56.186 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:55 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1306208500' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:00:56.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:55 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1306208500' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:00:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:57.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:56 smithi090 ceph-mon[30350]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:57.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.141+0000 7f8b1e600640 1 -- 172.21.15.90:0/3144184257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 msgr2=0x7f8b1810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.141+0000 7f8b1e600640 1 --2- 172.21.15.90:0/3144184257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 0x7f8b1810f930 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8b10009ca0 tx=0x7f8b1002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.141+0000 7f8b1e600640 1 -- 172.21.15.90:0/3144184257 shutdown_connections 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.141+0000 7f8b1e600640 1 --2- 172.21.15.90:0/3144184257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 0x7f8b1810f930 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.141+0000 7f8b1e600640 1 --2- 172.21.15.90:0/3144184257 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b181087b0 0x7f8b18108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.141+0000 7f8b1e600640 1 --2- 172.21.15.90:0/3144184257 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b18102da0 0x7f8b18103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.141+0000 7f8b1e600640 1 -- 172.21.15.90:0/3144184257 >> 172.21.15.90:0/3144184257 conn(0x7f8b180fe810 msgr2=0x7f8b18100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.141+0000 7f8b1e600640 1 -- 172.21.15.90:0/3144184257 shutdown_connections 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.141+0000 7f8b1e600640 1 -- 172.21.15.90:0/3144184257 wait complete. 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b1e600640 1 Processor -- start 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b1e600640 1 -- start start 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b1e600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b18102da0 0x7f8b181990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b1e600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b181087b0 0x7f8b18199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b1e600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 0x7f8b1819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:57.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b1e600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b18112cf0 con 0x7f8b18109130 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b1e600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8b18112b70 con 0x7f8b18102da0 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b1e600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8b18112e70 con 0x7f8b181087b0 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b17e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 0x7f8b1819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b17e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 0x7f8b1819e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52426/0 (socket says 172.21.15.90:52426) 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.142+0000 7f8b17e00640 1 -- 172.21.15.90:0/465910159 learned_addr learned my addr 172.21.15.90:0/465910159 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b16a00640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b181087b0 0x7f8b18199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b17400640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b18102da0 0x7f8b181990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b17e00640 1 -- 172.21.15.90:0/465910159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b181087b0 msgr2=0x7f8b18199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b17e00640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b181087b0 0x7f8b18199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b17e00640 1 -- 172.21.15.90:0/465910159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b18102da0 msgr2=0x7f8b181990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b17e00640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b18102da0 0x7f8b181990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b17e00640 1 -- 172.21.15.90:0/465910159 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1819f170 con 0x7f8b18109130 2024-09-10T22:00:57.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b17400640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b18102da0 0x7f8b181990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:57.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b17e00640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 0x7f8b1819e9a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8b10005cc0 tx=0x7f8b10032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:57.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b0fe00640 1 -- 172.21.15.90:0/465910159 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b10032e00 con 0x7f8b18109130 2024-09-10T22:00:57.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b0fe00640 1 -- 172.21.15.90:0/465910159 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8b100044d0 con 0x7f8b18109130 2024-09-10T22:00:57.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.143+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1819f400 con 0x7f8b18109130 2024-09-10T22:00:57.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.144+0000 7f8b0fe00640 1 -- 172.21.15.90:0/465910159 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b10030b70 con 0x7f8b18109130 2024-09-10T22:00:57.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.144+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b181a5b20 con 0x7f8b18109130 2024-09-10T22:00:57.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.145+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b18103180 con 0x7f8b18109130 2024-09-10T22:00:57.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.147+0000 7f8b0fe00640 1 -- 172.21.15.90:0/465910159 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f8b10004680 con 0x7f8b18109130 2024-09-10T22:00:57.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.148+0000 7f8b0fe00640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8afc0b53b0 0x7f8afc0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:57.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.149+0000 7f8b0fe00640 1 -- 172.21.15.90:0/465910159 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f8b10107370 con 0x7f8b18109130 2024-09-10T22:00:57.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.149+0000 7f8b17400640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8afc0b53b0 0x7f8afc0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:57.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.149+0000 7f8b17400640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8afc0b53b0 0x7f8afc0b7870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8b080058c0 tx=0x7f8b08005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:57.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.151+0000 7f8b0fe00640 1 -- 172.21.15.90:0/465910159 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8b100cdec0 con 0x7f8b18109130 2024-09-10T22:00:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:56 smithi154 ceph-mon[34495]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:57.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:56 smithi171 ceph-mon[34138]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:57.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.375+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b1819a1f0 con 0x7f8b18109130 2024-09-10T22:00:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.375+0000 7f8b0fe00640 1 -- 172.21.15.90:0/465910159 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11851 (secure 0 0 0) 0x7f8b1819a1f0 con 0x7f8b18109130 2024-09-10T22:00:57.377 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:57.377 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"btime":"2024-09-10T22:00:48:722515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14841,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14853,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24598,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24676,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":34607,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":34640,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":34646,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":34670,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:00:47.832003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34703,"qdb_cluster":[34703]},"id":1}]} 2024-09-10T22:00:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-09-10T22:00:57.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8afc0b53b0 msgr2=0x7f8afc0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8afc0b53b0 0x7f8afc0b7870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8b080058c0 tx=0x7f8b08005850 comp rx=0 tx=0).stop 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 msgr2=0x7f8b1819e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 0x7f8b1819e9a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8b10005cc0 tx=0x7f8b10032a10 comp rx=0 tx=0).stop 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 shutdown_connections 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8afc0b53b0 0x7f8afc0b7870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18109130 0x7f8b1819e9a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8b181087b0 0x7f8b18199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 --2- 172.21.15.90:0/465910159 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8b18102da0 0x7f8b181990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 >> 172.21.15.90:0/465910159 conn(0x7f8b180fe810 msgr2=0x7f8b18101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 shutdown_connections 2024-09-10T22:00:57.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:57.378+0000 7f8b1e600640 1 -- 172.21.15.90:0/465910159 wait complete. 2024-09-10T22:00:58.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:57 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/465910159' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:58.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:00:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:57 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/465910159' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:58.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:57 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/465910159' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:00:58.515 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:00:59.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:58 smithi090 ceph-mon[30350]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:58 smithi154 ceph-mon[34495]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:59.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.126+0000 7f58d5800640 1 -- 172.21.15.90:0/1587239741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d0104df0 msgr2=0x7f58d01051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.126+0000 7f58d5800640 1 --2- 172.21.15.90:0/1587239741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d0104df0 0x7f58d01051d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f58c0009ca0 tx=0x7f58c002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.126+0000 7f58d5800640 1 -- 172.21.15.90:0/1587239741 shutdown_connections 2024-09-10T22:00:59.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.126+0000 7f58d5800640 1 --2- 172.21.15.90:0/1587239741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d010b180 0x7f58d0111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.126+0000 7f58d5800640 1 --2- 172.21.15.90:0/1587239741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58d010a800 0x7f58d010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.126+0000 7f58d5800640 1 --2- 172.21.15.90:0/1587239741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d0104df0 0x7f58d01051d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.126+0000 7f58d5800640 1 -- 172.21.15.90:0/1587239741 >> 172.21.15.90:0/1587239741 conn(0x7f58d0100840 msgr2=0x7f58d0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.126+0000 7f58d5800640 1 -- 172.21.15.90:0/1587239741 shutdown_connections 2024-09-10T22:00:59.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.127+0000 7f58d5800640 1 -- 172.21.15.90:0/1587239741 wait complete. 2024-09-10T22:00:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.127+0000 7f58d5800640 1 Processor -- start 2024-09-10T22:00:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d5800640 1 -- start start 2024-09-10T22:00:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d5800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d0104df0 0x7f58d019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d5800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58d010a800 0x7f58d019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:59.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d5800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d010b180 0x7f58d01a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d5800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f58d0114d80 con 0x7f58d010b180 2024-09-10T22:00:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d5800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58d0114c00 con 0x7f58d0104df0 2024-09-10T22:00:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d5800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f58d0114f00 con 0x7f58d010a800 2024-09-10T22:00:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d4e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d010b180 0x7f58d01a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d4e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d010b180 0x7f58d01a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48998/0 (socket says 172.21.15.90:48998) 2024-09-10T22:00:59.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58d4e00640 1 -- 172.21.15.90:0/4148655861 learned_addr learned my addr 172.21.15.90:0/4148655861 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:00:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.128+0000 7f58cfe00640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d0104df0 0x7f58d019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58d4e00640 1 -- 172.21.15.90:0/4148655861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58d010a800 msgr2=0x7f58d019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58c7400640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58d010a800 0x7f58d019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58d4e00640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58d010a800 0x7f58d019b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58d4e00640 1 -- 172.21.15.90:0/4148655861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d0104df0 msgr2=0x7f58d019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58d4e00640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d0104df0 0x7f58d019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58d4e00640 1 -- 172.21.15.90:0/4148655861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d01a12c0 con 0x7f58d010b180 2024-09-10T22:00:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58cfe00640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d0104df0 0x7f58d019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:00:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58d4e00640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d010b180 0x7f58d01a0af0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009fd0 tx=0x7f58c800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:59.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58cd600640 1 -- 172.21.15.90:0/4148655861 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8011830 con 0x7f58d010b180 2024-09-10T22:00:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58cd600640 1 -- 172.21.15.90:0/4148655861 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58c8016070 con 0x7f58d010b180 2024-09-10T22:00:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d01a15b0 con 0x7f58d010b180 2024-09-10T22:00:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58cd600640 1 -- 172.21.15.90:0/4148655861 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8010d20 con 0x7f58d010b180 2024-09-10T22:00:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.129+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d01a7cf0 con 0x7f58d010b180 2024-09-10T22:00:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.130+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d01051d0 con 0x7f58d010b180 2024-09-10T22:00:59.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.133+0000 7f58cd600640 1 -- 172.21.15.90:0/4148655861 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f58c80119d0 con 0x7f58d010b180 2024-09-10T22:00:59.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.134+0000 7f58cd600640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58b40b53e0 0x7f58b40b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:00:59.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.134+0000 7f58cd600640 1 -- 172.21.15.90:0/4148655861 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f58c80e5c60 con 0x7f58d010b180 2024-09-10T22:00:59.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.134+0000 7f58cfe00640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58b40b53e0 0x7f58b40b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:00:59.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.135+0000 7f58cfe00640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58b40b53e0 0x7f58b40b78a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f58c0031040 tx=0x7f58c0000f40 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:00:59.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.136+0000 7f58cd600640 1 -- 172.21.15.90:0/4148655861 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f58c80ac7b0 con 0x7f58d010b180 2024-09-10T22:00:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:58 smithi171 ceph-mon[34138]: pgmap v326: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:00:59.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.320+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58d019c340 con 0x7f58d010b180 2024-09-10T22:00:59.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.322+0000 7f58cd600640 1 -- 172.21.15.90:0/4148655861 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24540 (secure 0 0 0) 0x7f58c80ac0b0 con 0x7f58d010b180 2024-09-10T22:00:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:00:59.324 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:00:47.831865+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:00:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58b40b53e0 msgr2=0x7f58b40b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58b40b53e0 0x7f58b40b78a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f58c0031040 tx=0x7f58c0000f40 comp rx=0 tx=0).stop 2024-09-10T22:00:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d010b180 msgr2=0x7f58d01a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:00:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d010b180 0x7f58d01a0af0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009fd0 tx=0x7f58c800b8e0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 shutdown_connections 2024-09-10T22:00:59.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58b40b53e0 0x7f58b40b78a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58d010b180 0x7f58d01a0af0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58d010a800 0x7f58d019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 --2- 172.21.15.90:0/4148655861 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58d0104df0 0x7f58d019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:00:59.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 >> 172.21.15.90:0/4148655861 conn(0x7f58d0100840 msgr2=0x7f58d0106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:00:59.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.324+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 shutdown_connections 2024-09-10T22:00:59.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:00:59.325+0000 7f58d5800640 1 -- 172.21.15.90:0/4148655861 wait complete. 2024-09-10T22:01:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:00:59 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4148655861' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:00:59 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4148655861' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:00:59 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4148655861' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:00.274 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T22:00:35.595646+0000', 'modified': '2024-09-10T22:00:47.832003+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 34703}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_34703': {'gid': 34703, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.154:6839/703390433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 703390433}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 703390433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 34703, 'qdb_cluster': [34703]} 2024-09-10T22:01:00.275 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T22:01:00.275 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:01:00.643 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:00 smithi090 ceph-mon[30350]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:00 smithi154 ceph-mon[34495]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:00 smithi171 ceph-mon[34138]: pgmap v327: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:01.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.221+0000 7fb7d1a00640 1 -- 172.21.15.90:0/2480469239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc102dc0 msgr2=0x7fb7cc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.221+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/2480469239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc102dc0 0x7fb7cc1031a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009900 tx=0x7fb7bc02a400 comp rx=0 tx=0).stop 2024-09-10T22:01:01.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.221+0000 7fb7d1a00640 1 -- 172.21.15.90:0/2480469239 shutdown_connections 2024-09-10T22:01:01.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.221+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/2480469239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7cc109150 0x7fb7cc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.221+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/2480469239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc1087d0 0x7fb7cc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.221+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/2480469239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc102dc0 0x7fb7cc1031a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.221+0000 7fb7d1a00640 1 -- 172.21.15.90:0/2480469239 >> 172.21.15.90:0/2480469239 conn(0x7fb7cc0fe830 msgr2=0x7fb7cc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.221+0000 7fb7d1a00640 1 -- 172.21.15.90:0/2480469239 shutdown_connections 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.221+0000 7fb7d1a00640 1 -- 172.21.15.90:0/2480469239 wait complete. 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.222+0000 7fb7d1a00640 1 Processor -- start 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.222+0000 7fb7d1a00640 1 -- start start 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7d1a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102dc0 0x7fb7cc1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7d1a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7cc1087d0 0x7fb7cc199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7d1a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc109150 0x7fb7cc19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7d1a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc112c80 con 0x7fb7cc109150 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7d1a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc112b00 con 0x7fb7cc1087d0 2024-09-10T22:01:01.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7d1a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc112e00 con 0x7fb7cc102dc0 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7cb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc109150 0x7fb7cc19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7caa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102dc0 0x7fb7cc1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7cb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc109150 0x7fb7cc19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49020/0 (socket says 172.21.15.90:49020) 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7cb400640 1 -- 172.21.15.90:0/1088865676 learned_addr learned my addr 172.21.15.90:0/1088865676 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.223+0000 7fb7ca000640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7cc1087d0 0x7fb7cc199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7cb400640 1 -- 172.21.15.90:0/1088865676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102dc0 msgr2=0x7fb7cc1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7cb400640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102dc0 0x7fb7cc1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7cb400640 1 -- 172.21.15.90:0/1088865676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7cc1087d0 msgr2=0x7fb7cc199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7cb400640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7cc1087d0 0x7fb7cc199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7cb400640 1 -- 172.21.15.90:0/1088865676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc19f1a0 con 0x7fb7cc109150 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7cb400640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc109150 0x7fb7cc19e9d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400cce0 tx=0x7fb7c400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7c3400640 1 -- 172.21.15.90:0/1088865676 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c400f450 con 0x7fb7cc109150 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7c3400640 1 -- 172.21.15.90:0/1088865676 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7c400fa80 con 0x7fb7cc109150 2024-09-10T22:01:01.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7c3400640 1 -- 172.21.15.90:0/1088865676 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c40189c0 con 0x7fb7cc109150 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.224+0000 7fb7d1a00640 1 -- 172.21.15.90:0/1088865676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc19f490 con 0x7fb7cc109150 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.225+0000 7fb7d1a00640 1 -- 172.21.15.90:0/1088865676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc1a5b50 con 0x7fb7cc109150 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.228+0000 7fb7d4611740 1 Processor -- start 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.228+0000 7fb7c3400640 1 -- 172.21.15.90:0/1088865676 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fb7c4020030 con 0x7fb7cc109150 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.228+0000 7fb7d4611740 1 -- start start 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.228+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6240 0x55feb17c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.228+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17d69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.228+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17d6f00 0x55feb17d92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.228+0000 7fb7d4611740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55feb173dd50 con 0x55feb17c6b60 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.228+0000 7fb7d4611740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55feb172d180 con 0x55feb17c6240 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.228+0000 7fb7d4611740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55feb1730f90 con 0x55feb17d6f00 2024-09-10T22:01:01.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7caa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17d69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7caa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17d69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49028/0 (socket says 172.21.15.90:49028) 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7caa00640 1 -- 172.21.15.90:0/3411972785 learned_addr learned my addr 172.21.15.90:0/3411972785 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7ca000640 1 --2- 172.21.15.90:0/3411972785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6240 0x55feb17c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7ca000640 1 --2- 172.21.15.90:0/3411972785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17d6f00 0x55feb17d92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7caa00640 1 -- 172.21.15.90:0/3411972785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17d6f00 msgr2=0x55feb17d92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7caa00640 1 --2- 172.21.15.90:0/3411972785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17d6f00 0x55feb17d92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7caa00640 1 -- 172.21.15.90:0/3411972785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6240 msgr2=0x55feb17c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7caa00640 1 --2- 172.21.15.90:0/3411972785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6240 0x55feb17c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.229+0000 7fb7caa00640 1 -- 172.21.15.90:0/3411972785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb16f7a70 con 0x55feb17c6b60 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.230+0000 7fb7c3400640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7b00b53b0 0x7fb7b00b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.230+0000 7fb7caa00640 1 --2- 172.21.15.90:0/3411972785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17d69c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc000c00 tx=0x7fb7bc031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4e7a17cd2c6042 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.230+0000 7fb7c3400640 1 -- 172.21.15.90:0/1088865676 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7fb7c40e5fd0 con 0x7fb7cc109150 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.230+0000 7fb7ca000640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7b00b53b0 0x7fb7b00b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.230+0000 7fb7bbe00640 1 -- 172.21.15.90:0/3411972785 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc035030 con 0x55feb17c6b60 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.230+0000 7fb7bbe00640 1 -- 172.21.15.90:0/3411972785 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7bc040070 con 0x55feb17c6b60 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.230+0000 7fb7bbe00640 1 -- 172.21.15.90:0/3411972785 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc003a50 con 0x55feb17c6b60 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 -- 172.21.15.90:0/3411972785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 msgr2=0x55feb17d69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7ca000640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7b00b53b0 0x7fb7b00b7870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc102150 tx=0x7fb7b400a770 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3411972785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17d69c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc000c00 tx=0x7fb7bc031dc0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 -- 172.21.15.90:0/3411972785 shutdown_connections 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3411972785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17d6f00 0x55feb17d92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3411972785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17d69c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3411972785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6240 0x55feb17c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 -- 172.21.15.90:0/3411972785 >> 172.21.15.90:0/3411972785 conn(0x55feb17c4020 msgr2=0x55feb17c5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 -- 172.21.15.90:0/3411972785 shutdown_connections 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 -- 172.21.15.90:0/3411972785 wait complete. 2024-09-10T22:01:01.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.231+0000 7fb7d4611740 1 -- start start 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.232+0000 7fb7d4611740 1 Processor -- start 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.232+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6240 0x55feb17d6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.232+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6b60 0x55feb17d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.232+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 0x55feb17d4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.232+0000 7fb7d4611740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55feb158e5f0 con 0x55feb17c6240 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.232+0000 7fb7d4611740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55feb16c8460 con 0x55feb17d6f00 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.232+0000 7fb7d4611740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55feb17d9bc0 con 0x55feb17c6b60 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.232+0000 7fb7d4611740 10 client.0 _ll_register_callbacks cb 0x55feb16d8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7caa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6240 0x55feb17d6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7caa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6240 0x55feb17d6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49030/0 (socket says 172.21.15.90:49030) 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7caa00640 1 -- 172.21.15.90:0/41659447 learned_addr learned my addr 172.21.15.90:0/41659447 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7ca000640 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6b60 0x55feb17d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7cb400640 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 0x55feb17d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7caa00640 1 -- 172.21.15.90:0/41659447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6b60 msgr2=0x55feb17d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:01.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7caa00640 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6b60 0x55feb17d3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7caa00640 1 -- 172.21.15.90:0/41659447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 msgr2=0x55feb17d4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7caa00640 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 0x55feb17d4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7caa00640 1 -- 172.21.15.90:0/41659447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb148b540 con 0x55feb17c6240 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7caa00640 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6240 0x55feb17d6980 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc006010 tx=0x7fb7bc031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef67062cec2fd48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.233+0000 7fb7b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.234+0000 7fb7b8c00640 1 -- 172.21.15.90:0/41659447 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc035040 con 0x55feb17c6240 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.234+0000 7fb7d4611740 1 -- 172.21.15.90:0/41659447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55feb16caa70 con 0x55feb17c6240 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.234+0000 7fb7b8c00640 1 -- 172.21.15.90:0/41659447 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7fb7bc03c850 con 0x55feb17c6240 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.234+0000 7fb7b8c00640 1 -- 172.21.15.90:0/41659447 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb7bc058050 con 0x55feb17c6240 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.234+0000 7fb7d4611740 10 client.14955 fetch_fsmap learned FSMap version 17 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.234+0000 7fb7d4611740 1 -- 172.21.15.90:0/41659447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55feb16e56f0 con 0x55feb17c6240 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.234+0000 7fb7b8c00640 1 -- 172.21.15.90:0/41659447 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55feb16caa70 con 0x55feb17c6240 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.234+0000 7fb7b8c00640 1 -- 172.21.15.90:0/41659447 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fb7bc04d8d0 con 0x55feb17c6240 2024-09-10T22:01:01.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.235+0000 7fb7d4611740 10 client.14955 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.235+0000 7fb7d4611740 10 client.14955 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.235+0000 7fb7d4611740 20 client.14955 populate_metadata read hostname 'smithi090' 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.235+0000 7fb7d4611740 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55feb17e7690 0x55feb17e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.235+0000 7fb7d4611740 4 client.14955 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.235+0000 7fb7d4611740 1 -- 172.21.15.90:0/41659447 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feb173fca0 con 0x55feb17e7690 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.235+0000 7fb7ca000640 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55feb17e7690 0x55feb17e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.236+0000 7fb7ca000640 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55feb17e7690 0x55feb17e8b00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c84a165dcee9adc server_cookie=80b811c966514ead in_seq=0 out_seq=0 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.236+0000 7fb7b8c00640 10 client.14955 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7b8c00640 1 -- 172.21.15.90:0/41659447 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55feb173fca0 con 0x55feb17e7690 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7b8c00640 10 client.14955 handle_command_reply: tid=0 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 client.14955 shutdown 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 -- 172.21.15.90:0/41659447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6240 msgr2=0x55feb17d6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6240 0x55feb17d6980 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc006010 tx=0x7fb7bc031630 comp rx=0 tx=0).stop 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 -- 172.21.15.90:0/41659447 shutdown_connections 2024-09-10T22:01:01.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55feb17e7690 0x55feb17e8b00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 0x55feb17d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6b60 0x55feb17d3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7b8c00640 0 client.14955 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:01:01.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 --2- 172.21.15.90:0/41659447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6240 0x55feb17d6980 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 -- 172.21.15.90:0/41659447 >> 172.21.15.90:0/41659447 conn(0x55feb17c4020 msgr2=0x55feb17c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 -- 172.21.15.90:0/41659447 shutdown_connections 2024-09-10T22:01:01.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.237+0000 7fb7d4611740 1 -- 172.21.15.90:0/41659447 wait complete. 2024-09-10T22:01:01.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.238+0000 7fb7d4611740 20 client.14955 trim_cache size 0 max 16384 2024-09-10T22:01:01.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7d4611740 1 Processor -- start 2024-09-10T22:01:01.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7d4611740 1 -- start start 2024-09-10T22:01:01.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17e6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6b60 0x55feb17beea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17d6f00 0x55feb17d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7d4611740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55feb17e6e40 con 0x55feb17d6f00 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7d4611740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55feb17e7510 con 0x55feb17c6b60 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7d4611740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55feb17e7be0 con 0x55feb17c6240 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7cb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17d6f00 0x55feb17d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7cb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17d6f00 0x55feb17d7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49034/0 (socket says 172.21.15.90:49034) 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.257+0000 7fb7cb400640 1 -- 172.21.15.90:0/728576738 learned_addr learned my addr 172.21.15.90:0/728576738 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7ca000640 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6b60 0x55feb17beea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7caa00640 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17e6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7cb400640 1 -- 172.21.15.90:0/728576738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 msgr2=0x55feb17e6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7cb400640 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17e6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7cb400640 1 -- 172.21.15.90:0/728576738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6b60 msgr2=0x55feb17beea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7cb400640 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6b60 0x55feb17beea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7cb400640 1 -- 172.21.15.90:0/728576738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feb17d22d0 con 0x55feb17d6f00 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7caa00640 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17e6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7cb400640 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17d6f00 0x55feb17d7f50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40e3760 tx=0x7fb7c400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617b66c0d341e161 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7bb400640 1 -- 172.21.15.90:0/728576738 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c40b5300 con 0x55feb17d6f00 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7bb400640 1 -- 172.21.15.90:0/728576738 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7c40ac420 con 0x55feb17d6f00 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.258+0000 7fb7bb400640 1 -- 172.21.15.90:0/728576738 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4093b60 con 0x55feb17d6f00 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.259+0000 7fb7d4611740 1 -- 172.21.15.90:0/728576738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17d6f00 msgr2=0x55feb17d7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.259+0000 7fb7d4611740 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17d6f00 0x55feb17d7f50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40e3760 tx=0x7fb7c400cf10 comp rx=0 tx=0).stop 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.259+0000 7fb7d4611740 1 -- 172.21.15.90:0/728576738 shutdown_connections 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.259+0000 7fb7d4611740 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17d6f00 0x55feb17d7f50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.259+0000 7fb7d4611740 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17c6b60 0x55feb17beea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.259+0000 7fb7d4611740 1 --2- 172.21.15.90:0/728576738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17e6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.259+0000 7fb7d4611740 1 -- 172.21.15.90:0/728576738 >> 172.21.15.90:0/728576738 conn(0x55feb17c4020 msgr2=0x55feb17d8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.259+0000 7fb7d4611740 1 -- 172.21.15.90:0/728576738 shutdown_connections 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.259+0000 7fb7d4611740 1 -- 172.21.15.90:0/728576738 wait complete. 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7d4611740 1 -- start start 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7d4611740 1 Processor -- start 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17f1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7d4611740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 0x55feb17ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7d4611740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55feb17e8980 con 0x55feb17c6b60 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7d4611740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55feb17e82b0 con 0x55feb17d6f00 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7d4611740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55feb17e9050 con 0x55feb17c6240 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7d4611740 10 client.0 _ll_register_callbacks cb 0x7fb7bc0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.260+0000 7fb7ca000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7ca000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17f16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49050/0 (socket says 172.21.15.90:49050) 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7ca000640 1 -- 172.21.15.90:0/3477543628 learned_addr learned my addr 172.21.15.90:0/3477543628 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7caa00640 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7ca000640 1 -- 172.21.15.90:0/3477543628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 msgr2=0x55feb17f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7ca000640 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7cb400640 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 0x55feb17ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7ca000640 1 -- 172.21.15.90:0/3477543628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 msgr2=0x55feb17ebe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7ca000640 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 0x55feb17ebe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7ca000640 1 -- 172.21.15.90:0/3477543628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feb17ec680 con 0x55feb17c6b60 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7caa00640 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7cb400640 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 0x55feb17ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7ca000640 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17f16b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4010800 tx=0x7fb7b400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68393d85eb090de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.261+0000 7fb7b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.262+0000 7fb7b8c00640 1 -- 172.21.15.90:0/3477543628 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4010e50 con 0x55feb17c6b60 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.262+0000 7fb7b8c00640 1 -- 172.21.15.90:0/3477543628 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7fb7b402ca50 con 0x55feb17c6b60 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.262+0000 7fb7d4611740 1 -- 172.21.15.90:0/3477543628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c40ac420 con 0x55feb17c6b60 2024-09-10T22:01:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.262+0000 7fb7b8c00640 1 -- 172.21.15.90:0/3477543628 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb7b402cc50 con 0x55feb17c6b60 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.262+0000 7fb7d4611740 10 client.14961 fetch_fsmap learned FSMap version 17 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.262+0000 7fb7d4611740 1 -- 172.21.15.90:0/3477543628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55feb1854c80 con 0x55feb17c6b60 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.262+0000 7fb7b8c00640 1 -- 172.21.15.90:0/3477543628 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb7c40ac420 con 0x55feb17c6b60 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.262+0000 7fb7b8c00640 1 -- 172.21.15.90:0/3477543628 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fb7b4004aa0 con 0x55feb17c6b60 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.262+0000 7fb7d4611740 10 client.14961 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.263+0000 7fb7d4611740 10 client.14961 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.263+0000 7fb7d4611740 20 client.14961 populate_metadata read hostname 'smithi090' 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.263+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55feb18598d0 0x55feb18585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.263+0000 7fb7d4611740 4 client.14961 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.263+0000 7fb7d4611740 1 -- 172.21.15.90:0/3477543628 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "status"}) -- 0x55feb173b750 con 0x55feb18598d0 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.263+0000 7fb7caa00640 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55feb18598d0 0x55feb18585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7caa00640 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55feb18598d0 0x55feb18585e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4351ee96e2c746a7 server_cookie=926ebea8ff95f999 in_seq=0 out_seq=0 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7b8c00640 10 client.14961 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:01:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7b8c00640 1 -- 172.21.15.90:0/3477543628 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55feb173b750 con 0x55feb18598d0 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7b8c00640 10 client.14961 handle_command_reply: tid=0 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 client.14961 shutdown 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 -- 172.21.15.90:0/3477543628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 msgr2=0x55feb17f16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17f16b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4010800 tx=0x7fb7b400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 -- 172.21.15.90:0/3477543628 shutdown_connections 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55feb18598d0 0x55feb18585e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55feb17d6f00 0x55feb17ebe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55feb17c6b60 0x55feb17f16b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 --2- 172.21.15.90:0/3477543628 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55feb17c6240 0x55feb17f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 -- 172.21.15.90:0/3477543628 >> 172.21.15.90:0/3477543628 conn(0x55feb17c4020 msgr2=0x55feb17beea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7b8c00640 0 client.14961 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:01:01.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 -- 172.21.15.90:0/3477543628 shutdown_connections 2024-09-10T22:01:01.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 1 -- 172.21.15.90:0/3477543628 wait complete. 2024-09-10T22:01:01.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.264+0000 7fb7d4611740 20 client.14961 trim_cache size 0 max 16384 2024-09-10T22:01:01.277 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:01:01.277 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:01:01.277 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-09-10T22:01:01.277 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34703, 2024-09-10T22:01:01.277 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:01:01.277 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:01:01.277 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:01:01.278 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 24.770099839, 2024-09-10T22:01:01.278 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 17, 2024-09-10T22:01:01.278 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 86, 2024-09-10T22:01:01.278 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T22:01:01.278 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 37.781935128000001 2024-09-10T22:01:01.278 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 -- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7b00b53b0 msgr2=0x7fb7b00b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7b00b53b0 0x7fb7b00b7870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc102150 tx=0x7fb7b400a770 comp rx=0 tx=0).stop 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 -- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc109150 msgr2=0x7fb7cc19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc109150 0x7fb7cc19e9d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb7c400cce0 tx=0x7fb7c400a620 comp rx=0 tx=0).stop 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 -- 172.21.15.90:0/1088865676 shutdown_connections 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7b00b53b0 0x7fb7b00b7870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc109150 0x7fb7cc19e9d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb7cc1087d0 0x7fb7cc199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 --2- 172.21.15.90:0/1088865676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb7cc102dc0 0x7fb7cc1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 -- 172.21.15.90:0/1088865676 >> 172.21.15.90:0/1088865676 conn(0x7fb7cc0fe830 msgr2=0x7fb7cc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 -- 172.21.15.90:0/1088865676 shutdown_connections 2024-09-10T22:01:01.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:01.265+0000 7fb7d1a00640 1 -- 172.21.15.90:0/1088865676 wait complete. 2024-09-10T22:01:02.127 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:01:02.127 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:01:02.127 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T22:01:02.497 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:02.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:02 smithi090 ceph-mon[30350]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:03.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.060+0000 7f920b200640 1 -- 172.21.15.90:0/2591291146 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c0776d0 msgr2=0x7f920c077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.060+0000 7f920b200640 1 --2- 172.21.15.90:0/2591291146 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c0776d0 0x7f920c077ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009900 tx=0x7f91fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.060+0000 7f920b200640 1 -- 172.21.15.90:0/2591291146 shutdown_connections 2024-09-10T22:01:03.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.060+0000 7f920b200640 1 --2- 172.21.15.90:0/2591291146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f920c1137e0 0x7f920c115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.060+0000 7f920b200640 1 --2- 172.21.15.90:0/2591291146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c077ff0 0x7f920c1132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.060+0000 7f920b200640 1 --2- 172.21.15.90:0/2591291146 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c0776d0 0x7f920c077ab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.060+0000 7f920b200640 1 -- 172.21.15.90:0/2591291146 >> 172.21.15.90:0/2591291146 conn(0x7f920c100840 msgr2=0x7f920c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:03.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.060+0000 7f920b200640 1 -- 172.21.15.90:0/2591291146 shutdown_connections 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.060+0000 7f920b200640 1 -- 172.21.15.90:0/2591291146 wait complete. 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.061+0000 7f920b200640 1 Processor -- start 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.061+0000 7f920b200640 1 -- start start 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c0776d0 0x7f920c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f920c077ff0 0x7f920c19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c1137e0 0x7f920c1a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f920c119140 con 0x7f920c1137e0 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f920c118fc0 con 0x7f920c077ff0 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f920c1192c0 con 0x7f920c0776d0 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c1137e0 0x7f920c1a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c1137e0 0x7f920c1a4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49072/0 (socket says 172.21.15.90:49072) 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920a800640 1 -- 172.21.15.90:0/16430235 learned_addr learned my addr 172.21.15.90:0/16430235 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:03.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f9209400640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f920c077ff0 0x7f920c19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f9209e00640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c0776d0 0x7f920c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920a800640 1 -- 172.21.15.90:0/16430235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c0776d0 msgr2=0x7f920c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920a800640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c0776d0 0x7f920c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920a800640 1 -- 172.21.15.90:0/16430235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f920c077ff0 msgr2=0x7f920c19fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920a800640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f920c077ff0 0x7f920c19fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.062+0000 7f920a800640 1 -- 172.21.15.90:0/16430235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c1a55d0 con 0x7f920c1137e0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.063+0000 7f920a800640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c1137e0 0x7f920c1a4e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f920400ccb0 tx=0x7f920400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.063+0000 7f9202a00640 1 -- 172.21.15.90:0/16430235 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920400f450 con 0x7f920c1137e0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.063+0000 7f9202a00640 1 -- 172.21.15.90:0/16430235 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f920400fa80 con 0x7f920c1137e0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.063+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c1a58c0 con 0x7f920c1137e0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.063+0000 7f9202a00640 1 -- 172.21.15.90:0/16430235 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9204018950 con 0x7f920c1137e0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.063+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c1ac000 con 0x7f920c1137e0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.064+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920c077ab0 con 0x7f920c1137e0 2024-09-10T22:01:03.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.066+0000 7f9202a00640 1 -- 172.21.15.90:0/16430235 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f9204020030 con 0x7f920c1137e0 2024-09-10T22:01:03.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.068+0000 7f9202a00640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f91f00b5460 0x7f91f00b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:03.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.068+0000 7f9202a00640 1 -- 172.21.15.90:0/16430235 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f92040e5ed0 con 0x7f920c1137e0 2024-09-10T22:01:03.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.068+0000 7f9209e00640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f91f00b5460 0x7f91f00b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:03.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.069+0000 7f9209e00640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f91f00b5460 0x7f91f00b7920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f91fc006010 tx=0x7f91fc005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:03.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.072+0000 7f9202a00640 1 -- 172.21.15.90:0/16430235 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9204014030 con 0x7f920c1137e0 2024-09-10T22:01:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:02 smithi154 ceph-mon[34495]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:02 smithi171 ceph-mon[34138]: pgmap v328: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:03.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.294+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f920c1ac360 con 0x7f920c1137e0 2024-09-10T22:01:03.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.294+0000 7f9202a00640 1 -- 172.21.15.90:0/16430235 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f92040aca20 con 0x7f920c1137e0 2024-09-10T22:01:03.296 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T22:01:03.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.296+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f91f00b5460 msgr2=0x7f91f00b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.296+0000 7f920b200640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f91f00b5460 0x7f91f00b7920 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f91fc006010 tx=0x7f91fc005e80 comp rx=0 tx=0).stop 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.296+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c1137e0 msgr2=0x7f920c1a4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.296+0000 7f920b200640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c1137e0 0x7f920c1a4e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f920400ccb0 tx=0x7f920400a620 comp rx=0 tx=0).stop 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.297+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 shutdown_connections 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.297+0000 7f920b200640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f91f00b5460 0x7f91f00b7920 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.297+0000 7f920b200640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f920c1137e0 0x7f920c1a4e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.297+0000 7f920b200640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f920c077ff0 0x7f920c19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.297+0000 7f920b200640 1 --2- 172.21.15.90:0/16430235 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f920c0776d0 0x7f920c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.297+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 >> 172.21.15.90:0/16430235 conn(0x7f920c100840 msgr2=0x7f920c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.297+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 shutdown_connections 2024-09-10T22:01:03.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:03.297+0000 7f920b200640 1 -- 172.21.15.90:0/16430235 wait complete. 2024-09-10T22:01:04.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:03 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/16430235' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:01:04.149 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:03 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/16430235' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:01:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:03 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/16430235' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:01:04.195 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T22:01:04.195 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:01:04.195 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-09-10T22:01:04.195 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T22:01:04.195 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:01:04.195 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T22:01:04.196 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:01:04.196 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:01:04.196 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T22:01:04.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:04.196 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-09-10T22:01:04.232 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:01:04.232 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:01:04.232 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:01:04.232 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:04.232 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-09-10T22:01:04.232 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42045sec preferred_lft 42045sec 2024-09-10T22:01:04.233 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link noprefixroute 2024-09-10T22:01:04.234 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:01:04.234 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:01:04.234 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:01:04.234 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:01:04.234 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:01:04.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-09-10T22:01:04.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-09-10T22:01:04.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:01:04.235 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:01:04.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:01:04.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:01:04.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:04.670 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:01:04.728 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-09-10T22:01:04.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:04.731 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-09-10T22:01:04.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:04 smithi090 ceph-mon[30350]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:04.861 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-09-10T22:01:04.862 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:01:04.862 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:01:04.862 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:01:04.862 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:01:04.863 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:01:04.863 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:01:04.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:01:04.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:01:04.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:01:04.863 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:01:04.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:01:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:04 smithi154 ceph-mon[34495]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:04 smithi171 ceph-mon[34138]: pgmap v329: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:05.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:01:05.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:05.525 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-09-10T22:01:05.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:05.561 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-09-10T22:01:05.636 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:01:05.637 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:01:05.637 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:01:05.637 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T22:01:05.637 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:01:05.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:01:05.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:01:05.818 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T22:01:05.818 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:01:05.818 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:01:05.818 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T22:01:05.818 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:01:05.818 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T22:01:05.818 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T22:01:05.818 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T22:01:05.818 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:01:05.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:01:06.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:05 smithi090 ceph-mon[30350]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:06.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:01:06.058 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:01:06.058 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:01:06.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-09-10T22:01:06.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:01:06.059 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:01:06.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:01:06.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:05 smithi154 ceph-mon[34495]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:06.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:01:06.226 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T22:01:06.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:01:06.227 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:01:06.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:05 smithi171 ceph-mon[34138]: pgmap v330: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:06.262 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T22:01:06.262 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:01:06.262 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:01:06.337 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-09-10T22:01:06.434 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:01:06.470 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-09-10T22:01:06.470 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-09-10T22:01:06.471 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/aa67a9b51a02440af118fb4a34c332faaad40a1704ef6bee09f5ec373d88fc36/merged 2024-09-10T22:01:06.472 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d5d9b09148e2eea2bbe0e8ce312021e74fc0f1e8e09a4437adfa136749d3246b/merged 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/a2dc6ee73be1567fd8f29eb80e26f1703773796ce8455114e9ea424c72750d7c/merged 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f9c986101c7a2f5bb7fa1e8bd02849530b16017961224132afbbb799d1392c45/merged 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e37fdfa87f4eac35931f91db6e1b46b42cc7a219cdc3b5db5ebc31f16ee9c77b/merged 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c00ae7ca7df917931de3ddaff2d63373ed5b918da32f164d120bb626ebb226b5/merged 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/bca883ef8c983cdd19c8073e9384d35dfcb291061c8b1a1238032a74f87ec46f/merged 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1facfaa64ed970af42ecf309afb8987b16b91f2d32389ffe22e1c3cb3b1014b6/merged 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/ef00c446100b225a6c68486a124b6d6f154b98394c2c94e127800d7ca868be21/merged 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/937017d01ece5d353e6824b353f0a967ab18de84c0941cf309598207c4f60be5/merged 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:01:06.473 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:01:06.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:06.474 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-09-10T22:01:06.551 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:01:06.551 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T22:01:06.595 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-09-10T22:01:06.653 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:01:06.690 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-09-10T22:01:06.690 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-09-10T22:01:06.690 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-09-10T22:01:06.691 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/aa67a9b51a02440af118fb4a34c332faaad40a1704ef6bee09f5ec373d88fc36/merged 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d5d9b09148e2eea2bbe0e8ce312021e74fc0f1e8e09a4437adfa136749d3246b/merged 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-09-10T22:01:06.692 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/a2dc6ee73be1567fd8f29eb80e26f1703773796ce8455114e9ea424c72750d7c/merged 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f9c986101c7a2f5bb7fa1e8bd02849530b16017961224132afbbb799d1392c45/merged 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e37fdfa87f4eac35931f91db6e1b46b42cc7a219cdc3b5db5ebc31f16ee9c77b/merged 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c00ae7ca7df917931de3ddaff2d63373ed5b918da32f164d120bb626ebb226b5/merged 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/bca883ef8c983cdd19c8073e9384d35dfcb291061c8b1a1238032a74f87ec46f/merged 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1facfaa64ed970af42ecf309afb8987b16b91f2d32389ffe22e1c3cb3b1014b6/merged 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/ef00c446100b225a6c68486a124b6d6f154b98394c2c94e127800d7ca868be21/merged 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/937017d01ece5d353e6824b353f0a967ab18de84c0941cf309598207c4f60be5/merged 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:01:06.693 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:01:06.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:06.694 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-09-10T22:01:06.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:2024-09-10T22:01:06.774+0000 7f6809f88480 -1 init, newargv = 0x5583a645af90 newargc=15 2024-09-10T22:01:06.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[78999]: starting ceph client 2024-09-10T22:01:06.786 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[78999]: starting fuse 2024-09-10T22:01:07.873 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-09-10T22:01:07.937 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:01:07.975 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-09-10T22:01:07.975 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:01:07.976 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/aa67a9b51a02440af118fb4a34c332faaad40a1704ef6bee09f5ec373d88fc36/merged 2024-09-10T22:01:07.977 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d5d9b09148e2eea2bbe0e8ce312021e74fc0f1e8e09a4437adfa136749d3246b/merged 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/a2dc6ee73be1567fd8f29eb80e26f1703773796ce8455114e9ea424c72750d7c/merged 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f9c986101c7a2f5bb7fa1e8bd02849530b16017961224132afbbb799d1392c45/merged 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e37fdfa87f4eac35931f91db6e1b46b42cc7a219cdc3b5db5ebc31f16ee9c77b/merged 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c00ae7ca7df917931de3ddaff2d63373ed5b918da32f164d120bb626ebb226b5/merged 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/bca883ef8c983cdd19c8073e9384d35dfcb291061c8b1a1238032a74f87ec46f/merged 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1facfaa64ed970af42ecf309afb8987b16b91f2d32389ffe22e1c3cb3b1014b6/merged 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/ef00c446100b225a6c68486a124b6d6f154b98394c2c94e127800d7ca868be21/merged 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/937017d01ece5d353e6824b353f0a967ab18de84c0941cf309598207c4f60be5/merged 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:01:07.978 INFO:teuthology.orchestra.run.smithi090.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-10T22:01:07.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:07.979 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-09-10T22:01:08.056 INFO:teuthology.orchestra.run.smithi090.stdout:96 2024-09-10T22:01:08.056 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [96] 2024-09-10T22:01:08.056 DEBUG:teuthology.orchestra.run.smithi090:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:01:08.056 DEBUG:teuthology.orchestra.run.smithi090:> import glob 2024-09-10T22:01:08.056 DEBUG:teuthology.orchestra.run.smithi090:> import re 2024-09-10T22:01:08.056 DEBUG:teuthology.orchestra.run.smithi090:> import os 2024-09-10T22:01:08.056 DEBUG:teuthology.orchestra.run.smithi090:> import subprocess 2024-09-10T22:01:08.056 DEBUG:teuthology.orchestra.run.smithi090:> 2024-09-10T22:01:08.056 DEBUG:teuthology.orchestra.run.smithi090:> def _find_admin_socket(client_name): 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> files = glob.glob(asok_path) 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> # Given a non-glob path, it better be there 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> if "*" not in asok_path: 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> assert(len(files) == 1) 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> return files[0] 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> for f in files: 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> contents = proc_f.read() 2024-09-10T22:01:08.057 DEBUG:teuthology.orchestra.run.smithi090:> if mountpoint in contents: 2024-09-10T22:01:08.058 DEBUG:teuthology.orchestra.run.smithi090:> return f 2024-09-10T22:01:08.058 DEBUG:teuthology.orchestra.run.smithi090:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:01:08.058 DEBUG:teuthology.orchestra.run.smithi090:> 2024-09-10T22:01:08.058 DEBUG:teuthology.orchestra.run.smithi090:> print(_find_admin_socket("client.admin")) 2024-09-10T22:01:08.058 DEBUG:teuthology.orchestra.run.smithi090:> ' 2024-09-10T22:01:08.217 INFO:teuthology.orchestra.run.smithi090.stdout:/var/run/ceph/ceph-client.admin.78999.asok 2024-09-10T22:01:08.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:01:08.231 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78999.asok 2024-09-10T22:01:08.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:08.232 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78999.asok status 2024-09-10T22:01:08.414 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:01:08.414 INFO:teuthology.orchestra.run.smithi090.stdout: "metadata": { 2024-09-10T22:01:08.414 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:01:08.414 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:01:08.414 INFO:teuthology.orchestra.run.smithi090.stdout: "entity_id": "admin", 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": "smithi090", 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "pid": "78999", 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "root": "/" 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_count": 0, 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_pinned_count": 0, 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 14973, 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "inst": { 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": { 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "client", 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "num": 14973 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:01:08.415 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 583001567 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "192.168.0.1:0", 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 583001567 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "inst_str": "client.14973 192.168.0.1:0/583001567", 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "addr_str": "192.168.0.1:0/583001567", 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "inode_count": 1, 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_epoch": 17, 2024-09-10T22:01:08.416 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch": 86, 2024-09-10T22:01:08.417 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:01:08.417 INFO:teuthology.orchestra.run.smithi090.stdout: "blocklisted": false, 2024-09-10T22:01:08.417 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs" 2024-09-10T22:01:08.417 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:01:08.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:08.427 DEBUG:teuthology.orchestra.run.smithi090:> stat --file-system '--printf=%T 2024-09-10T22:01:08.427 DEBUG:teuthology.orchestra.run.smithi090:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:01:08.463 INFO:teuthology.orchestra.run.smithi090.stdout:fuseblk 2024-09-10T22:01:08.463 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:01:08.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:08.463 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:01:08.561 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T22:01:08.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T22:01:08.672 INFO:teuthology.orchestra.run.smithi090.stdout:# file: . 2024-09-10T22:01:08.672 INFO:teuthology.orchestra.run.smithi090.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T22:01:08.672 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:01:08.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:01:08.681 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T22:01:08.681 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:08.719 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T22:01:08.807 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:08 smithi090 ceph-mon[30350]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:08.921 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:08 smithi154 ceph-mon[34495]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:08.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:08 smithi171 ceph-mon[34138]: pgmap v331: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:09.128 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:09.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.832+0000 7f69ca600640 1 -- 172.21.15.90:0/2608445566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c410f3e0 msgr2=0x7f69c41117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:09.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.832+0000 7f69ca600640 1 --2- 172.21.15.90:0/2608445566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c410f3e0 0x7f69c41117d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009ca0 tx=0x7f69bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.833+0000 7f69ca600640 1 -- 172.21.15.90:0/2608445566 shutdown_connections 2024-09-10T22:01:09.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.833+0000 7f69ca600640 1 --2- 172.21.15.90:0/2608445566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c410f3e0 0x7f69c41117d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.833+0000 7f69ca600640 1 --2- 172.21.15.90:0/2608445566 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c4075f40 0x7f69c410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.833+0000 7f69ca600640 1 --2- 172.21.15.90:0/2608445566 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69c4075540 0x7f69c4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.833+0000 7f69ca600640 1 -- 172.21.15.90:0/2608445566 >> 172.21.15.90:0/2608445566 conn(0x7f69c40fe830 msgr2=0x7f69c4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.833+0000 7f69ca600640 1 -- 172.21.15.90:0/2608445566 shutdown_connections 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.833+0000 7f69ca600640 1 -- 172.21.15.90:0/2608445566 wait complete. 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.833+0000 7f69ca600640 1 Processor -- start 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69ca600640 1 -- start start 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69ca600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69c4075540 0x7f69c419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69ca600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c4075f40 0x7f69c419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69ca600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c410f3e0 0x7f69c41a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69ca600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4114ee0 con 0x7f69c4075f40 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69ca600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4114d60 con 0x7f69c410f3e0 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69ca600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f69c4115060 con 0x7f69c4075540 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69c2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c4075f40 0x7f69c419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69c2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c4075f40 0x7f69c419b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41418/0 (socket says 172.21.15.90:41418) 2024-09-10T22:01:09.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69c2a00640 1 -- 172.21.15.90:0/1700815374 learned_addr learned my addr 172.21.15.90:0/1700815374 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69c3400640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69c4075540 0x7f69c419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.834+0000 7f69c3e00640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c410f3e0 0x7f69c41a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69c2a00640 1 -- 172.21.15.90:0/1700815374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69c4075540 msgr2=0x7f69c419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69c2a00640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69c4075540 0x7f69c419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69c2a00640 1 -- 172.21.15.90:0/1700815374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c410f3e0 msgr2=0x7f69c41a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69c2a00640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c410f3e0 0x7f69c41a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69c2a00640 1 -- 172.21.15.90:0/1700815374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c41a14a0 con 0x7f69c4075f40 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69c2a00640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c4075f40 0x7f69c419b930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f69b000d950 tx=0x7f69b000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69bbe00640 1 -- 172.21.15.90:0/1700815374 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0017030 con 0x7f69c4075f40 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69bbe00640 1 -- 172.21.15.90:0/1700815374 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f69b0004020 con 0x7f69c4075f40 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69bbe00640 1 -- 172.21.15.90:0/1700815374 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b00054e0 con 0x7f69c4075f40 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.835+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c41a1790 con 0x7f69c4075f40 2024-09-10T22:01:09.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.836+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c41a7e50 con 0x7f69c4075f40 2024-09-10T22:01:09.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.837+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c4075920 con 0x7f69c4075f40 2024-09-10T22:01:09.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.840+0000 7f69bbe00640 1 -- 172.21.15.90:0/1700815374 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f69b0009a30 con 0x7f69c4075f40 2024-09-10T22:01:09.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.842+0000 7f69bbe00640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69a80b53b0 0x7f69a80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:09.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.842+0000 7f69bbe00640 1 -- 172.21.15.90:0/1700815374 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f69b00e5ee0 con 0x7f69c4075f40 2024-09-10T22:01:09.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.842+0000 7f69c3400640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69a80b53b0 0x7f69a80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:09.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.843+0000 7f69c3400640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69a80b53b0 0x7f69a80b7870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009990 tx=0x7f69b4009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:09.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:09.843+0000 7f69bbe00640 1 -- 172.21.15.90:0/1700815374 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f69b00acab0 con 0x7f69c4075f40 2024-09-10T22:01:10.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.037+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69c419c520 con 0x7f69c4075f40 2024-09-10T22:01:10.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.038+0000 7f69bbe00640 1 -- 172.21.15.90:0/1700815374 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v86) ==== 81+0+886 (secure 0 0 0) 0x7f69c419c520 con 0x7f69c4075f40 2024-09-10T22:01:10.040 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2204308252 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:10.040 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1572343768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:10.040 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1613780567 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:10.040 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1385951279 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:10.040 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/323032163 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3233554768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1613780567 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3858975894 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2255342982 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3643795744 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2773946294 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2694078227 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/323032163 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3233554768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3503416190 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:10.041 INFO:teuthology.orchestra.run.smithi090.stderr:listed 15 entries 2024-09-10T22:01:10.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69a80b53b0 msgr2=0x7f69a80b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:10.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69a80b53b0 0x7f69a80b7870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009990 tx=0x7f69b4009110 comp rx=0 tx=0).stop 2024-09-10T22:01:10.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c4075f40 msgr2=0x7f69c419b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:10.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c4075f40 0x7f69c419b930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f69b000d950 tx=0x7f69b000dd50 comp rx=0 tx=0).stop 2024-09-10T22:01:10.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 shutdown_connections 2024-09-10T22:01:10.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f69a80b53b0 0x7f69a80b7870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f69c410f3e0 0x7f69c41a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69c4075f40 0x7f69c419b930 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 --2- 172.21.15.90:0/1700815374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f69c4075540 0x7f69c419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:10.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 >> 172.21.15.90:0/1700815374 conn(0x7f69c40fe830 msgr2=0x7f69c40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:10.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 shutdown_connections 2024-09-10T22:01:10.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:10.041+0000 7f69ca600640 1 -- 172.21.15.90:0/1700815374 wait complete. 2024-09-10T22:01:10.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:10 smithi090 ceph-mon[30350]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:10.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:10 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1700815374' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:01:10.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:10 smithi154 ceph-mon[34495]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:10.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:10 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1700815374' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:01:10.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:10 smithi171 ceph-mon[34138]: pgmap v332: 129 pgs: 129 active+clean; 579 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:10.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:10 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1700815374' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:01:11.035 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T22:01:11.035 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:01:11.072 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T22:01:11.484 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:12.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.027+0000 7fbdb5a00640 1 -- 172.21.15.90:0/2664643772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb010cf70 msgr2=0x7fbdb010f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.027+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/2664643772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb010cf70 0x7fbdb010f3d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbda8009ca0 tx=0x7fbda802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.027+0000 7fbdb5a00640 1 -- 172.21.15.90:0/2664643772 shutdown_connections 2024-09-10T22:01:12.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.027+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/2664643772 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb010cf70 0x7fbdb010f3d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.027+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/2664643772 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb0104430 0x7fbdb010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.027+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/2664643772 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb0103b10 0x7fbdb0103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.027+0000 7fbdb5a00640 1 -- 172.21.15.90:0/2664643772 >> 172.21.15.90:0/2664643772 conn(0x7fbdb00fd5b0 msgr2=0x7fbdb00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:12.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.027+0000 7fbdb5a00640 1 -- 172.21.15.90:0/2664643772 shutdown_connections 2024-09-10T22:01:12.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.027+0000 7fbdb5a00640 1 -- 172.21.15.90:0/2664643772 wait complete. 2024-09-10T22:01:12.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.029+0000 7fbdb5a00640 1 Processor -- start 2024-09-10T22:01:12.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.029+0000 7fbdb5a00640 1 -- start start 2024-09-10T22:01:12.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.029+0000 7fbdb5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb0103b10 0x7fbdb019b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:12.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.029+0000 7fbdb5a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb0104430 0x7fbdb019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:12.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.029+0000 7fbdb5a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb010cf70 0x7fbdb01a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:12.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.029+0000 7fbdb5a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0112ae0 con 0x7fbdb0103b10 2024-09-10T22:01:12.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.029+0000 7fbdb5a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0112960 con 0x7fbdb010cf70 2024-09-10T22:01:12.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.029+0000 7fbdb5a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0112c60 con 0x7fbdb0104430 2024-09-10T22:01:12.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb0103b10 0x7fbdb019b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:12.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdafe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb0103b10 0x7fbdb019b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41434/0 (socket says 172.21.15.90:41434) 2024-09-10T22:01:12.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdafe00640 1 -- 172.21.15.90:0/3361649677 learned_addr learned my addr 172.21.15.90:0/3361649677 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:12.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdb5000640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb010cf70 0x7fbdb01a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:12.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdaf400640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb0104430 0x7fbdb019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:12.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdafe00640 1 -- 172.21.15.90:0/3361649677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb0104430 msgr2=0x7fbdb019b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdafe00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb0104430 0x7fbdb019b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdafe00640 1 -- 172.21.15.90:0/3361649677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb010cf70 msgr2=0x7fbdb01a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdafe00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb010cf70 0x7fbdb01a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdafe00640 1 -- 172.21.15.90:0/3361649677 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb01a1140 con 0x7fbdb0103b10 2024-09-10T22:01:12.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.030+0000 7fbdafe00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb0103b10 0x7fbdb019b090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbda000e9f0 tx=0x7fbda000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:12.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.031+0000 7fbdacc00640 1 -- 172.21.15.90:0/3361649677 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda000cd80 con 0x7fbdb0103b10 2024-09-10T22:01:12.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.031+0000 7fbdacc00640 1 -- 172.21.15.90:0/3361649677 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbda00108c0 con 0x7fbdb0103b10 2024-09-10T22:01:12.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.031+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb01a1430 con 0x7fbdb0103b10 2024-09-10T22:01:12.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.031+0000 7fbdacc00640 1 -- 172.21.15.90:0/3361649677 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda00187c0 con 0x7fbdb0103b10 2024-09-10T22:01:12.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.031+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb01a7af0 con 0x7fbdb0103b10 2024-09-10T22:01:12.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.032+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb0103ef0 con 0x7fbdb0103b10 2024-09-10T22:01:12.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.034+0000 7fbdacc00640 1 -- 172.21.15.90:0/3361649677 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fbda0010430 con 0x7fbdb0103b10 2024-09-10T22:01:12.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.035+0000 7fbdacc00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbd940b53e0 0x7fbd940b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:12.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.036+0000 7fbdacc00640 1 -- 172.21.15.90:0/3361649677 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7fbda00e6940 con 0x7fbdb0103b10 2024-09-10T22:01:12.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.036+0000 7fbdaf400640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbd940b53e0 0x7fbd940b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:12.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.036+0000 7fbdaf400640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbd940b53e0 0x7fbd940b78a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c005e30 tx=0x7fbd9c005dc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:12.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.037+0000 7fbdacc00640 1 -- 172.21.15.90:0/3361649677 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbda00ad490 con 0x7fbdb0103b10 2024-09-10T22:01:12.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.234+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbdb01a7eb0 con 0x7fbdb0103b10 2024-09-10T22:01:12.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.235+0000 7fbdacc00640 1 -- 172.21.15.90:0/3361649677 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v86) ==== 81+0+886 (secure 0 0 0) 0x7fbda00acd90 con 0x7fbdb0103b10 2024-09-10T22:01:12.237 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2204308252 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:12.237 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1572343768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:12.237 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1613780567 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:12.237 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1385951279 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:12.237 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/323032163 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:12.237 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3233554768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:12.237 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1613780567 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:12.237 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3858975894 2024-09-11T21:50:49.799954+0000 2024-09-10T22:01:12.238 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2255342982 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:12.238 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3643795744 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:12.238 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2773946294 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:12.238 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2694078227 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:12.238 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/323032163 2024-09-11T21:51:17.626867+0000 2024-09-10T22:01:12.238 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3233554768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:12.238 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3503416190 2024-09-11T21:52:33.477408+0000 2024-09-10T22:01:12.238 INFO:teuthology.orchestra.run.smithi090.stderr:listed 15 entries 2024-09-10T22:01:12.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbd940b53e0 msgr2=0x7fbd940b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbd940b53e0 0x7fbd940b78a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c005e30 tx=0x7fbd9c005dc0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb0103b10 msgr2=0x7fbdb019b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:12.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb0103b10 0x7fbdb019b090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbda000e9f0 tx=0x7fbda000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 shutdown_connections 2024-09-10T22:01:12.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbd940b53e0 0x7fbd940b78a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbdb010cf70 0x7fbdb01a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbdb0104430 0x7fbdb019b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 --2- 172.21.15.90:0/3361649677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdb0103b10 0x7fbdb019b090 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:12.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 >> 172.21.15.90:0/3361649677 conn(0x7fbdb00fd5b0 msgr2=0x7fbdb00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:12.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 shutdown_connections 2024-09-10T22:01:12.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:12.238+0000 7fbdb5a00640 1 -- 172.21.15.90:0/3361649677 wait complete. 2024-09-10T22:01:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:12 smithi154 ceph-mon[34495]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:01:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:12 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3361649677' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:01:12.949 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:12 smithi090 ceph-mon[30350]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:01:12.949 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:12 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3361649677' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:01:12.986 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi090.front.sepia.ceph.com... 2024-09-10T22:01:12.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:12.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T22:01:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:12 smithi171 ceph-mon[34138]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:01:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:12 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3361649677' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:01:13.041 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi090.stderr:ceph-fuse[78999]: fuse finished with error 0 and tester_r 0 2024-09-10T22:01:13.045 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:01:14.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:14 smithi154 ceph-mon[34495]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:01:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:14 smithi171 ceph-mon[34138]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:01:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:14 smithi090 ceph-mon[30350]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-10T22:01:16.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:16 smithi154 ceph-mon[34495]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:16 smithi171 ceph-mon[34138]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:16 smithi090 ceph-mon[30350]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:18.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:18 smithi154 ceph-mon[34495]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:18 smithi171 ceph-mon[34138]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:18 smithi090 ceph-mon[30350]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:19.147 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-09-10T22:01:19.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:01:19.147 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T22:01:19.199 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:01:19.576 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:20.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.256+0000 7f2c89e00640 1 -- 172.21.15.90:0/1846056135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84102dc0 msgr2=0x7f2c841031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.256+0000 7f2c89e00640 1 --2- 172.21.15.90:0/1846056135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84102dc0 0x7f2c841031a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009900 tx=0x7f2c7402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.256+0000 7f2c89e00640 1 -- 172.21.15.90:0/1846056135 shutdown_connections 2024-09-10T22:01:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.256+0000 7f2c89e00640 1 --2- 172.21.15.90:0/1846056135 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c84109150 0x7f2c8410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.256+0000 7f2c89e00640 1 --2- 172.21.15.90:0/1846056135 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c841087d0 0x7f2c84108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.256+0000 7f2c89e00640 1 --2- 172.21.15.90:0/1846056135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84102dc0 0x7f2c841031a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.256+0000 7f2c89e00640 1 -- 172.21.15.90:0/1846056135 >> 172.21.15.90:0/1846056135 conn(0x7f2c840fe830 msgr2=0x7f2c84100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.256+0000 7f2c89e00640 1 -- 172.21.15.90:0/1846056135 shutdown_connections 2024-09-10T22:01:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.256+0000 7f2c89e00640 1 -- 172.21.15.90:0/1846056135 wait complete. 2024-09-10T22:01:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.257+0000 7f2c89e00640 1 Processor -- start 2024-09-10T22:01:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.258+0000 7f2c89e00640 1 -- start start 2024-09-10T22:01:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.258+0000 7f2c89e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c84102dc0 0x7f2c8419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.258+0000 7f2c89e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c841087d0 0x7f2c8419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.258+0000 7f2c89e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84109150 0x7f2c841a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.258+0000 7f2c89e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84112cd0 con 0x7f2c84109150 2024-09-10T22:01:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.258+0000 7f2c89e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84112b50 con 0x7f2c841087d0 2024-09-10T22:01:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.258+0000 7f2c89e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2c84112e50 con 0x7f2c84102dc0 2024-09-10T22:01:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.258+0000 7f2c83400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84109150 0x7f2c841a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c82a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c84102dc0 0x7f2c8419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c83400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84109150 0x7f2c841a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40486/0 (socket says 172.21.15.90:40486) 2024-09-10T22:01:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c83400640 1 -- 172.21.15.90:0/355584883 learned_addr learned my addr 172.21.15.90:0/355584883 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c82000640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c841087d0 0x7f2c8419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:20.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c83400640 1 -- 172.21.15.90:0/355584883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c84102dc0 msgr2=0x7f2c8419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:20.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c83400640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c84102dc0 0x7f2c8419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c83400640 1 -- 172.21.15.90:0/355584883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c841087d0 msgr2=0x7f2c8419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:20.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c83400640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c841087d0 0x7f2c8419b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c83400640 1 -- 172.21.15.90:0/355584883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c841a1470 con 0x7f2c84109150 2024-09-10T22:01:20.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.259+0000 7f2c83400640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84109150 0x7f2c841a0ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00cce0 tx=0x7f2c7c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:20.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.260+0000 7f2c88a00640 1 -- 172.21.15.90:0/355584883 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00f450 con 0x7f2c84109150 2024-09-10T22:01:20.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.260+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c841a1760 con 0x7f2c84109150 2024-09-10T22:01:20.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.260+0000 7f2c88a00640 1 -- 172.21.15.90:0/355584883 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c7c00fa80 con 0x7f2c84109150 2024-09-10T22:01:20.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.260+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841a7e20 con 0x7f2c84109150 2024-09-10T22:01:20.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.260+0000 7f2c88a00640 1 -- 172.21.15.90:0/355584883 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c018ad0 con 0x7f2c84109150 2024-09-10T22:01:20.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.261+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c841031a0 con 0x7f2c84109150 2024-09-10T22:01:20.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.263+0000 7f2c88a00640 1 -- 172.21.15.90:0/355584883 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f2c7c00f5f0 con 0x7f2c84109150 2024-09-10T22:01:20.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.265+0000 7f2c88a00640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2c680b5330 0x7f2c680b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:20.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.265+0000 7f2c88a00640 1 -- 172.21.15.90:0/355584883 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f2c7c0e6680 con 0x7f2c84109150 2024-09-10T22:01:20.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.265+0000 7f2c82a00640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2c680b5330 0x7f2c680b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:20.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.266+0000 7f2c82a00640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2c680b5330 0x7f2c680b77f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c74006010 tx=0x7f2c74005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:20.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.266+0000 7f2c88a00640 1 -- 172.21.15.90:0/355584883 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2c7c014030 con 0x7f2c84109150 2024-09-10T22:01:20.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.451+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c8419c4f0 con 0x7f2c84109150 2024-09-10T22:01:20.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.452+0000 7f2c88a00640 1 -- 172.21.15.90:0/355584883 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24540 (secure 0 0 0) 0x7f2c7c0ad260 con 0x7f2c84109150 2024-09-10T22:01:20.454 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:01:20.455 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:00:47.831865+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:01:20.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2c680b5330 msgr2=0x7f2c680b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:20.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2c680b5330 0x7f2c680b77f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c74006010 tx=0x7f2c74005e80 comp rx=0 tx=0).stop 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84109150 msgr2=0x7f2c841a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84109150 0x7f2c841a0ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00cce0 tx=0x7f2c7c00a620 comp rx=0 tx=0).stop 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 shutdown_connections 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2c680b5330 0x7f2c680b77f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c84109150 0x7f2c841a0ca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2c841087d0 0x7f2c8419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 --2- 172.21.15.90:0/355584883 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2c84102dc0 0x7f2c8419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 >> 172.21.15.90:0/355584883 conn(0x7f2c840fe830 msgr2=0x7f2c84104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 shutdown_connections 2024-09-10T22:01:20.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:20.455+0000 7f2c89e00640 1 -- 172.21.15.90:0/355584883 wait complete. 2024-09-10T22:01:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:20 smithi090 ceph-mon[30350]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:20 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/355584883' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:20 smithi154 ceph-mon[34495]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:20 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/355584883' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:20 smithi171 ceph-mon[34138]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:20 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/355584883' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:21.472 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:01:21.844 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:22.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.408+0000 7f52da400640 1 -- 172.21.15.90:0/997408482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d4104f90 msgr2=0x7f52d4105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.408+0000 7f52da400640 1 --2- 172.21.15.90:0/997408482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d4104f90 0x7f52d4105370 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009ca0 tx=0x7f52c402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.408+0000 7f52da400640 1 -- 172.21.15.90:0/997408482 shutdown_connections 2024-09-10T22:01:22.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.408+0000 7f52da400640 1 --2- 172.21.15.90:0/997408482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d410b320 0x7f52d4111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.408+0000 7f52da400640 1 --2- 172.21.15.90:0/997408482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d410a9a0 0x7f52d410ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.408+0000 7f52da400640 1 --2- 172.21.15.90:0/997408482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d4104f90 0x7f52d4105370 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.408+0000 7f52da400640 1 -- 172.21.15.90:0/997408482 >> 172.21.15.90:0/997408482 conn(0x7f52d4100a00 msgr2=0x7f52d4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.408+0000 7f52da400640 1 -- 172.21.15.90:0/997408482 shutdown_connections 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.408+0000 7f52da400640 1 -- 172.21.15.90:0/997408482 wait complete. 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.409+0000 7f52da400640 1 Processor -- start 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.409+0000 7f52da400640 1 -- start start 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52da400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d4104f90 0x7f52d419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52da400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d410a9a0 0x7f52d419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52da400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d410b320 0x7f52d41a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52da400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4114e50 con 0x7f52d410b320 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52da400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4114cd0 con 0x7f52d4104f90 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52da400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52d4114fd0 con 0x7f52d410a9a0 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52d3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d410b320 0x7f52d41a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52d3400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d4104f90 0x7f52d419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52d3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d410b320 0x7f52d41a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40492/0 (socket says 172.21.15.90:40492) 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52d3e00640 1 -- 172.21.15.90:0/306441654 learned_addr learned my addr 172.21.15.90:0/306441654 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52d2a00640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d410a9a0 0x7f52d419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52d3e00640 1 -- 172.21.15.90:0/306441654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d410a9a0 msgr2=0x7f52d419b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52d3e00640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d410a9a0 0x7f52d419b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52d3e00640 1 -- 172.21.15.90:0/306441654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d4104f90 msgr2=0x7f52d419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.410+0000 7f52d3e00640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d4104f90 0x7f52d419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.411+0000 7f52d3e00640 1 -- 172.21.15.90:0/306441654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d41a1450 con 0x7f52d410b320 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.411+0000 7f52d3e00640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d410b320 0x7f52d41a0c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c970 tx=0x7f52cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.411+0000 7f52cbe00640 1 -- 172.21.15.90:0/306441654 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc007bb0 con 0x7f52d410b320 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.411+0000 7f52cbe00640 1 -- 172.21.15.90:0/306441654 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52cc007d50 con 0x7f52d410b320 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.411+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d41a1740 con 0x7f52d410b320 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.411+0000 7f52cbe00640 1 -- 172.21.15.90:0/306441654 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52cc018660 con 0x7f52d410b320 2024-09-10T22:01:22.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.411+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d41a7e00 con 0x7f52d410b320 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.412+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d4105370 con 0x7f52d410b320 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.414+0000 7f52cbe00640 1 -- 172.21.15.90:0/306441654 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f52cc020030 con 0x7f52d410b320 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.416+0000 7f52cbe00640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52b80b5330 0x7f52b80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.416+0000 7f52cbe00640 1 -- 172.21.15.90:0/306441654 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f52cc0e5d50 con 0x7f52d410b320 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.416+0000 7f52d3400640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52b80b5330 0x7f52b80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.417+0000 7f52d3400640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52b80b5330 0x7f52b80b77f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52c4000f80 tx=0x7f52c4031000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:22.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.418+0000 7f52cbe00640 1 -- 172.21.15.90:0/306441654 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f52cc056220 con 0x7f52d410b320 2024-09-10T22:01:22.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.609+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52d402e5c0 con 0x7f52d410b320 2024-09-10T22:01:22.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.610+0000 7f52cbe00640 1 -- 172.21.15.90:0/306441654 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24540 (secure 0 0 0) 0x7f52cc014030 con 0x7f52d410b320 2024-09-10T22:01:22.613 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:01:22.613 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":86,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:00:47.831865+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:01:22.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52b80b5330 msgr2=0x7f52b80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52b80b5330 0x7f52b80b77f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52c4000f80 tx=0x7f52c4031000 comp rx=0 tx=0).stop 2024-09-10T22:01:22.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d410b320 msgr2=0x7f52d41a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:22.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d410b320 0x7f52d41a0c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52cc00c970 tx=0x7f52cc00ce30 comp rx=0 tx=0).stop 2024-09-10T22:01:22.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 shutdown_connections 2024-09-10T22:01:22.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52b80b5330 0x7f52b80b77f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52d410b320 0x7f52d41a0c80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52d410a9a0 0x7f52d419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 --2- 172.21.15.90:0/306441654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52d4104f90 0x7f52d419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:22.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 >> 172.21.15.90:0/306441654 conn(0x7f52d4100a00 msgr2=0x7f52d4102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:22.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 shutdown_connections 2024-09-10T22:01:22.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:22.613+0000 7f52da400640 1 -- 172.21.15.90:0/306441654 wait complete. 2024-09-10T22:01:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:22 smithi171 ceph-mon[34138]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:22 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/306441654' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:23.003 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:22 smithi090 ceph-mon[30350]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:23.004 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:22 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/306441654' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:22 smithi154 ceph-mon[34495]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T22:01:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:22 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/306441654' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:23.516 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T22:01:23.889 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.510+0000 7f5fbf200640 1 -- 172.21.15.90:0/740328661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0102c10 msgr2=0x7f5fc0102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.510+0000 7f5fbf200640 1 --2- 172.21.15.90:0/740328661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0102c10 0x7f5fc0102ff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009900 tx=0x7f5fb002a400 comp rx=0 tx=0).stop 2024-09-10T22:01:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.510+0000 7f5fbf200640 1 -- 172.21.15.90:0/740328661 shutdown_connections 2024-09-10T22:01:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.510+0000 7f5fbf200640 1 --2- 172.21.15.90:0/740328661 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0108fa0 0x7f5fc010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.510+0000 7f5fbf200640 1 --2- 172.21.15.90:0/740328661 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fc0108620 0x7f5fc0108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.510+0000 7f5fbf200640 1 --2- 172.21.15.90:0/740328661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0102c10 0x7f5fc0102ff0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.510+0000 7f5fbf200640 1 -- 172.21.15.90:0/740328661 >> 172.21.15.90:0/740328661 conn(0x7f5fc00fe660 msgr2=0x7f5fc0100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:24.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.510+0000 7f5fbf200640 1 -- 172.21.15.90:0/740328661 shutdown_connections 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.510+0000 7f5fbf200640 1 -- 172.21.15.90:0/740328661 wait complete. 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.511+0000 7f5fbf200640 1 Processor -- start 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.511+0000 7f5fbf200640 1 -- start start 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.511+0000 7f5fbf200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fc0102c10 0x7f5fc019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.511+0000 7f5fbf200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0108620 0x7f5fc019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.511+0000 7f5fbf200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0108fa0 0x7f5fc01a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbf200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc0112b20 con 0x7f5fc0108fa0 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbf200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc01129a0 con 0x7f5fc0108620 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbf200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc0112ca0 con 0x7f5fc0102c10 2024-09-10T22:01:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbe800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0108fa0 0x7f5fc01a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbe800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0108fa0 0x7f5fc01a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40514/0 (socket says 172.21.15.90:40514) 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbd400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0108620 0x7f5fc019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbe800640 1 -- 172.21.15.90:0/3488962203 learned_addr learned my addr 172.21.15.90:0/3488962203 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbde00640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fc0102c10 0x7f5fc019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbe800640 1 -- 172.21.15.90:0/3488962203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fc0102c10 msgr2=0x7f5fc019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbe800640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fc0102c10 0x7f5fc019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbe800640 1 -- 172.21.15.90:0/3488962203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0108620 msgr2=0x7f5fc019b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbe800640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0108620 0x7f5fc019b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.512+0000 7f5fbe800640 1 -- 172.21.15.90:0/3488962203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc01a1240 con 0x7f5fc0108fa0 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.513+0000 7f5fbe800640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0108fa0 0x7f5fc01a0a70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800efd0 tx=0x7f5fb800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.513+0000 7f5fb6a00640 1 -- 172.21.15.90:0/3488962203 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8003b70 con 0x7f5fc0108fa0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.513+0000 7f5fb6a00640 1 -- 172.21.15.90:0/3488962203 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5fb8003d10 con 0x7f5fc0108fa0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.513+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc01a1530 con 0x7f5fc0108fa0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.513+0000 7f5fb6a00640 1 -- 172.21.15.90:0/3488962203 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8010850 con 0x7f5fc0108fa0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.513+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc01a7bf0 con 0x7f5fc0108fa0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.514+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc0102ff0 con 0x7f5fc0108fa0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.516+0000 7f5fb6a00640 1 -- 172.21.15.90:0/3488962203 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f5fb801f020 con 0x7f5fc0108fa0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.519+0000 7f5fb6a00640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5fa40b53e0 0x7f5fa40b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.519+0000 7f5fb6a00640 1 -- 172.21.15.90:0/3488962203 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f5fb80e6090 con 0x7f5fc0108fa0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.519+0000 7f5fbde00640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5fa40b53e0 0x7f5fa40b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.520+0000 7f5fbde00640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5fa40b53e0 0x7f5fa40b78a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0006010 tx=0x7f5fb0005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:24.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.523+0000 7f5fb6a00640 1 -- 172.21.15.90:0/3488962203 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5fb80acbe0 con 0x7f5fc0108fa0 2024-09-10T22:01:24.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:24.754+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5fc002e5c0 con 0x7f5fc0108fa0 2024-09-10T22:01:24.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:24 smithi090 ceph-mon[30350]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:01:24.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:24 smithi171 ceph-mon[34138]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:01:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:24 smithi154 ceph-mon[34495]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:01:25.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.678+0000 7f5fb6a00640 1 -- 172.21.15.90:0/3488962203 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f5fb80ac4e0 con 0x7f5fc0108fa0 2024-09-10T22:01:25.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.681+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5fa40b53e0 msgr2=0x7f5fa40b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:25.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.681+0000 7f5fbf200640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5fa40b53e0 0x7f5fa40b78a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0006010 tx=0x7f5fb0005e80 comp rx=0 tx=0).stop 2024-09-10T22:01:25.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.681+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0108fa0 msgr2=0x7f5fc01a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:25.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.681+0000 7f5fbf200640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0108fa0 0x7f5fc01a0a70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800efd0 tx=0x7f5fb800c6a0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.682+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 shutdown_connections 2024-09-10T22:01:25.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.682+0000 7f5fbf200640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5fa40b53e0 0x7f5fa40b78a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.682+0000 7f5fbf200640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fc0108fa0 0x7f5fc01a0a70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.682+0000 7f5fbf200640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5fc0108620 0x7f5fc019b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.682+0000 7f5fbf200640 1 --2- 172.21.15.90:0/3488962203 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5fc0102c10 0x7f5fc019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:25.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.682+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 >> 172.21.15.90:0/3488962203 conn(0x7f5fc00fe660 msgr2=0x7f5fc0104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:25.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.682+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 shutdown_connections 2024-09-10T22:01:25.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:25.682+0000 7f5fbf200640 1 -- 172.21.15.90:0/3488962203 wait complete. 2024-09-10T22:01:25.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:25 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3488962203' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:01:26.069 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:25 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3488962203' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:01:26.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:25 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3488962203' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T22:01:26.573 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-10T22:01:26.573 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-10T22:01:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:26 smithi090 ceph-mon[30350]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:01:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:26 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3488962203' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:01:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:26 smithi090 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:01:26.947 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:26 smithi171 ceph-mon[34138]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:01:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:26 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3488962203' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:01:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:26 smithi171 ceph-mon[34138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:01:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:26 smithi154 ceph-mon[34495]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-10T22:01:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:26 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3488962203' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T22:01:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:26 smithi154 ceph-mon[34495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:01:27.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.531+0000 7f33afe00640 1 -- 172.21.15.90:0/2647171464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 msgr2=0x7f33a8105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.531+0000 7f33afe00640 1 --2- 172.21.15.90:0/2647171464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 0x7f33a8105370 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f339c009900 tx=0x7f339c02a410 comp rx=0 tx=0).stop 2024-09-10T22:01:27.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.531+0000 7f33afe00640 1 -- 172.21.15.90:0/2647171464 shutdown_connections 2024-09-10T22:01:27.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.531+0000 7f33afe00640 1 --2- 172.21.15.90:0/2647171464 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33a810b320 0x7f33a8111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.531+0000 7f33afe00640 1 --2- 172.21.15.90:0/2647171464 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a810a9a0 0x7f33a810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.531+0000 7f33afe00640 1 --2- 172.21.15.90:0/2647171464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 0x7f33a8105370 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.531+0000 7f33afe00640 1 -- 172.21.15.90:0/2647171464 >> 172.21.15.90:0/2647171464 conn(0x7f33a8100a00 msgr2=0x7f33a8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:27.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.531+0000 7f33afe00640 1 -- 172.21.15.90:0/2647171464 shutdown_connections 2024-09-10T22:01:27.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.531+0000 7f33afe00640 1 -- 172.21.15.90:0/2647171464 wait complete. 2024-09-10T22:01:27.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.532+0000 7f33afe00640 1 Processor -- start 2024-09-10T22:01:27.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.532+0000 7f33afe00640 1 -- start start 2024-09-10T22:01:27.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 0x7f33a819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33afe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a810a9a0 0x7f33a819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33afe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33a810b320 0x7f33a81a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33afe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8114e00 con 0x7f33a8104f90 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33afe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8114c80 con 0x7f33a810b320 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33afe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f33a8114f80 con 0x7f33a810a9a0 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33ad400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 0x7f33a819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33ad400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 0x7f33a819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40542/0 (socket says 172.21.15.90:40542) 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33ad400640 1 -- 172.21.15.90:0/821934972 learned_addr learned my addr 172.21.15.90:0/821934972 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33ade00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33a810b320 0x7f33a81a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33aca00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a810a9a0 0x7f33a819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33ad400640 1 -- 172.21.15.90:0/821934972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a810a9a0 msgr2=0x7f33a819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33ad400640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a810a9a0 0x7f33a819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33ad400640 1 -- 172.21.15.90:0/821934972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33a810b320 msgr2=0x7f33a81a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33ad400640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33a810b320 0x7f33a81a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:27.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33ad400640 1 -- 172.21.15.90:0/821934972 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a81a14a0 con 0x7f33a8104f90 2024-09-10T22:01:27.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.533+0000 7f33aca00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a810a9a0 0x7f33a819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:27.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.534+0000 7f33ad400640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 0x7f33a819b3f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f339c009230 tx=0x7f339c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:27.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.534+0000 7f33a6000640 1 -- 172.21.15.90:0/821934972 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c03f030 con 0x7f33a8104f90 2024-09-10T22:01:27.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.534+0000 7f33a6000640 1 -- 172.21.15.90:0/821934972 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f339c003a80 con 0x7f33a8104f90 2024-09-10T22:01:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.534+0000 7f33a6000640 1 -- 172.21.15.90:0/821934972 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c035070 con 0x7f33a8104f90 2024-09-10T22:01:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.534+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a81a1730 con 0x7f33a8104f90 2024-09-10T22:01:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.534+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a81a7e50 con 0x7f33a8104f90 2024-09-10T22:01:27.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.535+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a8105370 con 0x7f33a8104f90 2024-09-10T22:01:27.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.537+0000 7f33a6000640 1 -- 172.21.15.90:0/821934972 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f339c03bc30 con 0x7f33a8104f90 2024-09-10T22:01:27.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.539+0000 7f33a6000640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f33900b53b0 0x7f33900b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:27.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.539+0000 7f33a6000640 1 -- 172.21.15.90:0/821934972 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f339c106960 con 0x7f33a8104f90 2024-09-10T22:01:27.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.539+0000 7f33aca00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f33900b53b0 0x7f33900b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:27.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.540+0000 7f33aca00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f33900b53b0 0x7f33900b7870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f33940058c0 tx=0x7f3394005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:27.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.543+0000 7f33a6000640 1 -- 172.21.15.90:0/821934972 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f339c0cd4b0 con 0x7f33a8104f90 2024-09-10T22:01:27.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:27.765+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f33a802e5c0 con 0x7f33a8104f90 2024-09-10T22:01:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.690+0000 7f33a6000640 1 -- 172.21.15.90:0/821934972 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f339c0ccdb0 con 0x7f33a8104f90 2024-09-10T22:01:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.693+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f33900b53b0 msgr2=0x7f33900b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.693+0000 7f33afe00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f33900b53b0 0x7f33900b7870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f33940058c0 tx=0x7f3394005850 comp rx=0 tx=0).stop 2024-09-10T22:01:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.693+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 msgr2=0x7f33a819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.693+0000 7f33afe00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 0x7f33a819b3f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f339c009230 tx=0x7f339c004270 comp rx=0 tx=0).stop 2024-09-10T22:01:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.693+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 shutdown_connections 2024-09-10T22:01:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.694+0000 7f33afe00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f33900b53b0 0x7f33900b7870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.694+0000 7f33afe00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f33a810b320 0x7f33a81a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.694+0000 7f33afe00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f33a810a9a0 0x7f33a819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.694+0000 7f33afe00640 1 --2- 172.21.15.90:0/821934972 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f33a8104f90 0x7f33a819b3f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.694+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 >> 172.21.15.90:0/821934972 conn(0x7f33a8100a00 msgr2=0x7f33a8102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.694+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 shutdown_connections 2024-09-10T22:01:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:28.694+0000 7f33afe00640 1 -- 172.21.15.90:0/821934972 wait complete. 2024-09-10T22:01:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:28 smithi171 ceph-mon[34138]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:28 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/821934972' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-10T22:01:29.094 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:28 smithi090 ceph-mon[30350]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:29.094 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:28 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/821934972' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-10T22:01:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:28 smithi154 ceph-mon[34495]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:01:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:28 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/821934972' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-10T22:01:29.538 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T22:01:29.916 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:29.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:29 smithi171 ceph-mon[34138]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T22:01:29.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:29 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/821934972' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T22:01:29.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:29 smithi171 ceph-mon[34138]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T22:01:29.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:29 smithi171 ceph-mon[34138]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:01:29.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:29 smithi171 ceph-mon[34138]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T22:01:29.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:29 smithi171 ceph-mon[34138]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T22:01:30.041 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:29 smithi090 ceph-mon[30350]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T22:01:30.041 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:29 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/821934972' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T22:01:30.041 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:29 smithi090 ceph-mon[30350]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T22:01:30.041 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:29 smithi090 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:01:30.041 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:29 smithi090 ceph-mon[30350]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T22:01:30.041 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:29 smithi090 ceph-mon[30350]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T22:01:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:29 smithi154 ceph-mon[34495]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T22:01:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:29 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/821934972' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T22:01:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:29 smithi154 ceph-mon[34495]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T22:01:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:29 smithi154 ceph-mon[34495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T22:01:30.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:29 smithi154 ceph-mon[34495]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-10T22:01:30.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:29 smithi154 ceph-mon[34495]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-10T22:01:30.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.874+0000 7f2f33800640 1 -- 172.21.15.90:0/2206580048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c113b20 msgr2=0x7f2f2c115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:30.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.874+0000 7f2f33800640 1 --2- 172.21.15.90:0/2206580048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c113b20 0x7f2f2c115f10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009ca0 tx=0x7f2f2402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.874+0000 7f2f33800640 1 -- 172.21.15.90:0/2206580048 shutdown_connections 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.874+0000 7f2f33800640 1 --2- 172.21.15.90:0/2206580048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c113b20 0x7f2f2c115f10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.874+0000 7f2f33800640 1 --2- 172.21.15.90:0/2206580048 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c075e60 0x7f2f2c1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.874+0000 7f2f33800640 1 --2- 172.21.15.90:0/2206580048 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c075540 0x7f2f2c075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.874+0000 7f2f33800640 1 -- 172.21.15.90:0/2206580048 >> 172.21.15.90:0/2206580048 conn(0x7f2f2c0fe830 msgr2=0x7f2f2c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.874+0000 7f2f33800640 1 -- 172.21.15.90:0/2206580048 shutdown_connections 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.874+0000 7f2f33800640 1 -- 172.21.15.90:0/2206580048 wait complete. 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f33800640 1 Processor -- start 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f33800640 1 -- start start 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f33800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c075540 0x7f2f2c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f33800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e60 0x7f2f2c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f33800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c113b20 0x7f2f2c1a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:30.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f33800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c119290 con 0x7f2f2c075e60 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f33800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c119110 con 0x7f2f2c113b20 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f33800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c119410 con 0x7f2f2c075540 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f2be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e60 0x7f2f2c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f2be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e60 0x7f2f2c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54764/0 (socket says 172.21.15.90:54764) 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.875+0000 7f2f2be00640 1 -- 172.21.15.90:0/3450922997 learned_addr learned my addr 172.21.15.90:0/3450922997 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f2be00640 1 -- 172.21.15.90:0/3450922997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c075540 msgr2=0x7f2f2c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f2be00640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c075540 0x7f2f2c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f31800640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c113b20 0x7f2f2c1a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f2be00640 1 -- 172.21.15.90:0/3450922997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c113b20 msgr2=0x7f2f2c1a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f2be00640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c113b20 0x7f2f2c1a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:30.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f2be00640 1 -- 172.21.15.90:0/3450922997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c1a5870 con 0x7f2f2c075e60 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f31800640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c113b20 0x7f2f2c1a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f2be00640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e60 0x7f2f2c19fd00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800d950 tx=0x7f2f1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f29600640 1 -- 172.21.15.90:0/3450922997 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18017070 con 0x7f2f2c075e60 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f29600640 1 -- 172.21.15.90:0/3450922997 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f18009c30 con 0x7f2f2c075e60 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f29600640 1 -- 172.21.15.90:0/3450922997 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800f620 con 0x7f2f2c075e60 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.876+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c1a5b60 con 0x7f2f2c075e60 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.877+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c1ac220 con 0x7f2f2c075e60 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.878+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2c075920 con 0x7f2f2c075e60 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.880+0000 7f2f29600640 1 -- 172.21.15.90:0/3450922997 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f2f18010460 con 0x7f2f2c075e60 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.881+0000 7f2f29600640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f100b5430 0x7f2f100b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.881+0000 7f2f29600640 1 -- 172.21.15.90:0/3450922997 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7f2f180e5c10 con 0x7f2f2c075e60 2024-09-10T22:01:30.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.881+0000 7f2f30e00640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f100b5430 0x7f2f100b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:30.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.882+0000 7f2f30e00640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f100b5430 0x7f2f100b78f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c005fd0 tx=0x7f2f1c005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:30.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:30.885+0000 7f2f29600640 1 -- 172.21.15.90:0/3450922997 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2f180ac760 con 0x7f2f2c075e60 2024-09-10T22:01:30.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:30 smithi171 ceph-mon[34138]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T22:01:30.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:30 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] up:active 2024-09-10T22:01:30.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:30 smithi171 ceph-mon[34138]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T22:01:30.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:30 smithi171 ceph-mon[34138]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T22:01:30.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:30 smithi171 ceph-mon[34138]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=f=up:creating} 9 up:standby 2024-09-10T22:01:30.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:30 smithi171 ceph-mon[34138]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-09-10T22:01:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:30 smithi090 ceph-mon[30350]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T22:01:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:30 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] up:active 2024-09-10T22:01:31.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:30 smithi090 ceph-mon[30350]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T22:01:31.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:30 smithi090 ceph-mon[30350]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T22:01:31.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:30 smithi090 ceph-mon[30350]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=f=up:creating} 9 up:standby 2024-09-10T22:01:31.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:30 smithi090 ceph-mon[30350]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-09-10T22:01:31.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.110+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2f2c1a08f0 con 0x7f2f2c075e60 2024-09-10T22:01:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:30 smithi154 ceph-mon[34495]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T22:01:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:30 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] up:active 2024-09-10T22:01:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:30 smithi154 ceph-mon[34495]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T22:01:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:30 smithi154 ceph-mon[34495]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-10T22:01:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:30 smithi154 ceph-mon[34495]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=f=up:creating} 9 up:standby 2024-09-10T22:01:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:30 smithi154 ceph-mon[34495]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-09-10T22:01:31.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.723+0000 7f2f29600640 1 -- 172.21.15.90:0/3450922997 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f2f18009de0 con 0x7f2f2c075e60 2024-09-10T22:01:31.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f100b5430 msgr2=0x7f2f100b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:31.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f100b5430 0x7f2f100b78f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c005fd0 tx=0x7f2f1c005850 comp rx=0 tx=0).stop 2024-09-10T22:01:31.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e60 msgr2=0x7f2f2c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:31.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e60 0x7f2f2c19fd00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800d950 tx=0x7f2f1800de10 comp rx=0 tx=0).stop 2024-09-10T22:01:31.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 shutdown_connections 2024-09-10T22:01:31.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f100b5430 0x7f2f100b78f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:31.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c113b20 0x7f2f2c1a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:31.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e60 0x7f2f2c19fd00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:31.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 --2- 172.21.15.90:0/3450922997 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c075540 0x7f2f2c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:31.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 >> 172.21.15.90:0/3450922997 conn(0x7f2f2c0fe830 msgr2=0x7f2f2c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:31.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 shutdown_connections 2024-09-10T22:01:31.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:31.727+0000 7f2f33800640 1 -- 172.21.15.90:0/3450922997 wait complete. 2024-09-10T22:01:31.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:31 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] up:active 2024-09-10T22:01:31.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:31 smithi171 ceph-mon[34138]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T22:01:31.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:31 smithi171 ceph-mon[34138]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=f=up:active} 9 up:standby 2024-09-10T22:01:31.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:31 smithi171 ceph-mon[34138]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=f=up:active,3=e=up:creating} 8 up:standby 2024-09-10T22:01:31.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:31 smithi171 ceph-mon[34138]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-10T22:01:31.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:31 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3450922997' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:31 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] up:active 2024-09-10T22:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:31 smithi090 ceph-mon[30350]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T22:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:31 smithi090 ceph-mon[30350]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=f=up:active} 9 up:standby 2024-09-10T22:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:31 smithi090 ceph-mon[30350]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=f=up:active,3=e=up:creating} 8 up:standby 2024-09-10T22:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:31 smithi090 ceph-mon[30350]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-10T22:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:31 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3450922997' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:01:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:31 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] up:active 2024-09-10T22:01:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:31 smithi154 ceph-mon[34495]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T22:01:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:31 smithi154 ceph-mon[34495]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=f=up:active} 9 up:standby 2024-09-10T22:01:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:31 smithi154 ceph-mon[34495]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=f=up:active,3=e=up:creating} 8 up:standby 2024-09-10T22:01:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:31 smithi154 ceph-mon[34495]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-09-10T22:01:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:31 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3450922997' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T22:01:32.658 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T22:01:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-09-10T22:01:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3450922997' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:01:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] up:active 2024-09-10T22:01:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T22:01:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T22:01:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: Cluster is now healthy 2024-09-10T22:01:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=f=up:active,3=e=up:active} 8 up:standby 2024-09-10T22:01:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T22:01:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-10T22:01:32.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:32 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:01:33.056 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-09-10T22:01:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3450922997' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:01:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] up:active 2024-09-10T22:01:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T22:01:33.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T22:01:33.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: Cluster is now healthy 2024-09-10T22:01:33.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=f=up:active,3=e=up:active} 8 up:standby 2024-09-10T22:01:33.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T22:01:33.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-10T22:01:33.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:32 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:01:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: pgmap v343: 129 pgs: 129 active+clean; 581 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-09-10T22:01:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3450922997' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T22:01:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] up:active 2024-09-10T22:01:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T22:01:33.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T22:01:33.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: Cluster is now healthy 2024-09-10T22:01:33.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=f=up:active,3=e=up:active} 8 up:standby 2024-09-10T22:01:33.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T22:01:33.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-10T22:01:33.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:32 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:01:33.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.758+0000 7fdd9b000640 1 -- 172.21.15.90:0/3478258758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd941087d0 msgr2=0x7fdd94108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:33.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.758+0000 7fdd9b000640 1 --2- 172.21.15.90:0/3478258758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd941087d0 0x7fdd94108c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009900 tx=0x7fdd7c02a410 comp rx=0 tx=0).stop 2024-09-10T22:01:33.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.759+0000 7fdd9b000640 1 -- 172.21.15.90:0/3478258758 shutdown_connections 2024-09-10T22:01:33.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.759+0000 7fdd9b000640 1 --2- 172.21.15.90:0/3478258758 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd94109150 0x7fdd9410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:33.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.759+0000 7fdd9b000640 1 --2- 172.21.15.90:0/3478258758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd941087d0 0x7fdd94108c10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:33.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.759+0000 7fdd9b000640 1 --2- 172.21.15.90:0/3478258758 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd94102dc0 0x7fdd941031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:33.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.759+0000 7fdd9b000640 1 -- 172.21.15.90:0/3478258758 >> 172.21.15.90:0/3478258758 conn(0x7fdd940fe830 msgr2=0x7fdd94100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:33.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.759+0000 7fdd9b000640 1 -- 172.21.15.90:0/3478258758 shutdown_connections 2024-09-10T22:01:33.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.759+0000 7fdd9b000640 1 -- 172.21.15.90:0/3478258758 wait complete. 2024-09-10T22:01:33.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.760+0000 7fdd9b000640 1 Processor -- start 2024-09-10T22:01:33.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.760+0000 7fdd9b000640 1 -- start start 2024-09-10T22:01:33.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.760+0000 7fdd9b000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd94102dc0 0x7fdd9419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:33.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.760+0000 7fdd9b000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd941087d0 0x7fdd9419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:33.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd9b000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd94109150 0x7fdd941a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:33.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd9b000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94112cd0 con 0x7fdd94109150 2024-09-10T22:01:33.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd9b000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94112b50 con 0x7fdd94102dc0 2024-09-10T22:01:33.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd9b000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94112e50 con 0x7fdd941087d0 2024-09-10T22:01:33.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd93e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd94102dc0 0x7fdd9419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:33.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd93e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd94102dc0 0x7fdd9419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:42000/0 (socket says 172.21.15.90:42000) 2024-09-10T22:01:33.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd93e00640 1 -- 172.21.15.90:0/916118581 learned_addr learned my addr 172.21.15.90:0/916118581 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:33.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd99000640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd94109150 0x7fdd941a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:33.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd8b400640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd941087d0 0x7fdd9419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:33.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd93e00640 1 -- 172.21.15.90:0/916118581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd941087d0 msgr2=0x7fdd9419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:33.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd93e00640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd941087d0 0x7fdd9419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:33.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd93e00640 1 -- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd94109150 msgr2=0x7fdd941a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:33.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd93e00640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd94109150 0x7fdd941a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:33.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.761+0000 7fdd93e00640 1 -- 172.21.15.90:0/916118581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd941a1430 con 0x7fdd94102dc0 2024-09-10T22:01:33.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.762+0000 7fdd93e00640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd94102dc0 0x7fdd9419b380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400ebc0 tx=0x7fdd8400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:33.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.762+0000 7fdd91600640 1 -- 172.21.15.90:0/916118581 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84010630 con 0x7fdd94102dc0 2024-09-10T22:01:33.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.762+0000 7fdd91600640 1 -- 172.21.15.90:0/916118581 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd84010c60 con 0x7fdd94102dc0 2024-09-10T22:01:33.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.762+0000 7fdd91600640 1 -- 172.21.15.90:0/916118581 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84018b00 con 0x7fdd94102dc0 2024-09-10T22:01:33.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.762+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd941a1720 con 0x7fdd94102dc0 2024-09-10T22:01:33.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.762+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd941a7de0 con 0x7fdd94102dc0 2024-09-10T22:01:33.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.763+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd941031a0 con 0x7fdd94102dc0 2024-09-10T22:01:33.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.767+0000 7fdd91600640 1 -- 172.21.15.90:0/916118581 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7fdd84018ca0 con 0x7fdd94102dc0 2024-09-10T22:01:33.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.768+0000 7fdd91600640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd780b53b0 0x7fdd780b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:33.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.769+0000 7fdd91600640 1 -- 172.21.15.90:0/916118581 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9017+0+0 (secure 0 0 0) 0x7fdd840e6a60 con 0x7fdd94102dc0 2024-09-10T22:01:33.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.769+0000 7fdd8b400640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd780b53b0 0x7fdd780b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:33.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.769+0000 7fdd91600640 1 -- 172.21.15.90:0/916118581 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdd840ad5b0 con 0x7fdd94102dc0 2024-09-10T22:01:33.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:33.769+0000 7fdd8b400640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd780b53b0 0x7fdd780b7870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c006010 tx=0x7fdd7c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:33.887 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b[65608]: 2024-09-10T22:01:33.703+0000 7f3f85c00640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-10T22:01:33.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] up:active 2024-09-10T22:01:33.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T22:01:33.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:01:33.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:33.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:33.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]} : dispatch 2024-09-10T22:01:33.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]} : dispatch 2024-09-10T22:01:33.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-10T22:01:33.888 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]} : dispatch 2024-09-10T22:01:33.889 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-10T22:01:33.889 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]} : dispatch 2024-09-10T22:01:33.889 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-10T22:01:33.889 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]} : dispatch 2024-09-10T22:01:33.889 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:33 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]} : dispatch 2024-09-10T22:01:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] up:active 2024-09-10T22:01:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T22:01:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:01:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]} : dispatch 2024-09-10T22:01:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]} : dispatch 2024-09-10T22:01:33.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-10T22:01:33.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]} : dispatch 2024-09-10T22:01:33.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-10T22:01:33.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]} : dispatch 2024-09-10T22:01:33.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-10T22:01:33.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]} : dispatch 2024-09-10T22:01:33.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:33 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]} : dispatch 2024-09-10T22:01:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.013+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fdd9402e5c0 con 0x7fdd94102dc0 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] up:active 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]} : dispatch 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]} : dispatch 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]} : dispatch 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]} : dispatch 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]} : dispatch 2024-09-10T22:01:34.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]} : dispatch 2024-09-10T22:01:34.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]} : dispatch 2024-09-10T22:01:34.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:33 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]} : dispatch 2024-09-10T22:01:34.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.740+0000 7fdd91600640 1 -- 172.21.15.90:0/916118581 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7fdd840aceb0 con 0x7fdd94102dc0 2024-09-10T22:01:34.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.742+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd780b53b0 msgr2=0x7fdd780b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:34.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.742+0000 7fdd9b000640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd780b53b0 0x7fdd780b7870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c006010 tx=0x7fdd7c005a20 comp rx=0 tx=0).stop 2024-09-10T22:01:34.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.742+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd94102dc0 msgr2=0x7fdd9419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:34.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.742+0000 7fdd9b000640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd94102dc0 0x7fdd9419b380 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400ebc0 tx=0x7fdd8400c6a0 comp rx=0 tx=0).stop 2024-09-10T22:01:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.743+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 shutdown_connections 2024-09-10T22:01:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.743+0000 7fdd9b000640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd780b53b0 0x7fdd780b7870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.743+0000 7fdd9b000640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd94109150 0x7fdd941a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.743+0000 7fdd9b000640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd941087d0 0x7fdd9419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.743+0000 7fdd9b000640 1 --2- 172.21.15.90:0/916118581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd94102dc0 0x7fdd9419b380 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.743+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 >> 172.21.15.90:0/916118581 conn(0x7fdd940fe830 msgr2=0x7fdd94101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:34.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.743+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 shutdown_connections 2024-09-10T22:01:34.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:34.743+0000 7fdd9b000640 1 -- 172.21.15.90:0/916118581 wait complete. 2024-09-10T22:01:34.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/916118581' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]': finished 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-10T22:01:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2024-09-10T22:01:34.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:34 smithi171 ceph-mon[34138]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T22:01:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T22:01:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/916118581' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:01:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:01:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-09-10T22:01:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]': finished 2024-09-10T22:01:35.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T22:01:35.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-10T22:01:35.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-10T22:01:35.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-09-10T22:01:35.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-10T22:01:35.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-10T22:01:35.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2024-09-10T22:01:35.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:34 smithi090 ceph-mon[30350]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T22:01:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: pgmap v344: 129 pgs: 129 active+clean; 584 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T22:01:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/916118581' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:01:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T22:01:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-09-10T22:01:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1, 8, 11]}]': finished 2024-09-10T22:01:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T22:01:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-10T22:01:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-09-10T22:01:35.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-09-10T22:01:35.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-10T22:01:35.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-09-10T22:01:35.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2024-09-10T22:01:35.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:34 smithi154 ceph-mon[34495]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T22:01:35.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:35 smithi171 ceph-mon[34138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:01:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:35 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:01:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:35 smithi171 ceph-mon[34138]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T22:01:35.998 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:35 smithi090 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:01:35.998 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:35 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:01:35.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:35.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:35.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:35.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:35.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:35 smithi090 ceph-mon[30350]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T22:01:36.000 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-10T22:01:36.000 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T22:01:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:35 smithi154 ceph-mon[34495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T22:01:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:35 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:01:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:36.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:36.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:35 smithi154 ceph-mon[34495]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T22:01:36.417 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:36 smithi090 ceph-mon[30350]: pgmap v347: 129 pgs: 4 peering, 125 active+clean; 586 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.7 KiB/s wr, 23 op/s 2024-09-10T22:01:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:36 smithi171 ceph-mon[34138]: pgmap v347: 129 pgs: 4 peering, 125 active+clean; 586 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.7 KiB/s wr, 23 op/s 2024-09-10T22:01:37.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.988+0000 7f2d01200640 1 -- 172.21.15.90:0/3987615394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc1087d0 msgr2=0x7f2cfc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:37.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.988+0000 7f2d01200640 1 --2- 172.21.15.90:0/3987615394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc1087d0 0x7f2cfc108c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009900 tx=0x7f2ce802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.989+0000 7f2d01200640 1 -- 172.21.15.90:0/3987615394 shutdown_connections 2024-09-10T22:01:37.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.989+0000 7f2d01200640 1 --2- 172.21.15.90:0/3987615394 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cfc109150 0x7f2cfc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.989+0000 7f2d01200640 1 --2- 172.21.15.90:0/3987615394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc1087d0 0x7f2cfc108c10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.989+0000 7f2d01200640 1 --2- 172.21.15.90:0/3987615394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc102dc0 0x7f2cfc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.989+0000 7f2d01200640 1 -- 172.21.15.90:0/3987615394 >> 172.21.15.90:0/3987615394 conn(0x7f2cfc0fe830 msgr2=0x7f2cfc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:37.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.989+0000 7f2d01200640 1 -- 172.21.15.90:0/3987615394 shutdown_connections 2024-09-10T22:01:37.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.989+0000 7f2d01200640 1 -- 172.21.15.90:0/3987615394 wait complete. 2024-09-10T22:01:37.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.990+0000 7f2d01200640 1 Processor -- start 2024-09-10T22:01:37.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.990+0000 7f2d01200640 1 -- start start 2024-09-10T22:01:37.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.990+0000 7f2d01200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc102dc0 0x7f2cfc199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:37.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.990+0000 7f2d01200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc1087d0 0x7f2cfc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:37.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.990+0000 7f2d01200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cfc109150 0x7f2cfc19e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:37.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.990+0000 7f2d01200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc112cd0 con 0x7f2cfc102dc0 2024-09-10T22:01:37.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.990+0000 7f2d01200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc112b50 con 0x7f2cfc1087d0 2024-09-10T22:01:37.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.990+0000 7f2d01200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2cfc112e50 con 0x7f2cfc109150 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc102dc0 0x7f2cfc199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc102dc0 0x7f2cfc199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54810/0 (socket says 172.21.15.90:54810) 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9e00640 1 -- 172.21.15.90:0/2805533629 learned_addr learned my addr 172.21.15.90:0/2805533629 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9400640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc1087d0 0x7f2cfc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cfa800640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cfc109150 0x7f2cfc19e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9e00640 1 -- 172.21.15.90:0/2805533629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cfc109150 msgr2=0x7f2cfc19e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9e00640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cfc109150 0x7f2cfc19e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9e00640 1 -- 172.21.15.90:0/2805533629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc1087d0 msgr2=0x7f2cfc199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9e00640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc1087d0 0x7f2cfc199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9e00640 1 -- 172.21.15.90:0/2805533629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cfc19f1b0 con 0x7f2cfc102dc0 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9400640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc1087d0 0x7f2cfc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cfa800640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cfc109150 0x7f2cfc19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:37.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.991+0000 7f2cf9e00640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc102dc0 0x7f2cfc199100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c920 tx=0x7f2cec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.992+0000 7f2cf2a00640 1 -- 172.21.15.90:0/2805533629 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec007c70 con 0x7f2cfc102dc0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.992+0000 7f2cf2a00640 1 -- 172.21.15.90:0/2805533629 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2cec007e10 con 0x7f2cfc102dc0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.992+0000 7f2cf2a00640 1 -- 172.21.15.90:0/2805533629 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cec018710 con 0x7f2cfc102dc0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.992+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cfc19f4a0 con 0x7f2cfc102dc0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.992+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cfc1a5b60 con 0x7f2cfc102dc0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.993+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cfc1031a0 con 0x7f2cfc102dc0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.995+0000 7f2cf2a00640 1 -- 172.21.15.90:0/2805533629 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 20) ==== 153267+0+0 (secure 0 0 0) 0x7f2cec00f450 con 0x7f2cfc102dc0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.996+0000 7f2cf2a00640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2ce00b5330 0x7f2ce00b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.997+0000 7f2cf2a00640 1 -- 172.21.15.90:0/2805533629 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f2cec0e6360 con 0x7f2cfc102dc0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.997+0000 7f2cf9400640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2ce00b5330 0x7f2ce00b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:36.997+0000 7f2cf9400640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2ce00b5330 0x7f2ce00b77f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8006010 tx=0x7f2ce8005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:37.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.000+0000 7f2cf2a00640 1 -- 172.21.15.90:0/2805533629 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2cec014030 con 0x7f2cfc102dc0 2024-09-10T22:01:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:36 smithi154 ceph-mon[34495]: pgmap v347: 129 pgs: 4 peering, 125 active+clean; 586 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5.7 KiB/s wr, 23 op/s 2024-09-10T22:01:37.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.223+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2cfc063590 con 0x7f2ce00b5330 2024-09-10T22:01:37.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.265+0000 7f2cf2a00640 1 -- 172.21.15.90:0/2805533629 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2cfc063590 con 0x7f2ce00b5330 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2ce00b5330 msgr2=0x7f2ce00b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2ce00b5330 0x7f2ce00b77f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8006010 tx=0x7f2ce8005a20 comp rx=0 tx=0).stop 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc102dc0 msgr2=0x7f2cfc199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc102dc0 0x7f2cfc199100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2cec00c920 tx=0x7f2cec00cde0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 shutdown_connections 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2ce00b5330 0x7f2ce00b77f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2cfc109150 0x7f2cfc19e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2cfc1087d0 0x7f2cfc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 --2- 172.21.15.90:0/2805533629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cfc102dc0 0x7f2cfc199100 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 >> 172.21.15.90:0/2805533629 conn(0x7f2cfc0fe830 msgr2=0x7f2cfc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:37.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 shutdown_connections 2024-09-10T22:01:37.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:37.267+0000 7f2d01200640 1 -- 172.21.15.90:0/2805533629 wait complete. 2024-09-10T22:01:37.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:01:37 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:01:37.230+0000 7fb7acc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:37.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:01:37 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:01:37.230+0000 7fb7acc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:37.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:01:37 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:01:37.230+0000 7fb7acc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:37.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:01:37 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:01:37.230+0000 7fb7acc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:37.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:01:37 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:01:37.230+0000 7fb7acc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:01:38.439 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T22:01:38.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:38.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:38.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:38 smithi090 ceph-mon[30350]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:38.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:38 smithi090 ceph-mon[30350]: pgmap v348: 129 pgs: 4 peering, 125 active+clean; 587 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.0 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:01:38.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:01:38.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:01:38.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:38.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:38.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:38.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:38 smithi171 ceph-mon[34138]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:38.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:38 smithi171 ceph-mon[34138]: pgmap v348: 129 pgs: 4 peering, 125 active+clean; 587 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.0 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:01:38.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:01:38.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:01:38.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:38.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:38.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:38.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:38 smithi154 ceph-mon[34495]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:38.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:38 smithi154 ceph-mon[34495]: pgmap v348: 129 pgs: 4 peering, 125 active+clean; 587 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6.0 KiB/s wr, 24 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T22:01:38.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:01:38.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:01:38.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:01:38.814 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:39.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:39 smithi171 ceph-mon[34138]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-09-10T22:01:39.499 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:39 smithi090 ceph-mon[30350]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-09-10T22:01:39.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.505+0000 7f99c2c00640 1 -- 172.21.15.90:0/797604144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 msgr2=0x7f99bc108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:39.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.505+0000 7f99c2c00640 1 --2- 172.21.15.90:0/797604144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 0x7f99bc108a50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009900 tx=0x7f99a802a410 comp rx=0 tx=0).stop 2024-09-10T22:01:39.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.505+0000 7f99c2c00640 1 -- 172.21.15.90:0/797604144 shutdown_connections 2024-09-10T22:01:39.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.505+0000 7f99c2c00640 1 --2- 172.21.15.90:0/797604144 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99bc108f90 0x7f99bc10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:39.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.505+0000 7f99c2c00640 1 --2- 172.21.15.90:0/797604144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 0x7f99bc108a50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:39.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.505+0000 7f99c2c00640 1 --2- 172.21.15.90:0/797604144 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99bc102c00 0x7f99bc102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:39.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.505+0000 7f99c2c00640 1 -- 172.21.15.90:0/797604144 >> 172.21.15.90:0/797604144 conn(0x7f99bc0fe6d0 msgr2=0x7f99bc100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:39.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.505+0000 7f99c2c00640 1 -- 172.21.15.90:0/797604144 shutdown_connections 2024-09-10T22:01:39.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.505+0000 7f99c2c00640 1 -- 172.21.15.90:0/797604144 wait complete. 2024-09-10T22:01:39.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.506+0000 7f99c2c00640 1 Processor -- start 2024-09-10T22:01:39.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.506+0000 7f99c2c00640 1 -- start start 2024-09-10T22:01:39.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c2c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99bc102c00 0x7f99bc19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:39.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c2c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 0x7f99bc19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:39.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c2c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99bc108f90 0x7f99bc1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:39.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c2c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc112b50 con 0x7f99bc108610 2024-09-10T22:01:39.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c2c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc1129d0 con 0x7f99bc102c00 2024-09-10T22:01:39.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c2c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc112cd0 con 0x7f99bc108f90 2024-09-10T22:01:39.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 0x7f99bc19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:39.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 0x7f99bc19b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58852/0 (socket says 172.21.15.90:58852) 2024-09-10T22:01:39.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c0e00640 1 -- 172.21.15.90:0/323785963 learned_addr learned my addr 172.21.15.90:0/323785963 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:39.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c1800640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99bc102c00 0x7f99bc19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:39.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c2200640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99bc108f90 0x7f99bc1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:39.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c0e00640 1 -- 172.21.15.90:0/323785963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99bc108f90 msgr2=0x7f99bc1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:39.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c0e00640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99bc108f90 0x7f99bc1a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:39.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c0e00640 1 -- 172.21.15.90:0/323785963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99bc102c00 msgr2=0x7f99bc19b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c0e00640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99bc102c00 0x7f99bc19b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.507+0000 7f99c0e00640 1 -- 172.21.15.90:0/323785963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc1a1210 con 0x7f99bc108610 2024-09-10T22:01:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.508+0000 7f99c0e00640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 0x7f99bc19b6a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99a802a930 tx=0x7f99a8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.508+0000 7f99ba000640 1 -- 172.21.15.90:0/323785963 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a803f070 con 0x7f99bc108610 2024-09-10T22:01:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.508+0000 7f99ba000640 1 -- 172.21.15.90:0/323785963 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99a8003e40 con 0x7f99bc108610 2024-09-10T22:01:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.508+0000 7f99ba000640 1 -- 172.21.15.90:0/323785963 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8035030 con 0x7f99bc108610 2024-09-10T22:01:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.508+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99bc1a14a0 con 0x7f99bc108610 2024-09-10T22:01:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.508+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99bc1a7bc0 con 0x7f99bc108610 2024-09-10T22:01:39.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.509+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99bc102fe0 con 0x7f99bc108610 2024-09-10T22:01:39.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.512+0000 7f99ba000640 1 -- 172.21.15.90:0/323785963 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f99a8031400 con 0x7f99bc108610 2024-09-10T22:01:39.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.513+0000 7f99ba000640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99a40b5460 0x7f99a40b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:39.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.513+0000 7f99ba000640 1 -- 172.21.15.90:0/323785963 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f99a8107190 con 0x7f99bc108610 2024-09-10T22:01:39.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.513+0000 7f99c1800640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99a40b5460 0x7f99a40b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:39.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.514+0000 7f99c1800640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99a40b5460 0x7f99a40b7920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005fd0 tx=0x7f99b0009870 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:39.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.515+0000 7f99ba000640 1 -- 172.21.15.90:0/323785963 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f99a80cda70 con 0x7f99bc108610 2024-09-10T22:01:39.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:39 smithi154 ceph-mon[34495]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-09-10T22:01:39.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.728+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f99bc063590 con 0x7f99a40b5460 2024-09-10T22:01:39.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.757+0000 7f99ba000640 1 -- 172.21.15.90:0/323785963 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f99bc063590 con 0x7f99a40b5460 2024-09-10T22:01:39.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99a40b5460 msgr2=0x7f99a40b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:39.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99a40b5460 0x7f99a40b7920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f99b0005fd0 tx=0x7f99b0009870 comp rx=0 tx=0).stop 2024-09-10T22:01:39.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 msgr2=0x7f99bc19b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:39.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 0x7f99bc19b6a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f99a802a930 tx=0x7f99a8003c20 comp rx=0 tx=0).stop 2024-09-10T22:01:39.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 shutdown_connections 2024-09-10T22:01:39.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99a40b5460 0x7f99a40b7920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:39.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99bc108f90 0x7f99bc1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:39.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99bc108610 0x7f99bc19b6a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:39.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 --2- 172.21.15.90:0/323785963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f99bc102c00 0x7f99bc19b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:39.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 >> 172.21.15.90:0/323785963 conn(0x7f99bc0fe6d0 msgr2=0x7f99bc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:39.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 shutdown_connections 2024-09-10T22:01:39.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:39.760+0000 7f99c2c00640 1 -- 172.21.15.90:0/323785963 wait complete. 2024-09-10T22:01:40.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:40 smithi171 ceph-mon[34138]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 588 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 25 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T22:01:40.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:40 smithi171 ceph-mon[34138]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:40.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:40 smithi090 ceph-mon[30350]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 588 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 25 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T22:01:40.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:40 smithi090 ceph-mon[30350]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:40.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:40 smithi154 ceph-mon[34495]: pgmap v349: 129 pgs: 2 peering, 127 active+clean; 588 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.2 KiB/s wr, 25 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T22:01:40.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:40 smithi154 ceph-mon[34495]: from='client.15039 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:40.997 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T22:01:41.365 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.121+0000 7f1dd8200640 1 -- 172.21.15.90:0/3568595457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 msgr2=0x7f1dd0101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.121+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3568595457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 0x7f1dd0101600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009900 tx=0x7f1dc402a410 comp rx=0 tx=0).stop 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.121+0000 7f1dd8200640 1 -- 172.21.15.90:0/3568595457 shutdown_connections 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.121+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3568595457 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dd0105ef0 0x7f1dd010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.121+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3568595457 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd0105570 0x7f1dd01059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.121+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3568595457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 0x7f1dd0101600 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.121+0000 7f1dd8200640 1 -- 172.21.15.90:0/3568595457 >> 172.21.15.90:0/3568595457 conn(0x7f1dd0078100 msgr2=0x7f1dd00ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.121+0000 7f1dd8200640 1 -- 172.21.15.90:0/3568595457 shutdown_connections 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.121+0000 7f1dd8200640 1 -- 172.21.15.90:0/3568595457 wait complete. 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.122+0000 7f1dd8200640 1 Processor -- start 2024-09-10T22:01:42.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.122+0000 7f1dd8200640 1 -- start start 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.122+0000 7f1dd8200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 0x7f1dd019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.122+0000 7f1dd8200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dd0105570 0x7f1dd019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.122+0000 7f1dd8200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd0105ef0 0x7f1dd01a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.122+0000 7f1dd8200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0112ca0 con 0x7f1dd0101220 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.122+0000 7f1dd8200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0112b20 con 0x7f1dd0105ef0 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.122+0000 7f1dd8200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1dd0112e20 con 0x7f1dd0105570 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd5800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 0x7f1dd019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd5800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 0x7f1dd019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58876/0 (socket says 172.21.15.90:58876) 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd5800640 1 -- 172.21.15.90:0/3736267529 learned_addr learned my addr 172.21.15.90:0/3736267529 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd4e00640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dd0105570 0x7f1dd019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd6200640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd0105ef0 0x7f1dd01a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd5800640 1 -- 172.21.15.90:0/3736267529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dd0105570 msgr2=0x7f1dd019b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd5800640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dd0105570 0x7f1dd019b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd5800640 1 -- 172.21.15.90:0/3736267529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd0105ef0 msgr2=0x7f1dd01a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:42.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd5800640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd0105ef0 0x7f1dd01a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd5800640 1 -- 172.21.15.90:0/3736267529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dd01a14a0 con 0x7f1dd0101220 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd6200640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd0105ef0 0x7f1dd01a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.123+0000 7f1dd4e00640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dd0105570 0x7f1dd019b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.124+0000 7f1dd5800640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 0x7f1dd019b3f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009230 tx=0x7f1dc40319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.124+0000 7f1dce000640 1 -- 172.21.15.90:0/3736267529 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc403f030 con 0x7f1dd0101220 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.124+0000 7f1dce000640 1 -- 172.21.15.90:0/3736267529 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1dc40043d0 con 0x7f1dd0101220 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.124+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dd01a1730 con 0x7f1dd0101220 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.124+0000 7f1dce000640 1 -- 172.21.15.90:0/3736267529 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4035070 con 0x7f1dd0101220 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.124+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dd01a7e50 con 0x7f1dd0101220 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.125+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dd0101600 con 0x7f1dd0101220 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.127+0000 7f1dce000640 1 -- 172.21.15.90:0/3736267529 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f1dc403b8a0 con 0x7f1dd0101220 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.129+0000 7f1dce000640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1db80b5440 0x7f1db80b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:42.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.129+0000 7f1dce000640 1 -- 172.21.15.90:0/3736267529 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f1dc4106960 con 0x7f1dd0101220 2024-09-10T22:01:42.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.129+0000 7f1dd4e00640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1db80b5440 0x7f1db80b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:42.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.130+0000 7f1dd4e00640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1db80b5440 0x7f1db80b7900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0108500 tx=0x7f1dbc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:42.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.133+0000 7f1dce000640 1 -- 172.21.15.90:0/3736267529 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1dc40cd240 con 0x7f1dd0101220 2024-09-10T22:01:42.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.344+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f1dd010ec20 con 0x7f1db80b5440 2024-09-10T22:01:42.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.350+0000 7f1dce000640 1 -- 172.21.15.90:0/3736267529 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1dd010ec20 con 0x7f1db80b5440 2024-09-10T22:01:42.352 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_0/15d452fc-d875-48e7-8057-49ebca12b7b4 2024-09-10T22:01:42.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1db80b5440 msgr2=0x7f1db80b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1db80b5440 0x7f1db80b7900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0108500 tx=0x7f1dbc005f90 comp rx=0 tx=0).stop 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 msgr2=0x7f1dd019b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 0x7f1dd019b3f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1dc4009230 tx=0x7f1dc40319a0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 shutdown_connections 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1db80b5440 0x7f1db80b7900 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1dd0105ef0 0x7f1dd01a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1dd0105570 0x7f1dd019b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 --2- 172.21.15.90:0/3736267529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1dd0101220 0x7f1dd019b3f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 >> 172.21.15.90:0/3736267529 conn(0x7f1dd0078100 msgr2=0x7f1dd0108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 shutdown_connections 2024-09-10T22:01:42.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:42.353+0000 7f1dd8200640 1 -- 172.21.15.90:0/3736267529 wait complete. 2024-09-10T22:01:42.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:42 smithi154 ceph-mon[34495]: pgmap v350: 129 pgs: 129 active+clean; 608 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 KiB/s wr, 22 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T22:01:42.945 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:42 smithi090 ceph-mon[30350]: pgmap v350: 129 pgs: 129 active+clean; 608 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 KiB/s wr, 22 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T22:01:42.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:42 smithi171 ceph-mon[34138]: pgmap v350: 129 pgs: 129 active+clean; 608 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.5 KiB/s wr, 22 op/s; 13 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T22:01:43.631 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T22:01:43.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:43 smithi090 ceph-mon[30350]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:43.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:43 smithi154 ceph-mon[34495]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:43.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:43 smithi171 ceph-mon[34138]: from='client.15045 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:44.004 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:44.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:44 smithi090 ceph-mon[30350]: pgmap v351: 129 pgs: 129 active+clean; 608 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.5 KiB/s wr, 18 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-09-10T22:01:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:44 smithi154 ceph-mon[34495]: pgmap v351: 129 pgs: 129 active+clean; 608 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.5 KiB/s wr, 18 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-09-10T22:01:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.830+0000 7f76c8a00640 1 -- 172.21.15.90:0/942508399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410a800 msgr2=0x7f76c410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.830+0000 7f76c8a00640 1 --2- 172.21.15.90:0/942508399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410a800 0x7f76c410ac40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f76b0009900 tx=0x7f76b002a410 comp rx=0 tx=0).stop 2024-09-10T22:01:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.830+0000 7f76c8a00640 1 -- 172.21.15.90:0/942508399 shutdown_connections 2024-09-10T22:01:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.830+0000 7f76c8a00640 1 --2- 172.21.15.90:0/942508399 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76c410b180 0x7f76c4111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.830+0000 7f76c8a00640 1 --2- 172.21.15.90:0/942508399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410a800 0x7f76c410ac40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.830+0000 7f76c8a00640 1 --2- 172.21.15.90:0/942508399 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c4104df0 0x7f76c41051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.830+0000 7f76c8a00640 1 -- 172.21.15.90:0/942508399 >> 172.21.15.90:0/942508399 conn(0x7f76c4100840 msgr2=0x7f76c4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.830+0000 7f76c8a00640 1 -- 172.21.15.90:0/942508399 shutdown_connections 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.830+0000 7f76c8a00640 1 -- 172.21.15.90:0/942508399 wait complete. 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.831+0000 7f76c8a00640 1 Processor -- start 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.832+0000 7f76c8a00640 1 -- start start 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.832+0000 7f76c8a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76c4104df0 0x7f76c419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.832+0000 7f76c8a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c410a800 0x7f76c419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.832+0000 7f76c8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410b180 0x7f76c41a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.832+0000 7f76c8a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4114d40 con 0x7f76c410b180 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.832+0000 7f76c8a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4114bc0 con 0x7f76c4104df0 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.832+0000 7f76c8a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4114ec0 con 0x7f76c410a800 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410b180 0x7f76c41a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410b180 0x7f76c41a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58894/0 (socket says 172.21.15.90:58894) 2024-09-10T22:01:44.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3e00640 1 -- 172.21.15.90:0/1974494018 learned_addr learned my addr 172.21.15.90:0/1974494018 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3400640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76c4104df0 0x7f76c419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c2a00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c410a800 0x7f76c419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3e00640 1 -- 172.21.15.90:0/1974494018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c410a800 msgr2=0x7f76c419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3e00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c410a800 0x7f76c419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3e00640 1 -- 172.21.15.90:0/1974494018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76c4104df0 msgr2=0x7f76c419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3e00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76c4104df0 0x7f76c419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3e00640 1 -- 172.21.15.90:0/1974494018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c41a12c0 con 0x7f76c410b180 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.833+0000 7f76c3e00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410b180 0x7f76c41a0af0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00eaa0 tx=0x7f76bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.834+0000 7f76bbe00640 1 -- 172.21.15.90:0/1974494018 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc00cdb0 con 0x7f76c410b180 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.834+0000 7f76bbe00640 1 -- 172.21.15.90:0/1974494018 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f76bc005020 con 0x7f76c410b180 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.834+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c41a15b0 con 0x7f76c410b180 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.834+0000 7f76bbe00640 1 -- 172.21.15.90:0/1974494018 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0116d0 con 0x7f76c410b180 2024-09-10T22:01:44.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.834+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c41a7c70 con 0x7f76c410b180 2024-09-10T22:01:44.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.835+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c41051d0 con 0x7f76c410b180 2024-09-10T22:01:44.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.837+0000 7f76bbe00640 1 -- 172.21.15.90:0/1974494018 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f76bc00f040 con 0x7f76c410b180 2024-09-10T22:01:44.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.838+0000 7f76bbe00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76a80b5460 0x7f76a80b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:44.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.839+0000 7f76bbe00640 1 -- 172.21.15.90:0/1974494018 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f76bc0e6570 con 0x7f76c410b180 2024-09-10T22:01:44.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.839+0000 7f76c3400640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76a80b5460 0x7f76a80b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:44.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.839+0000 7f76c3400640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76a80b5460 0x7f76a80b7920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009890 tx=0x7f76b4008040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:44.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:44.840+0000 7f76bbe00640 1 -- 172.21.15.90:0/1974494018 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f76bc016030 con 0x7f76c410b180 2024-09-10T22:01:44.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:44 smithi171 ceph-mon[34138]: pgmap v351: 129 pgs: 129 active+clean; 608 KiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 4.5 KiB/s wr, 18 op/s; 11 B/s, 1 keys/s, 1 objects/s recovering 2024-09-10T22:01:45.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.050+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f76c4063590 con 0x7f76a80b5460 2024-09-10T22:01:45.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.079+0000 7f76bbe00640 1 -- 172.21.15.90:0/1974494018 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f76c4063590 con 0x7f76a80b5460 2024-09-10T22:01:45.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76a80b5460 msgr2=0x7f76a80b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:45.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76a80b5460 0x7f76a80b7920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009890 tx=0x7f76b4008040 comp rx=0 tx=0).stop 2024-09-10T22:01:45.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410b180 msgr2=0x7f76c41a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:45.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410b180 0x7f76c41a0af0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f76bc00eaa0 tx=0x7f76bc00ef60 comp rx=0 tx=0).stop 2024-09-10T22:01:45.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 shutdown_connections 2024-09-10T22:01:45.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76a80b5460 0x7f76a80b7920 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:45.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c410b180 0x7f76c41a0af0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:45.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76c410a800 0x7f76c419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:45.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 --2- 172.21.15.90:0/1974494018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76c4104df0 0x7f76c419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:45.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.081+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 >> 172.21.15.90:0/1974494018 conn(0x7f76c4100840 msgr2=0x7f76c4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:45.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.082+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 shutdown_connections 2024-09-10T22:01:45.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:45.082+0000 7f76c8a00640 1 -- 172.21.15.90:0/1974494018 wait complete. 2024-09-10T22:01:46.085 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T22:01:46.450 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:46 smithi090 ceph-mon[30350]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:46.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:46 smithi090 ceph-mon[30350]: pgmap v352: 129 pgs: 129 active+clean; 614 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.8 KiB/s wr, 13 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T22:01:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:46 smithi154 ceph-mon[34495]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:46 smithi154 ceph-mon[34495]: pgmap v352: 129 pgs: 129 active+clean; 614 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.8 KiB/s wr, 13 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T22:01:46.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:46 smithi171 ceph-mon[34138]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:46.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:46 smithi171 ceph-mon[34138]: pgmap v352: 129 pgs: 129 active+clean; 614 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 3.8 KiB/s wr, 13 op/s; 10 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T22:01:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 -- 172.21.15.90:0/1397410005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdac104fc0 msgr2=0x7efdac1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 --2- 172.21.15.90:0/1397410005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdac104fc0 0x7efdac1053a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009900 tx=0x7efd9c02a410 comp rx=0 tx=0).stop 2024-09-10T22:01:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 -- 172.21.15.90:0/1397410005 shutdown_connections 2024-09-10T22:01:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 --2- 172.21.15.90:0/1397410005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdac10b350 0x7efdac111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 --2- 172.21.15.90:0/1397410005 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdac10a9d0 0x7efdac10ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 --2- 172.21.15.90:0/1397410005 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdac104fc0 0x7efdac1053a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 -- 172.21.15.90:0/1397410005 >> 172.21.15.90:0/1397410005 conn(0x7efdac1009f0 msgr2=0x7efdac102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 -- 172.21.15.90:0/1397410005 shutdown_connections 2024-09-10T22:01:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 -- 172.21.15.90:0/1397410005 wait complete. 2024-09-10T22:01:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.154+0000 7efdb2600640 1 Processor -- start 2024-09-10T22:01:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdb2600640 1 -- start start 2024-09-10T22:01:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdb2600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdac104fc0 0x7efdac19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdb2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdac10a9d0 0x7efdac19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdb2600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdac10b350 0x7efdac1a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdb2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efdac114e10 con 0x7efdac10a9d0 2024-09-10T22:01:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdb2600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efdac114c90 con 0x7efdac10b350 2024-09-10T22:01:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdb2600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efdac114f90 con 0x7efdac104fc0 2024-09-10T22:01:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdaaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdac10a9d0 0x7efdac19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdaaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdac10a9d0 0x7efdac19b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58910/0 (socket says 172.21.15.90:58910) 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdaaa00640 1 -- 172.21.15.90:0/4089811785 learned_addr learned my addr 172.21.15.90:0/4089811785 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.155+0000 7efdab400640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdac104fc0 0x7efdac19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efdabe00640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdac10b350 0x7efdac1a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efdaaa00640 1 -- 172.21.15.90:0/4089811785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdac104fc0 msgr2=0x7efdac19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efdaaa00640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdac104fc0 0x7efdac19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efdaaa00640 1 -- 172.21.15.90:0/4089811785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdac10b350 msgr2=0x7efdac1a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efdaaa00640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdac10b350 0x7efdac1a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efdaaa00640 1 -- 172.21.15.90:0/4089811785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdac1a1490 con 0x7efdac10a9d0 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efdab400640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdac104fc0 0x7efdac19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efdaaa00640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdac10a9d0 0x7efdac19b920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efd9800e9f0 tx=0x7efd9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efda3e00640 1 -- 172.21.15.90:0/4089811785 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9800cd80 con 0x7efdac10a9d0 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdac1a1780 con 0x7efdac10a9d0 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efda3e00640 1 -- 172.21.15.90:0/4089811785 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efd980108c0 con 0x7efdac10a9d0 2024-09-10T22:01:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.156+0000 7efda3e00640 1 -- 172.21.15.90:0/4089811785 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd980187c0 con 0x7efdac10a9d0 2024-09-10T22:01:47.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.157+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdac1a7e40 con 0x7efdac10a9d0 2024-09-10T22:01:47.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.158+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdac1053a0 con 0x7efdac10a9d0 2024-09-10T22:01:47.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.160+0000 7efda3e00640 1 -- 172.21.15.90:0/4089811785 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7efd98010430 con 0x7efdac10a9d0 2024-09-10T22:01:47.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.161+0000 7efda3e00640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efd900b53c0 0x7efd900b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:47.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.161+0000 7efda3e00640 1 -- 172.21.15.90:0/4089811785 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7efd980e6720 con 0x7efdac10a9d0 2024-09-10T22:01:47.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.162+0000 7efdab400640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efd900b53c0 0x7efd900b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:47.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.162+0000 7efdab400640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efd900b53c0 0x7efd900b7880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:47.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.166+0000 7efda3e00640 1 -- 172.21.15.90:0/4089811785 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efd980ad000 con 0x7efdac10a9d0 2024-09-10T22:01:47.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.377+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7efdac063590 con 0x7efd900b53c0 2024-09-10T22:01:47.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.382+0000 7efda3e00640 1 -- 172.21.15.90:0/4089811785 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7efdac063590 con 0x7efd900b53c0 2024-09-10T22:01:47.383 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_1/90af6c37-d00c-44c9-9c9f-86fb96385bd1 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efd900b53c0 msgr2=0x7efd900b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efd900b53c0 0x7efd900b7880 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000c00 tx=0x7efd9c005a20 comp rx=0 tx=0).stop 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdac10a9d0 msgr2=0x7efdac19b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdac10a9d0 0x7efdac19b920 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efd9800e9f0 tx=0x7efd9800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 shutdown_connections 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efd900b53c0 0x7efd900b7880 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdac10b350 0x7efdac1a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdac10a9d0 0x7efdac19b920 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 --2- 172.21.15.90:0/4089811785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdac104fc0 0x7efdac19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 >> 172.21.15.90:0/4089811785 conn(0x7efdac1009f0 msgr2=0x7efdac101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:47.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 shutdown_connections 2024-09-10T22:01:47.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:47.385+0000 7efdb2600640 1 -- 172.21.15.90:0/4089811785 wait complete. 2024-09-10T22:01:48.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:01:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:48 smithi171 ceph-mon[34138]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:48.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:48 smithi171 ceph-mon[34138]: pgmap v353: 129 pgs: 129 active+clean; 614 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T22:01:49.023 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:48 smithi090 ceph-mon[30350]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:48 smithi090 ceph-mon[30350]: pgmap v353: 129 pgs: 129 active+clean; 614 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T22:01:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:48 smithi154 ceph-mon[34495]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:01:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:48 smithi154 ceph-mon[34495]: pgmap v353: 129 pgs: 129 active+clean; 614 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.3 KiB/s wr, 12 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T22:01:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.791+0000 7f268b000640 1 -- 172.21.15.90:0/650198001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 msgr2=0x7f26841031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.791+0000 7f268b000640 1 --2- 172.21.15.90:0/650198001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 0x7f26841031a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2674009900 tx=0x7f267402a410 comp rx=0 tx=0).stop 2024-09-10T22:01:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.791+0000 7f268b000640 1 -- 172.21.15.90:0/650198001 shutdown_connections 2024-09-10T22:01:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.791+0000 7f268b000640 1 --2- 172.21.15.90:0/650198001 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2684109150 0x7f268410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.791+0000 7f268b000640 1 --2- 172.21.15.90:0/650198001 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f26841087d0 0x7f2684108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.791+0000 7f268b000640 1 --2- 172.21.15.90:0/650198001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 0x7f26841031a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.791+0000 7f268b000640 1 -- 172.21.15.90:0/650198001 >> 172.21.15.90:0/650198001 conn(0x7f26840fe830 msgr2=0x7f2684100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.791+0000 7f268b000640 1 -- 172.21.15.90:0/650198001 shutdown_connections 2024-09-10T22:01:49.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.791+0000 7f268b000640 1 -- 172.21.15.90:0/650198001 wait complete. 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.792+0000 7f268b000640 1 Processor -- start 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.792+0000 7f268b000640 1 -- start start 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.792+0000 7f268b000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 0x7f268419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.792+0000 7f268b000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26841087d0 0x7f268419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.792+0000 7f268b000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2684109150 0x7f26841a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.792+0000 7f268b000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2684112cc0 con 0x7f2684102dc0 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.792+0000 7f268b000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2684112b40 con 0x7f2684109150 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.792+0000 7f268b000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2684112e40 con 0x7f26841087d0 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.792+0000 7f2683e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 0x7f268419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2683e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 0x7f268419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48374/0 (socket says 172.21.15.90:48374) 2024-09-10T22:01:49.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2683e00640 1 -- 172.21.15.90:0/982896887 learned_addr learned my addr 172.21.15.90:0/982896887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:49.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2683400640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26841087d0 0x7f268419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:49.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2689000640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2684109150 0x7f26841a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:49.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2683e00640 1 -- 172.21.15.90:0/982896887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26841087d0 msgr2=0x7f268419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:49.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2683e00640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26841087d0 0x7f268419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:49.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2683e00640 1 -- 172.21.15.90:0/982896887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2684109150 msgr2=0x7f26841a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:49.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2683e00640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2684109150 0x7f26841a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:49.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2683e00640 1 -- 172.21.15.90:0/982896887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26841a1490 con 0x7f2684102dc0 2024-09-10T22:01:49.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.793+0000 7f2683e00640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 0x7f268419b3e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2674009230 tx=0x7f2674004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.794+0000 7f2680c00640 1 -- 172.21.15.90:0/982896887 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267403f030 con 0x7f2684102dc0 2024-09-10T22:01:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.794+0000 7f2680c00640 1 -- 172.21.15.90:0/982896887 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2674003a80 con 0x7f2684102dc0 2024-09-10T22:01:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.794+0000 7f2680c00640 1 -- 172.21.15.90:0/982896887 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674035070 con 0x7f2684102dc0 2024-09-10T22:01:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.794+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26841a1720 con 0x7f2684102dc0 2024-09-10T22:01:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.794+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26841a7e40 con 0x7f2684102dc0 2024-09-10T22:01:49.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.795+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26841031a0 con 0x7f2684102dc0 2024-09-10T22:01:49.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.797+0000 7f2680c00640 1 -- 172.21.15.90:0/982896887 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f267403bc30 con 0x7f2684102dc0 2024-09-10T22:01:49.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.799+0000 7f2680c00640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f26680b5440 0x7f26680b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:49.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.799+0000 7f2680c00640 1 -- 172.21.15.90:0/982896887 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f2674106fb0 con 0x7f2684102dc0 2024-09-10T22:01:49.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.799+0000 7f2683400640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f26680b5440 0x7f26680b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:49.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.799+0000 7f2683400640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f26680b5440 0x7f26680b7900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2684108570 tx=0x7f267000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:49.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:49.800+0000 7f2680c00640 1 -- 172.21.15.90:0/982896887 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f26740cd890 con 0x7f2684102dc0 2024-09-10T22:01:50.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.020+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f268402e5c0 con 0x7f2684102dc0 2024-09-10T22:01:50.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.021+0000 7f2680c00640 1 -- 172.21.15.90:0/982896887 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7f26740cd190 con 0x7f2684102dc0 2024-09-10T22:01:50.022 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:01:50.022 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"btime":"2024-09-10T22:01:34:736152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:01:34.015563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:01:50.022 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-09-10T22:01:50.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f26680b5440 msgr2=0x7f26680b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:50.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f26680b5440 0x7f26680b7900 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2684108570 tx=0x7f267000b410 comp rx=0 tx=0).stop 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 msgr2=0x7f268419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 0x7f268419b3e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2674009230 tx=0x7f2674004270 comp rx=0 tx=0).stop 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 shutdown_connections 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f26680b5440 0x7f26680b7900 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2684109150 0x7f26841a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f26841087d0 0x7f268419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 --2- 172.21.15.90:0/982896887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2684102dc0 0x7f268419b3e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 >> 172.21.15.90:0/982896887 conn(0x7f26840fe830 msgr2=0x7f2684104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 shutdown_connections 2024-09-10T22:01:50.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:50.024+0000 7f268b000640 1 -- 172.21.15.90:0/982896887 wait complete. 2024-09-10T22:01:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:50 smithi090 ceph-mon[30350]: pgmap v354: 129 pgs: 129 active+clean; 614 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.4 KiB/s wr, 13 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T22:01:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:50 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/982896887' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:01:50.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:50 smithi171 ceph-mon[34138]: pgmap v354: 129 pgs: 129 active+clean; 614 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.4 KiB/s wr, 13 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T22:01:50.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:50 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/982896887' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:01:51.009 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:01:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:50 smithi154 ceph-mon[34495]: pgmap v354: 129 pgs: 129 active+clean; 614 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 3.4 KiB/s wr, 13 op/s; 9 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T22:01:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:50 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/982896887' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:01:51.378 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:52.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.019+0000 7f7da3200640 1 -- 172.21.15.90:0/3893233855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4108fa0 msgr2=0x7f7da410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:52.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.019+0000 7f7da3200640 1 --2- 172.21.15.90:0/3893233855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4108fa0 0x7f7da410f7a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009900 tx=0x7f7d9c02a410 comp rx=0 tx=0).stop 2024-09-10T22:01:52.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.019+0000 7f7da3200640 1 -- 172.21.15.90:0/3893233855 shutdown_connections 2024-09-10T22:01:52.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.019+0000 7f7da3200640 1 --2- 172.21.15.90:0/3893233855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4108fa0 0x7f7da410f7a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:52.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.019+0000 7f7da3200640 1 --2- 172.21.15.90:0/3893233855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da4108620 0x7f7da4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:52.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.019+0000 7f7da3200640 1 --2- 172.21.15.90:0/3893233855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7da4102c10 0x7f7da4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:52.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.019+0000 7f7da3200640 1 -- 172.21.15.90:0/3893233855 >> 172.21.15.90:0/3893233855 conn(0x7f7da40fe6c0 msgr2=0x7f7da4100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:52.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.019+0000 7f7da3200640 1 -- 172.21.15.90:0/3893233855 shutdown_connections 2024-09-10T22:01:52.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.019+0000 7f7da3200640 1 -- 172.21.15.90:0/3893233855 wait complete. 2024-09-10T22:01:52.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.020+0000 7f7da3200640 1 Processor -- start 2024-09-10T22:01:52.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.020+0000 7f7da3200640 1 -- start start 2024-09-10T22:01:52.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.020+0000 7f7da3200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4102c10 0x7f7da40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:52.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.020+0000 7f7da3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7da4108620 0x7f7da40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:52.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.020+0000 7f7da3200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da4108fa0 0x7f7da4075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:52.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.020+0000 7f7da3200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4112b20 con 0x7f7da4102c10 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.020+0000 7f7da3200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7da41129a0 con 0x7f7da4108fa0 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.020+0000 7f7da3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4112ca0 con 0x7f7da4108620 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4102c10 0x7f7da40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4102c10 0x7f7da40754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48390/0 (socket says 172.21.15.90:48390) 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1e00640 1 -- 172.21.15.90:0/2787865675 learned_addr learned my addr 172.21.15.90:0/2787865675 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da2800640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da4108fa0 0x7f7da4075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1400640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7da4108620 0x7f7da40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1e00640 1 -- 172.21.15.90:0/2787865675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7da4108620 msgr2=0x7f7da40759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1e00640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7da4108620 0x7f7da40759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1e00640 1 -- 172.21.15.90:0/2787865675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da4108fa0 msgr2=0x7f7da4075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1e00640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da4108fa0 0x7f7da4075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1e00640 1 -- 172.21.15.90:0/2787865675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da41a7380 con 0x7f7da4102c10 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.021+0000 7f7da1e00640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4102c10 0x7f7da40754a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400ea80 tx=0x7f7d9400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:52.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.022+0000 7f7d9aa00640 1 -- 172.21.15.90:0/2787865675 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400ce40 con 0x7f7da4102c10 2024-09-10T22:01:52.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.022+0000 7f7d9aa00640 1 -- 172.21.15.90:0/2787865675 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d94005020 con 0x7f7da4102c10 2024-09-10T22:01:52.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.022+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da41a7670 con 0x7f7da4102c10 2024-09-10T22:01:52.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.022+0000 7f7d9aa00640 1 -- 172.21.15.90:0/2787865675 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d940116d0 con 0x7f7da4102c10 2024-09-10T22:01:52.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.022+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41a7b80 con 0x7f7da4102c10 2024-09-10T22:01:52.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.023+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da4102ff0 con 0x7f7da4102c10 2024-09-10T22:01:52.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.025+0000 7f7d9aa00640 1 -- 172.21.15.90:0/2787865675 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f7d9400f040 con 0x7f7da4102c10 2024-09-10T22:01:52.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.027+0000 7f7d9aa00640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d880b54e0 0x7f7d880b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:52.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.027+0000 7f7d9aa00640 1 -- 172.21.15.90:0/2787865675 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f7d940e6380 con 0x7f7da4102c10 2024-09-10T22:01:52.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.027+0000 7f7da1400640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d880b54e0 0x7f7d880b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:52.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.027+0000 7f7da1400640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d880b54e0 0x7f7d880b79a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7da4076f90 tx=0x7f7d9000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:52.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.030+0000 7f7d9aa00640 1 -- 172.21.15.90:0/2787865675 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7d94016030 con 0x7f7da4102c10 2024-09-10T22:01:52.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.216+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7da410a7f0 con 0x7f7da4102c10 2024-09-10T22:01:52.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.218+0000 7f7d9aa00640 1 -- 172.21.15.90:0/2787865675 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25040 (secure 0 0 0) 0x7f7d940ac610 con 0x7f7da4102c10 2024-09-10T22:01:52.219 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:01:52.219 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:01:35.182168+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:01:52.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d880b54e0 msgr2=0x7f7d880b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d880b54e0 0x7f7d880b79a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7da4076f90 tx=0x7f7d9000b410 comp rx=0 tx=0).stop 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4102c10 msgr2=0x7f7da40754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4102c10 0x7f7da40754a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400ea80 tx=0x7f7d9400ef40 comp rx=0 tx=0).stop 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 shutdown_connections 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7d880b54e0 0x7f7d880b79a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7da4108fa0 0x7f7da4075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7da4108620 0x7f7da40759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 --2- 172.21.15.90:0/2787865675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7da4102c10 0x7f7da40754a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 >> 172.21.15.90:0/2787865675 conn(0x7f7da40fe6c0 msgr2=0x7f7da40fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 shutdown_connections 2024-09-10T22:01:52.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:52.221+0000 7f7da3200640 1 -- 172.21.15.90:0/2787865675 wait complete. 2024-09-10T22:01:52.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:52 smithi090 ceph-mon[30350]: pgmap v355: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-10T22:01:52.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:52 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2787865675' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:52 smithi171 ceph-mon[34138]: pgmap v355: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-10T22:01:52.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:52 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2787865675' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:52 smithi154 ceph-mon[34495]: pgmap v355: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-10T22:01:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:52 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2787865675' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:01:53.193 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:01:53.193 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:01:53.561 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:54.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.163+0000 7fa173e00640 1 -- 172.21.15.90:0/1589451896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa174102040 msgr2=0x7fa17410ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:54.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.163+0000 7fa173e00640 1 --2- 172.21.15.90:0/1589451896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa174102040 0x7fa17410ed90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa160009900 tx=0x7fa16002a480 comp rx=0 tx=0).stop 2024-09-10T22:01:54.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.163+0000 7fa173e00640 1 -- 172.21.15.90:0/1589451896 shutdown_connections 2024-09-10T22:01:54.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.163+0000 7fa173e00640 1 --2- 172.21.15.90:0/1589451896 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa17410f2d0 0x7fa1741116c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:54.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.163+0000 7fa173e00640 1 --2- 172.21.15.90:0/1589451896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa174102040 0x7fa17410ed90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:54.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.163+0000 7fa173e00640 1 --2- 172.21.15.90:0/1589451896 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa174101720 0x7fa174101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:54.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.163+0000 7fa173e00640 1 -- 172.21.15.90:0/1589451896 >> 172.21.15.90:0/1589451896 conn(0x7fa1740fd5b0 msgr2=0x7fa1740ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:54.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.163+0000 7fa173e00640 1 -- 172.21.15.90:0/1589451896 shutdown_connections 2024-09-10T22:01:54.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.163+0000 7fa173e00640 1 -- 172.21.15.90:0/1589451896 wait complete. 2024-09-10T22:01:54.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.165+0000 7fa173e00640 1 Processor -- start 2024-09-10T22:01:54.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.165+0000 7fa173e00640 1 -- start start 2024-09-10T22:01:54.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.165+0000 7fa173e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa174101720 0x7fa17419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:54.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa173e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa174102040 0x7fa17419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:54.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa173e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa17410f2d0 0x7fa1741a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:54.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa173e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa174114ce0 con 0x7fa17410f2d0 2024-09-10T22:01:54.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa173e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa174114b60 con 0x7fa174102040 2024-09-10T22:01:54.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa173e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa174114e60 con 0x7fa174101720 2024-09-10T22:01:54.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa172a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa174101720 0x7fa17419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:54.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa172a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa174101720 0x7fa17419b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:40030/0 (socket says 172.21.15.90:40030) 2024-09-10T22:01:54.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa172a00640 1 -- 172.21.15.90:0/981302354 learned_addr learned my addr 172.21.15.90:0/981302354 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:54.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa172000640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa174102040 0x7fa17419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:54.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa173400640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa17410f2d0 0x7fa1741a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:54.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa172a00640 1 -- 172.21.15.90:0/981302354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa174102040 msgr2=0x7fa17419b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:54.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa172a00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa174102040 0x7fa17419b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:54.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa172a00640 1 -- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa17410f2d0 msgr2=0x7fa1741a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:54.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa172a00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa17410f2d0 0x7fa1741a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:54.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.166+0000 7fa172a00640 1 -- 172.21.15.90:0/981302354 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1741a11e0 con 0x7fa174101720 2024-09-10T22:01:54.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.167+0000 7fa172a00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa174101720 0x7fa17419b130 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa16400d950 tx=0x7fa16400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:54.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.167+0000 7fa178a00640 1 -- 172.21.15.90:0/981302354 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164017070 con 0x7fa174101720 2024-09-10T22:01:54.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.167+0000 7fa178a00640 1 -- 172.21.15.90:0/981302354 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa164009c30 con 0x7fa174101720 2024-09-10T22:01:54.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.167+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1741a14d0 con 0x7fa174101720 2024-09-10T22:01:54.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.167+0000 7fa178a00640 1 -- 172.21.15.90:0/981302354 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16400f620 con 0x7fa174101720 2024-09-10T22:01:54.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.167+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1741a7d60 con 0x7fa174101720 2024-09-10T22:01:54.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.169+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa17406b130 con 0x7fa174101720 2024-09-10T22:01:54.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.171+0000 7fa178a00640 1 -- 172.21.15.90:0/981302354 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fa164005030 con 0x7fa174101720 2024-09-10T22:01:54.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.172+0000 7fa178a00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa1580b54e0 0x7fa1580b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:54.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.172+0000 7fa178a00640 1 -- 172.21.15.90:0/981302354 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fa1640e5d30 con 0x7fa174101720 2024-09-10T22:01:54.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.173+0000 7fa172000640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa1580b54e0 0x7fa1580b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:54.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.173+0000 7fa172000640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa1580b54e0 0x7fa1580b79a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa160006010 tx=0x7fa160005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:54.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.174+0000 7fa178a00640 1 -- 172.21.15.90:0/981302354 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa164009de0 con 0x7fa174101720 2024-09-10T22:01:54.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.366+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa174110640 con 0x7fa1580b54e0 2024-09-10T22:01:54.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.374+0000 7fa178a00640 1 -- 172.21.15.90:0/981302354 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340655 (secure 0 0 0) 0x7fa174110640 con 0x7fa1580b54e0 2024-09-10T22:01:54.379 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:01:54.381 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-09-10T22:01:54.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.378+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa1580b54e0 msgr2=0x7fa1580b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:54.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.378+0000 7fa173e00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa1580b54e0 0x7fa1580b79a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa160006010 tx=0x7fa160005e80 comp rx=0 tx=0).stop 2024-09-10T22:01:54.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.378+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa174101720 msgr2=0x7fa17419b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:54.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.378+0000 7fa173e00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa174101720 0x7fa17419b130 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa16400d950 tx=0x7fa16400de10 comp rx=0 tx=0).stop 2024-09-10T22:01:54.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.378+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 shutdown_connections 2024-09-10T22:01:54.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.379+0000 7fa173e00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa1580b54e0 0x7fa1580b79a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:54.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.379+0000 7fa173e00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa17410f2d0 0x7fa1741a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:54.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.379+0000 7fa173e00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa174102040 0x7fa17419b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:54.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.379+0000 7fa173e00640 1 --2- 172.21.15.90:0/981302354 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa174101720 0x7fa17419b130 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:54.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.379+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 >> 172.21.15.90:0/981302354 conn(0x7fa1740fd5b0 msgr2=0x7fa1741114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:54.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.380+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 shutdown_connections 2024-09-10T22:01:54.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:54.380+0000 7fa173e00640 1 -- 172.21.15.90:0/981302354 wait complete. 2024-09-10T22:01:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:54 smithi171 ceph-mon[34138]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-10T22:01:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:54 smithi090 ceph-mon[30350]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-10T22:01:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:54 smithi154 ceph-mon[34495]: pgmap v356: 129 pgs: 129 active+clean; 640 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2024-09-10T22:01:55.346 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-10T22:01:53.599299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655842,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":300,"num_read_kb":249,"num_write":369,"num_write_kb":2288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348768,"kb_used_data":11984,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593056,"statfs":{"total":1151940427776,"available":1151583289344,"internally_reserved":0,"allocated":12271616,"data_stored":8120269,"data_compressed":426882,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32912,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":89,"num_write":23,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003470"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:51.782636+0000","last_change":"2024-09-10T22:00:30.824265+0000","last_active":"2024-09-10T22:01:51.782636+0000","last_peered":"2024-09-10T22:01:51.782636+0000","last_clean":"2024-09-10T22:01:51.782636+0000","last_became_active":"2024-09-10T22:00:30.823905+0000","last_became_peered":"2024-09-10T22:00:30.823905+0000","last_unstale":"2024-09-10T22:01:51.782636+0000","last_undegraded":"2024-09-10T22:01:51.782636+0000","last_fullsized":"2024-09-10T22:01:51.782636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:16.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618713+0000","last_change":"2024-09-10T22:00:34.497719+0000","last_active":"2024-09-10T22:01:35.618713+0000","last_peered":"2024-09-10T22:01:35.618713+0000","last_clean":"2024-09-10T22:01:35.618713+0000","last_became_active":"2024-09-10T22:00:34.497380+0000","last_became_peered":"2024-09-10T22:00:34.497380+0000","last_unstale":"2024-09-10T22:01:35.618713+0000","last_undegraded":"2024-09-10T22:01:35.618713+0000","last_fullsized":"2024-09-10T22:01:35.618713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:26.336299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193383+0000","last_change":"2024-09-10T22:00:43.710835+0000","last_active":"2024-09-10T22:01:35.193383+0000","last_peered":"2024-09-10T22:01:35.193383+0000","last_clean":"2024-09-10T22:01:35.193383+0000","last_became_active":"2024-09-10T22:00:43.710363+0000","last_became_peered":"2024-09-10T22:00:43.710363+0000","last_unstale":"2024-09-10T22:01:35.193383+0000","last_undegraded":"2024-09-10T22:01:35.193383+0000","last_fullsized":"2024-09-10T22:01:35.193383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:28.173482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618909+0000","last_change":"2024-09-10T22:00:31.083228+0000","last_active":"2024-09-10T22:01:35.618909+0000","last_peered":"2024-09-10T22:01:35.618909+0000","last_clean":"2024-09-10T22:01:35.618909+0000","last_became_active":"2024-09-10T22:00:31.082911+0000","last_became_peered":"2024-09-10T22:00:31.082911+0000","last_unstale":"2024-09-10T22:01:35.618909+0000","last_undegraded":"2024-09-10T22:01:35.618909+0000","last_fullsized":"2024-09-10T22:01:35.618909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:28.598628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189606+0000","last_change":"2024-09-10T22:00:34.509939+0000","last_active":"2024-09-10T22:01:35.189606+0000","last_peered":"2024-09-10T22:01:35.189606+0000","last_clean":"2024-09-10T22:01:35.189606+0000","last_became_active":"2024-09-10T22:00:34.509536+0000","last_became_peered":"2024-09-10T22:00:34.509536+0000","last_unstale":"2024-09-10T22:01:35.189606+0000","last_undegraded":"2024-09-10T22:01:35.189606+0000","last_fullsized":"2024-09-10T22:01:35.189606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:27.602378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189753+0000","last_change":"2024-09-10T22:00:43.714662+0000","last_active":"2024-09-10T22:01:35.189753+0000","last_peered":"2024-09-10T22:01:35.189753+0000","last_clean":"2024-09-10T22:01:35.189753+0000","last_became_active":"2024-09-10T22:00:43.714074+0000","last_became_peered":"2024-09-10T22:00:43.714074+0000","last_unstale":"2024-09-10T22:01:35.189753+0000","last_undegraded":"2024-09-10T22:01:35.189753+0000","last_fullsized":"2024-09-10T22:01:35.189753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:41.204519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.210089+0000","last_change":"2024-09-10T22:01:35.209979+0000","last_active":"2024-09-10T22:01:35.210089+0000","last_peered":"2024-09-10T22:01:35.210089+0000","last_clean":"2024-09-10T22:01:35.210089+0000","last_became_active":"2024-09-10T22:01:35.197192+0000","last_became_peered":"2024-09-10T22:01:35.197192+0000","last_unstale":"2024-09-10T22:01:35.210089+0000","last_undegraded":"2024-09-10T22:01:35.210089+0000","last_fullsized":"2024-09-10T22:01:35.210089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:47.707679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877107+0000","last_change":"2024-09-10T22:00:34.502795+0000","last_active":"2024-09-10T22:01:35.877107+0000","last_peered":"2024-09-10T22:01:35.877107+0000","last_clean":"2024-09-10T22:01:35.877107+0000","last_became_active":"2024-09-10T22:00:34.501112+0000","last_became_peered":"2024-09-10T22:00:34.501112+0000","last_unstale":"2024-09-10T22:01:35.877107+0000","last_undegraded":"2024-09-10T22:01:35.877107+0000","last_fullsized":"2024-09-10T22:01:35.877107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:45.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.619835+0000","last_change":"2024-09-10T22:01:35.619765+0000","last_active":"2024-09-10T22:01:35.619835+0000","last_peered":"2024-09-10T22:01:35.619835+0000","last_clean":"2024-09-10T22:01:35.619835+0000","last_became_active":"2024-09-10T22:01:35.619407+0000","last_became_peered":"2024-09-10T22:01:35.619407+0000","last_unstale":"2024-09-10T22:01:35.619835+0000","last_undegraded":"2024-09-10T22:01:35.619835+0000","last_fullsized":"2024-09-10T22:01:35.619835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:17.064585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189727+0000","last_change":"2024-09-10T22:00:30.818224+0000","last_active":"2024-09-10T22:01:35.189727+0000","last_peered":"2024-09-10T22:01:35.189727+0000","last_clean":"2024-09-10T22:01:35.189727+0000","last_became_active":"2024-09-10T22:00:30.817781+0000","last_became_peered":"2024-09-10T22:00:30.817781+0000","last_unstale":"2024-09-10T22:01:35.189727+0000","last_undegraded":"2024-09-10T22:01:35.189727+0000","last_fullsized":"2024-09-10T22:01:35.189727+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:47.097642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.444107+0000","last_change":"2024-09-10T22:00:34.499281+0000","last_active":"2024-09-10T22:01:35.444107+0000","last_peered":"2024-09-10T22:01:35.444107+0000","last_clean":"2024-09-10T22:01:35.444107+0000","last_became_active":"2024-09-10T22:00:34.498914+0000","last_became_peered":"2024-09-10T22:00:34.498914+0000","last_unstale":"2024-09-10T22:01:35.444107+0000","last_undegraded":"2024-09-10T22:01:35.444107+0000","last_fullsized":"2024-09-10T22:01:35.444107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:47.838346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189862+0000","last_change":"2024-09-10T22:00:43.712066+0000","last_active":"2024-09-10T22:01:35.189862+0000","last_peered":"2024-09-10T22:01:35.189862+0000","last_clean":"2024-09-10T22:01:35.189862+0000","last_became_active":"2024-09-10T22:00:43.711614+0000","last_became_peered":"2024-09-10T22:00:43.711614+0000","last_unstale":"2024-09-10T22:01:35.189862+0000","last_undegraded":"2024-09-10T22:01:35.189862+0000","last_fullsized":"2024-09-10T22:01:35.189862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:42.403367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195148+0000","last_change":"2024-09-10T22:00:34.501943+0000","last_active":"2024-09-10T22:01:35.195148+0000","last_peered":"2024-09-10T22:01:35.195148+0000","last_clean":"2024-09-10T22:01:35.195148+0000","last_became_active":"2024-09-10T22:00:34.501602+0000","last_became_peered":"2024-09-10T22:00:34.501602+0000","last_unstale":"2024-09-10T22:01:35.195148+0000","last_undegraded":"2024-09-10T22:01:35.195148+0000","last_fullsized":"2024-09-10T22:01:35.195148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:53.200974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.200844+0000","last_change":"2024-09-10T22:01:35.200791+0000","last_active":"2024-09-10T22:01:35.200844+0000","last_peered":"2024-09-10T22:01:35.200844+0000","last_clean":"2024-09-10T22:01:35.200844+0000","last_became_active":"2024-09-10T22:01:35.196059+0000","last_became_peered":"2024-09-10T22:01:35.196059+0000","last_unstale":"2024-09-10T22:01:35.200844+0000","last_undegraded":"2024-09-10T22:01:35.200844+0000","last_fullsized":"2024-09-10T22:01:35.200844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:51.349216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189977+0000","last_change":"2024-09-10T22:00:43.760567+0000","last_active":"2024-09-10T22:01:35.189977+0000","last_peered":"2024-09-10T22:01:35.189977+0000","last_clean":"2024-09-10T22:01:35.189977+0000","last_became_active":"2024-09-10T22:00:43.760064+0000","last_became_peered":"2024-09-10T22:00:43.760064+0000","last_unstale":"2024-09-10T22:01:35.189977+0000","last_undegraded":"2024-09-10T22:01:35.189977+0000","last_fullsized":"2024-09-10T22:01:35.189977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:55.469525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618453+0000","last_change":"2024-09-10T22:00:34.486225+0000","last_active":"2024-09-10T22:01:35.618453+0000","last_peered":"2024-09-10T22:01:35.618453+0000","last_clean":"2024-09-10T22:01:35.618453+0000","last_became_active":"2024-09-10T22:00:34.485896+0000","last_became_peered":"2024-09-10T22:00:34.485896+0000","last_unstale":"2024-09-10T22:01:35.618453+0000","last_undegraded":"2024-09-10T22:01:35.618453+0000","last_fullsized":"2024-09-10T22:01:35.618453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:00.151927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194272+0000","last_change":"2024-09-10T22:00:30.817958+0000","last_active":"2024-09-10T22:01:35.194272+0000","last_peered":"2024-09-10T22:01:35.194272+0000","last_clean":"2024-09-10T22:01:35.194272+0000","last_became_active":"2024-09-10T22:00:30.817646+0000","last_became_peered":"2024-09-10T22:00:30.817646+0000","last_unstale":"2024-09-10T22:01:35.194272+0000","last_undegraded":"2024-09-10T22:01:35.194272+0000","last_fullsized":"2024-09-10T22:01:35.194272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:27.722184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190037+0000","last_change":"2024-09-10T22:00:43.716465+0000","last_active":"2024-09-10T22:01:35.190037+0000","last_peered":"2024-09-10T22:01:35.190037+0000","last_clean":"2024-09-10T22:01:35.190037+0000","last_became_active":"2024-09-10T22:00:43.716165+0000","last_became_peered":"2024-09-10T22:00:43.716165+0000","last_unstale":"2024-09-10T22:01:35.190037+0000","last_undegraded":"2024-09-10T22:01:35.190037+0000","last_fullsized":"2024-09-10T22:01:35.190037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:18.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193477+0000","last_change":"2024-09-10T22:00:34.493992+0000","last_active":"2024-09-10T22:01:35.193477+0000","last_peered":"2024-09-10T22:01:35.193477+0000","last_clean":"2024-09-10T22:01:35.193477+0000","last_became_active":"2024-09-10T22:00:34.493670+0000","last_became_peered":"2024-09-10T22:00:34.493670+0000","last_unstale":"2024-09-10T22:01:35.193477+0000","last_undegraded":"2024-09-10T22:01:35.193477+0000","last_fullsized":"2024-09-10T22:01:35.193477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:04.991819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618513+0000","last_change":"2024-09-10T22:00:30.819830+0000","last_active":"2024-09-10T22:01:35.618513+0000","last_peered":"2024-09-10T22:01:35.618513+0000","last_clean":"2024-09-10T22:01:35.618513+0000","last_became_active":"2024-09-10T22:00:30.819401+0000","last_became_peered":"2024-09-10T22:00:30.819401+0000","last_unstale":"2024-09-10T22:01:35.618513+0000","last_undegraded":"2024-09-10T22:01:35.618513+0000","last_fullsized":"2024-09-10T22:01:35.618513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:37.823107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595833+0000","last_change":"2024-09-10T22:00:43.718500+0000","last_active":"2024-09-10T22:01:35.595833+0000","last_peered":"2024-09-10T22:01:35.595833+0000","last_clean":"2024-09-10T22:01:35.595833+0000","last_became_active":"2024-09-10T22:00:43.717667+0000","last_became_peered":"2024-09-10T22:00:43.717667+0000","last_unstale":"2024-09-10T22:01:35.595833+0000","last_undegraded":"2024-09-10T22:01:35.595833+0000","last_fullsized":"2024-09-10T22:01:35.595833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:23.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189221+0000","last_change":"2024-09-10T22:00:34.494162+0000","last_active":"2024-09-10T22:01:35.189221+0000","last_peered":"2024-09-10T22:01:35.189221+0000","last_clean":"2024-09-10T22:01:35.189221+0000","last_became_active":"2024-09-10T22:00:34.493819+0000","last_became_peered":"2024-09-10T22:00:34.493819+0000","last_unstale":"2024-09-10T22:01:35.189221+0000","last_undegraded":"2024-09-10T22:01:35.189221+0000","last_fullsized":"2024-09-10T22:01:35.189221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:43.264363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877624+0000","last_change":"2024-09-10T22:00:31.081559+0000","last_active":"2024-09-10T22:01:35.877624+0000","last_peered":"2024-09-10T22:01:35.877624+0000","last_clean":"2024-09-10T22:01:35.877624+0000","last_became_active":"2024-09-10T22:00:31.081257+0000","last_became_peered":"2024-09-10T22:00:31.081257+0000","last_unstale":"2024-09-10T22:01:35.877624+0000","last_undegraded":"2024-09-10T22:01:35.877624+0000","last_fullsized":"2024-09-10T22:01:35.877624+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:53.235765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.445506+0000","last_change":"2024-09-10T22:01:35.445171+0000","last_active":"2024-09-10T22:01:35.445506+0000","last_peered":"2024-09-10T22:01:35.445506+0000","last_clean":"2024-09-10T22:01:35.445506+0000","last_became_active":"2024-09-10T22:01:35.444651+0000","last_became_peered":"2024-09-10T22:01:35.444651+0000","last_unstale":"2024-09-10T22:01:35.445506+0000","last_undegraded":"2024-09-10T22:01:35.445506+0000","last_fullsized":"2024-09-10T22:01:35.445506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:36.457151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.751094+0000","last_change":"2024-09-10T22:01:35.750982+0000","last_active":"2024-09-10T22:01:35.751094+0000","last_peered":"2024-09-10T22:01:35.751094+0000","last_clean":"2024-09-10T22:01:35.751094+0000","last_became_active":"2024-09-10T22:01:35.750572+0000","last_became_peered":"2024-09-10T22:01:35.750572+0000","last_unstale":"2024-09-10T22:01:35.751094+0000","last_undegraded":"2024-09-10T22:01:35.751094+0000","last_fullsized":"2024-09-10T22:01:35.751094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:36.596313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190527+0000","last_change":"2024-09-10T22:00:31.311864+0000","last_active":"2024-09-10T22:01:35.190527+0000","last_peered":"2024-09-10T22:01:35.190527+0000","last_clean":"2024-09-10T22:01:35.190527+0000","last_became_active":"2024-09-10T22:00:31.311611+0000","last_became_peered":"2024-09-10T22:00:31.311611+0000","last_unstale":"2024-09-10T22:01:35.190527+0000","last_undegraded":"2024-09-10T22:01:35.190527+0000","last_fullsized":"2024-09-10T22:01:35.190527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:27.969999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193840+0000","last_change":"2024-09-10T22:00:43.708023+0000","last_active":"2024-09-10T22:01:35.193840+0000","last_peered":"2024-09-10T22:01:35.193840+0000","last_clean":"2024-09-10T22:01:35.193840+0000","last_became_active":"2024-09-10T22:00:43.707086+0000","last_became_peered":"2024-09-10T22:00:43.707086+0000","last_unstale":"2024-09-10T22:01:35.193840+0000","last_undegraded":"2024-09-10T22:01:35.193840+0000","last_fullsized":"2024-09-10T22:01:35.193840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:15.585104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618945+0000","last_change":"2024-09-10T22:00:34.494104+0000","last_active":"2024-09-10T22:01:35.618945+0000","last_peered":"2024-09-10T22:01:35.618945+0000","last_clean":"2024-09-10T22:01:35.618945+0000","last_became_active":"2024-09-10T22:00:34.493404+0000","last_became_peered":"2024-09-10T22:00:34.493404+0000","last_unstale":"2024-09-10T22:01:35.618945+0000","last_undegraded":"2024-09-10T22:01:35.618945+0000","last_fullsized":"2024-09-10T22:01:35.618945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:47.444100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:48.513188+0000","last_change":"2024-09-10T22:00:30.819427+0000","last_active":"2024-09-10T22:01:48.513188+0000","last_peered":"2024-09-10T22:01:48.513188+0000","last_clean":"2024-09-10T22:01:48.513188+0000","last_became_active":"2024-09-10T22:00:30.819098+0000","last_became_peered":"2024-09-10T22:00:30.819098+0000","last_unstale":"2024-09-10T22:01:48.513188+0000","last_undegraded":"2024-09-10T22:01:48.513188+0000","last_fullsized":"2024-09-10T22:01:48.513188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:23.068155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618087+0000","last_change":"2024-09-10T22:00:43.760602+0000","last_active":"2024-09-10T22:01:35.618087+0000","last_peered":"2024-09-10T22:01:35.618087+0000","last_clean":"2024-09-10T22:01:35.618087+0000","last_became_active":"2024-09-10T22:00:43.760172+0000","last_became_peered":"2024-09-10T22:00:43.760172+0000","last_unstale":"2024-09-10T22:01:35.618087+0000","last_undegraded":"2024-09-10T22:01:35.618087+0000","last_fullsized":"2024-09-10T22:01:35.618087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:40.332101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749946+0000","last_change":"2024-09-10T22:00:34.501632+0000","last_active":"2024-09-10T22:01:35.749946+0000","last_peered":"2024-09-10T22:01:35.749946+0000","last_clean":"2024-09-10T22:01:35.749946+0000","last_became_active":"2024-09-10T22:00:34.500924+0000","last_became_peered":"2024-09-10T22:00:34.500924+0000","last_unstale":"2024-09-10T22:01:35.749946+0000","last_undegraded":"2024-09-10T22:01:35.749946+0000","last_fullsized":"2024-09-10T22:01:35.749946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:28.052570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190019+0000","last_change":"2024-09-10T22:00:31.081391+0000","last_active":"2024-09-10T22:01:35.190019+0000","last_peered":"2024-09-10T22:01:35.190019+0000","last_clean":"2024-09-10T22:01:35.190019+0000","last_became_active":"2024-09-10T22:00:31.081018+0000","last_became_peered":"2024-09-10T22:00:31.081018+0000","last_unstale":"2024-09-10T22:01:35.190019+0000","last_undegraded":"2024-09-10T22:01:35.190019+0000","last_fullsized":"2024-09-10T22:01:35.190019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:09.450785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193019+0000","last_change":"2024-09-10T22:00:43.711592+0000","last_active":"2024-09-10T22:01:35.193019+0000","last_peered":"2024-09-10T22:01:35.193019+0000","last_clean":"2024-09-10T22:01:35.193019+0000","last_became_active":"2024-09-10T22:00:43.711144+0000","last_became_peered":"2024-09-10T22:00:43.711144+0000","last_unstale":"2024-09-10T22:01:35.193019+0000","last_undegraded":"2024-09-10T22:01:35.193019+0000","last_fullsized":"2024-09-10T22:01:35.193019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:01.818907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.192436+0000","last_change":"2024-09-10T22:00:34.494070+0000","last_active":"2024-09-10T22:01:35.192436+0000","last_peered":"2024-09-10T22:01:35.192436+0000","last_clean":"2024-09-10T22:01:35.192436+0000","last_became_active":"2024-09-10T22:00:34.493324+0000","last_became_peered":"2024-09-10T22:00:34.493324+0000","last_unstale":"2024-09-10T22:01:35.192436+0000","last_undegraded":"2024-09-10T22:01:35.192436+0000","last_fullsized":"2024-09-10T22:01:35.192436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:29.969743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595852+0000","last_change":"2024-09-10T22:00:31.082439+0000","last_active":"2024-09-10T22:01:35.595852+0000","last_peered":"2024-09-10T22:01:35.595852+0000","last_clean":"2024-09-10T22:01:35.595852+0000","last_became_active":"2024-09-10T22:00:31.082025+0000","last_became_peered":"2024-09-10T22:00:31.082025+0000","last_unstale":"2024-09-10T22:01:35.595852+0000","last_undegraded":"2024-09-10T22:01:35.595852+0000","last_fullsized":"2024-09-10T22:01:35.595852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:21.933419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189264+0000","last_change":"2024-09-10T22:00:43.760997+0000","last_active":"2024-09-10T22:01:35.189264+0000","last_peered":"2024-09-10T22:01:35.189264+0000","last_clean":"2024-09-10T22:01:35.189264+0000","last_became_active":"2024-09-10T22:00:43.760124+0000","last_became_peered":"2024-09-10T22:00:43.760124+0000","last_unstale":"2024-09-10T22:01:35.189264+0000","last_undegraded":"2024-09-10T22:01:35.189264+0000","last_fullsized":"2024-09-10T22:01:35.189264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:29.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.192492+0000","last_change":"2024-09-10T22:00:34.511730+0000","last_active":"2024-09-10T22:01:35.192492+0000","last_peered":"2024-09-10T22:01:35.192492+0000","last_clean":"2024-09-10T22:01:35.192492+0000","last_became_active":"2024-09-10T22:00:34.511440+0000","last_became_peered":"2024-09-10T22:00:34.511440+0000","last_unstale":"2024-09-10T22:01:35.192492+0000","last_undegraded":"2024-09-10T22:01:35.192492+0000","last_fullsized":"2024-09-10T22:01:35.192492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:09.053835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":65,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:47.777312+0000","last_change":"2024-09-10T22:00:30.825999+0000","last_active":"2024-09-10T22:01:47.777312+0000","last_peered":"2024-09-10T22:01:47.777312+0000","last_clean":"2024-09-10T22:01:47.777312+0000","last_became_active":"2024-09-10T22:00:30.825334+0000","last_became_peered":"2024-09-10T22:00:30.825334+0000","last_unstale":"2024-09-10T22:01:47.777312+0000","last_undegraded":"2024-09-10T22:01:47.777312+0000","last_fullsized":"2024-09-10T22:01:47.777312+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:14.684222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749605+0000","last_change":"2024-09-10T22:00:43.710517+0000","last_active":"2024-09-10T22:01:35.749605+0000","last_peered":"2024-09-10T22:01:35.749605+0000","last_clean":"2024-09-10T22:01:35.749605+0000","last_became_active":"2024-09-10T22:00:43.709252+0000","last_became_peered":"2024-09-10T22:00:43.709252+0000","last_unstale":"2024-09-10T22:01:35.749605+0000","last_undegraded":"2024-09-10T22:01:35.749605+0000","last_fullsized":"2024-09-10T22:01:35.749605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:21.355152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443126+0000","last_change":"2024-09-10T22:00:34.498168+0000","last_active":"2024-09-10T22:01:35.443126+0000","last_peered":"2024-09-10T22:01:35.443126+0000","last_clean":"2024-09-10T22:01:35.443126+0000","last_became_active":"2024-09-10T22:00:34.497867+0000","last_became_peered":"2024-09-10T22:00:34.497867+0000","last_unstale":"2024-09-10T22:01:35.443126+0000","last_undegraded":"2024-09-10T22:01:35.443126+0000","last_fullsized":"2024-09-10T22:01:35.443126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:14.570265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":61,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:47.791684+0000","last_change":"2024-09-10T22:00:31.081590+0000","last_active":"2024-09-10T22:01:47.791684+0000","last_peered":"2024-09-10T22:01:47.791684+0000","last_clean":"2024-09-10T22:01:47.791684+0000","last_became_active":"2024-09-10T22:00:31.081250+0000","last_became_peered":"2024-09-10T22:00:31.081250+0000","last_unstale":"2024-09-10T22:01:47.791684+0000","last_undegraded":"2024-09-10T22:01:47.791684+0000","last_fullsized":"2024-09-10T22:01:47.791684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:12.872832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":18,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749980+0000","last_change":"2024-09-10T22:00:43.761351+0000","last_active":"2024-09-10T22:01:35.749980+0000","last_peered":"2024-09-10T22:01:35.749980+0000","last_clean":"2024-09-10T22:01:35.749980+0000","last_became_active":"2024-09-10T22:00:43.760903+0000","last_became_peered":"2024-09-10T22:00:43.760903+0000","last_unstale":"2024-09-10T22:01:35.749980+0000","last_undegraded":"2024-09-10T22:01:35.749980+0000","last_fullsized":"2024-09-10T22:01:35.749980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:14.113974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749203+0000","last_change":"2024-09-10T22:00:34.504671+0000","last_active":"2024-09-10T22:01:35.749203+0000","last_peered":"2024-09-10T22:01:35.749203+0000","last_clean":"2024-09-10T22:01:35.749203+0000","last_became_active":"2024-09-10T22:00:34.504368+0000","last_became_peered":"2024-09-10T22:00:34.504368+0000","last_unstale":"2024-09-10T22:01:35.749203+0000","last_undegraded":"2024-09-10T22:01:35.749203+0000","last_fullsized":"2024-09-10T22:01:35.749203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:59.771589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194611+0000","last_change":"2024-09-10T22:00:30.820401+0000","last_active":"2024-09-10T22:01:35.194611+0000","last_peered":"2024-09-10T22:01:35.194611+0000","last_clean":"2024-09-10T22:01:35.194611+0000","last_became_active":"2024-09-10T22:00:30.820194+0000","last_became_peered":"2024-09-10T22:00:30.820194+0000","last_unstale":"2024-09-10T22:01:35.194611+0000","last_undegraded":"2024-09-10T22:01:35.194611+0000","last_fullsized":"2024-09-10T22:01:35.194611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:47.767102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443924+0000","last_change":"2024-09-10T22:00:43.762001+0000","last_active":"2024-09-10T22:01:35.443924+0000","last_peered":"2024-09-10T22:01:35.443924+0000","last_clean":"2024-09-10T22:01:35.443924+0000","last_became_active":"2024-09-10T22:00:43.761323+0000","last_became_peered":"2024-09-10T22:00:43.761323+0000","last_unstale":"2024-09-10T22:01:35.443924+0000","last_undegraded":"2024-09-10T22:01:35.443924+0000","last_fullsized":"2024-09-10T22:01:35.443924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:50.113545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618461+0000","last_change":"2024-09-10T22:00:34.498591+0000","last_active":"2024-09-10T22:01:35.618461+0000","last_peered":"2024-09-10T22:01:35.618461+0000","last_clean":"2024-09-10T22:01:35.618461+0000","last_became_active":"2024-09-10T22:00:34.496449+0000","last_became_peered":"2024-09-10T22:00:34.496449+0000","last_unstale":"2024-09-10T22:01:35.618461+0000","last_undegraded":"2024-09-10T22:01:35.618461+0000","last_fullsized":"2024-09-10T22:01:35.618461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:54.815268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.191416+0000","last_change":"2024-09-10T22:00:30.817201+0000","last_active":"2024-09-10T22:01:35.191416+0000","last_peered":"2024-09-10T22:01:35.191416+0000","last_clean":"2024-09-10T22:01:35.191416+0000","last_became_active":"2024-09-10T22:00:30.816792+0000","last_became_peered":"2024-09-10T22:00:30.816792+0000","last_unstale":"2024-09-10T22:01:35.191416+0000","last_undegraded":"2024-09-10T22:01:35.191416+0000","last_fullsized":"2024-09-10T22:01:35.191416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:48.239811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194713+0000","last_change":"2024-09-10T21:59:01.830452+0000","last_active":"2024-09-10T22:01:35.194713+0000","last_peered":"2024-09-10T22:01:35.194713+0000","last_clean":"2024-09-10T22:01:35.194713+0000","last_became_active":"2024-09-10T21:59:01.823744+0000","last_became_peered":"2024-09-10T21:59:01.823744+0000","last_unstale":"2024-09-10T22:01:35.194713+0000","last_undegraded":"2024-09-10T22:01:35.194713+0000","last_fullsized":"2024-09-10T22:01:35.194713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_clean_scrub_stamp":"2024-09-10T21:54:48.621299+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:39.573257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194993+0000","last_change":"2024-09-10T22:00:43.713888+0000","last_active":"2024-09-10T22:01:35.194993+0000","last_peered":"2024-09-10T22:01:35.194993+0000","last_clean":"2024-09-10T22:01:35.194993+0000","last_became_active":"2024-09-10T22:00:43.713329+0000","last_became_peered":"2024-09-10T22:00:43.713329+0000","last_unstale":"2024-09-10T22:01:35.194993+0000","last_undegraded":"2024-09-10T22:01:35.194993+0000","last_fullsized":"2024-09-10T22:01:35.194993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:01.062089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195205+0000","last_change":"2024-09-10T22:00:34.502589+0000","last_active":"2024-09-10T22:01:35.195205+0000","last_peered":"2024-09-10T22:01:35.195205+0000","last_clean":"2024-09-10T22:01:35.195205+0000","last_became_active":"2024-09-10T22:00:34.502216+0000","last_became_peered":"2024-09-10T22:00:34.502216+0000","last_unstale":"2024-09-10T22:01:35.195205+0000","last_undegraded":"2024-09-10T22:01:35.195205+0000","last_fullsized":"2024-09-10T22:01:35.195205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:44.959375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":64,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:46.776918+0000","last_change":"2024-09-10T22:00:31.311807+0000","last_active":"2024-09-10T22:01:46.776918+0000","last_peered":"2024-09-10T22:01:46.776918+0000","last_clean":"2024-09-10T22:01:46.776918+0000","last_became_active":"2024-09-10T22:00:31.311424+0000","last_became_peered":"2024-09-10T22:00:31.311424+0000","last_unstale":"2024-09-10T22:01:46.776918+0000","last_undegraded":"2024-09-10T22:01:46.776918+0000","last_fullsized":"2024-09-10T22:01:46.776918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:09.754777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.876960+0000","last_change":"2024-09-10T22:00:43.711882+0000","last_active":"2024-09-10T22:01:35.876960+0000","last_peered":"2024-09-10T22:01:35.876960+0000","last_clean":"2024-09-10T22:01:35.876960+0000","last_became_active":"2024-09-10T22:00:43.711408+0000","last_became_peered":"2024-09-10T22:00:43.711408+0000","last_unstale":"2024-09-10T22:01:35.876960+0000","last_undegraded":"2024-09-10T22:01:35.876960+0000","last_fullsized":"2024-09-10T22:01:35.876960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:31.155723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195251+0000","last_change":"2024-09-10T22:00:34.505653+0000","last_active":"2024-09-10T22:01:35.195251+0000","last_peered":"2024-09-10T22:01:35.195251+0000","last_clean":"2024-09-10T22:01:35.195251+0000","last_became_active":"2024-09-10T22:00:34.505317+0000","last_became_peered":"2024-09-10T22:00:34.505317+0000","last_unstale":"2024-09-10T22:01:35.195251+0000","last_undegraded":"2024-09-10T22:01:35.195251+0000","last_fullsized":"2024-09-10T22:01:35.195251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:35.050892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.444093+0000","last_change":"2024-09-10T22:00:30.817431+0000","last_active":"2024-09-10T22:01:35.444093+0000","last_peered":"2024-09-10T22:01:35.444093+0000","last_clean":"2024-09-10T22:01:35.444093+0000","last_became_active":"2024-09-10T22:00:30.816952+0000","last_became_peered":"2024-09-10T22:00:30.816952+0000","last_unstale":"2024-09-10T22:01:35.444093+0000","last_undegraded":"2024-09-10T22:01:35.444093+0000","last_fullsized":"2024-09-10T22:01:35.444093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:37.881641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189475+0000","last_change":"2024-09-10T22:00:43.720627+0000","last_active":"2024-09-10T22:01:35.189475+0000","last_peered":"2024-09-10T22:01:35.189475+0000","last_clean":"2024-09-10T22:01:35.189475+0000","last_became_active":"2024-09-10T22:00:43.720235+0000","last_became_peered":"2024-09-10T22:00:43.720235+0000","last_unstale":"2024-09-10T22:01:35.189475+0000","last_undegraded":"2024-09-10T22:01:35.189475+0000","last_fullsized":"2024-09-10T22:01:35.189475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:50.549536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195435+0000","last_change":"2024-09-10T22:00:34.501649+0000","last_active":"2024-09-10T22:01:35.195435+0000","last_peered":"2024-09-10T22:01:35.195435+0000","last_clean":"2024-09-10T22:01:35.195435+0000","last_became_active":"2024-09-10T22:00:34.498882+0000","last_became_peered":"2024-09-10T22:00:34.498882+0000","last_unstale":"2024-09-10T22:01:35.195435+0000","last_undegraded":"2024-09-10T22:01:35.195435+0000","last_fullsized":"2024-09-10T22:01:35.195435+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.750071+0000","last_change":"2024-09-10T22:00:30.821943+0000","last_active":"2024-09-10T22:01:35.750071+0000","last_peered":"2024-09-10T22:01:35.750071+0000","last_clean":"2024-09-10T22:01:35.750071+0000","last_became_active":"2024-09-10T22:00:30.821372+0000","last_became_peered":"2024-09-10T22:00:30.821372+0000","last_unstale":"2024-09-10T22:01:35.750071+0000","last_undegraded":"2024-09-10T22:01:35.750071+0000","last_fullsized":"2024-09-10T22:01:35.750071+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:04.360383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.442853+0000","last_change":"2024-09-10T22:00:43.710045+0000","last_active":"2024-09-10T22:01:35.442853+0000","last_peered":"2024-09-10T22:01:35.442853+0000","last_clean":"2024-09-10T22:01:35.442853+0000","last_became_active":"2024-09-10T22:00:43.709416+0000","last_became_peered":"2024-09-10T22:00:43.709416+0000","last_unstale":"2024-09-10T22:01:35.442853+0000","last_undegraded":"2024-09-10T22:01:35.442853+0000","last_fullsized":"2024-09-10T22:01:35.442853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:56.005549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195561+0000","last_change":"2024-09-10T22:00:34.496163+0000","last_active":"2024-09-10T22:01:35.195561+0000","last_peered":"2024-09-10T22:01:35.195561+0000","last_clean":"2024-09-10T22:01:35.195561+0000","last_became_active":"2024-09-10T22:00:34.495814+0000","last_became_peered":"2024-09-10T22:00:34.495814+0000","last_unstale":"2024-09-10T22:01:35.195561+0000","last_undegraded":"2024-09-10T22:01:35.195561+0000","last_fullsized":"2024-09-10T22:01:35.195561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:05.398154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189944+0000","last_change":"2024-09-10T22:00:30.812702+0000","last_active":"2024-09-10T22:01:35.189944+0000","last_peered":"2024-09-10T22:01:35.189944+0000","last_clean":"2024-09-10T22:01:35.189944+0000","last_became_active":"2024-09-10T22:00:30.812369+0000","last_became_peered":"2024-09-10T22:00:30.812369+0000","last_unstale":"2024-09-10T22:01:35.189944+0000","last_undegraded":"2024-09-10T22:01:35.189944+0000","last_fullsized":"2024-09-10T22:01:35.189944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:52.332876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443239+0000","last_change":"2024-09-10T22:00:43.715573+0000","last_active":"2024-09-10T22:01:35.443239+0000","last_peered":"2024-09-10T22:01:35.443239+0000","last_clean":"2024-09-10T22:01:35.443239+0000","last_became_active":"2024-09-10T22:00:43.714475+0000","last_became_peered":"2024-09-10T22:00:43.714475+0000","last_unstale":"2024-09-10T22:01:35.443239+0000","last_undegraded":"2024-09-10T22:01:35.443239+0000","last_fullsized":"2024-09-10T22:01:35.443239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:55.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443379+0000","last_change":"2024-09-10T22:00:34.505126+0000","last_active":"2024-09-10T22:01:35.443379+0000","last_peered":"2024-09-10T22:01:35.443379+0000","last_clean":"2024-09-10T22:01:35.443379+0000","last_became_active":"2024-09-10T22:00:34.504653+0000","last_became_peered":"2024-09-10T22:00:34.504653+0000","last_unstale":"2024-09-10T22:01:35.443379+0000","last_undegraded":"2024-09-10T22:01:35.443379+0000","last_fullsized":"2024-09-10T22:01:35.443379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:10.202902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.903455+0000","last_change":"2024-09-10T22:01:35.903381+0000","last_active":"2024-09-10T22:01:35.903455+0000","last_peered":"2024-09-10T22:01:35.903455+0000","last_clean":"2024-09-10T22:01:35.903455+0000","last_became_active":"2024-09-10T22:01:35.878262+0000","last_became_peered":"2024-09-10T22:01:35.878262+0000","last_unstale":"2024-09-10T22:01:35.903455+0000","last_undegraded":"2024-09-10T22:01:35.903455+0000","last_fullsized":"2024-09-10T22:01:35.903455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:45.419449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.750036+0000","last_change":"2024-09-10T22:00:43.720728+0000","last_active":"2024-09-10T22:01:35.750036+0000","last_peered":"2024-09-10T22:01:35.750036+0000","last_clean":"2024-09-10T22:01:35.750036+0000","last_became_active":"2024-09-10T22:00:43.720305+0000","last_became_peered":"2024-09-10T22:00:43.720305+0000","last_unstale":"2024-09-10T22:01:35.750036+0000","last_undegraded":"2024-09-10T22:01:35.750036+0000","last_fullsized":"2024-09-10T22:01:35.750036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:54.986822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194548+0000","last_change":"2024-09-10T22:00:34.501571+0000","last_active":"2024-09-10T22:01:35.194548+0000","last_peered":"2024-09-10T22:01:35.194548+0000","last_clean":"2024-09-10T22:01:35.194548+0000","last_became_active":"2024-09-10T22:00:34.501211+0000","last_became_peered":"2024-09-10T22:00:34.501211+0000","last_unstale":"2024-09-10T22:01:35.194548+0000","last_undegraded":"2024-09-10T22:01:35.194548+0000","last_fullsized":"2024-09-10T22:01:35.194548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:05.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:50.798500+0000","last_change":"2024-09-10T22:00:30.824296+0000","last_active":"2024-09-10T22:01:50.798500+0000","last_peered":"2024-09-10T22:01:50.798500+0000","last_clean":"2024-09-10T22:01:50.798500+0000","last_became_active":"2024-09-10T22:00:30.823886+0000","last_became_peered":"2024-09-10T22:00:30.823886+0000","last_unstale":"2024-09-10T22:01:50.798500+0000","last_undegraded":"2024-09-10T22:01:50.798500+0000","last_fullsized":"2024-09-10T22:01:50.798500+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:26.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595913+0000","last_change":"2024-09-10T22:00:43.717580+0000","last_active":"2024-09-10T22:01:35.595913+0000","last_peered":"2024-09-10T22:01:35.595913+0000","last_clean":"2024-09-10T22:01:35.595913+0000","last_became_active":"2024-09-10T22:00:43.708377+0000","last_became_peered":"2024-09-10T22:00:43.708377+0000","last_unstale":"2024-09-10T22:01:35.595913+0000","last_undegraded":"2024-09-10T22:01:35.595913+0000","last_fullsized":"2024-09-10T22:01:35.595913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:54.682292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195209+0000","last_change":"2024-09-10T22:00:34.514348+0000","last_active":"2024-09-10T22:01:35.195209+0000","last_peered":"2024-09-10T22:01:35.195209+0000","last_clean":"2024-09-10T22:01:35.195209+0000","last_became_active":"2024-09-10T22:00:34.513797+0000","last_became_peered":"2024-09-10T22:00:34.513797+0000","last_unstale":"2024-09-10T22:01:35.195209+0000","last_undegraded":"2024-09-10T22:01:35.195209+0000","last_fullsized":"2024-09-10T22:01:35.195209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:12.797643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443518+0000","last_change":"2024-09-10T22:00:30.826486+0000","last_active":"2024-09-10T22:01:35.443518+0000","last_peered":"2024-09-10T22:01:35.443518+0000","last_clean":"2024-09-10T22:01:35.443518+0000","last_became_active":"2024-09-10T22:00:30.826111+0000","last_became_peered":"2024-09-10T22:00:30.826111+0000","last_unstale":"2024-09-10T22:01:35.443518+0000","last_undegraded":"2024-09-10T22:01:35.443518+0000","last_fullsized":"2024-09-10T22:01:35.443518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:45.609925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193020+0000","last_change":"2024-09-10T22:00:43.709432+0000","last_active":"2024-09-10T22:01:35.193020+0000","last_peered":"2024-09-10T22:01:35.193020+0000","last_clean":"2024-09-10T22:01:35.193020+0000","last_became_active":"2024-09-10T22:00:43.708627+0000","last_became_peered":"2024-09-10T22:00:43.708627+0000","last_unstale":"2024-09-10T22:01:35.193020+0000","last_undegraded":"2024-09-10T22:01:35.193020+0000","last_fullsized":"2024-09-10T22:01:35.193020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:12.799976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749410+0000","last_change":"2024-09-10T22:00:34.499203+0000","last_active":"2024-09-10T22:01:35.749410+0000","last_peered":"2024-09-10T22:01:35.749410+0000","last_clean":"2024-09-10T22:01:35.749410+0000","last_became_active":"2024-09-10T22:00:34.497877+0000","last_became_peered":"2024-09-10T22:00:34.497877+0000","last_unstale":"2024-09-10T22:01:35.749410+0000","last_undegraded":"2024-09-10T22:01:35.749410+0000","last_fullsized":"2024-09-10T22:01:35.749410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:41.897603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195380+0000","last_change":"2024-09-10T22:00:30.820852+0000","last_active":"2024-09-10T22:01:35.195380+0000","last_peered":"2024-09-10T22:01:35.195380+0000","last_clean":"2024-09-10T22:01:35.195380+0000","last_became_active":"2024-09-10T22:00:30.820023+0000","last_became_peered":"2024-09-10T22:00:30.820023+0000","last_unstale":"2024-09-10T22:01:35.195380+0000","last_undegraded":"2024-09-10T22:01:35.195380+0000","last_fullsized":"2024-09-10T22:01:35.195380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:54.245142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443707+0000","last_change":"2024-09-10T22:00:43.716323+0000","last_active":"2024-09-10T22:01:35.443707+0000","last_peered":"2024-09-10T22:01:35.443707+0000","last_clean":"2024-09-10T22:01:35.443707+0000","last_became_active":"2024-09-10T22:00:43.715827+0000","last_became_peered":"2024-09-10T22:00:43.715827+0000","last_unstale":"2024-09-10T22:01:35.443707+0000","last_undegraded":"2024-09-10T22:01:35.443707+0000","last_fullsized":"2024-09-10T22:01:35.443707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:10.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189839+0000","last_change":"2024-09-10T22:00:34.500114+0000","last_active":"2024-09-10T22:01:35.189839+0000","last_peered":"2024-09-10T22:01:35.189839+0000","last_clean":"2024-09-10T22:01:35.189839+0000","last_became_active":"2024-09-10T22:00:34.499334+0000","last_became_peered":"2024-09-10T22:00:34.499334+0000","last_unstale":"2024-09-10T22:01:35.189839+0000","last_undegraded":"2024-09-10T22:01:35.189839+0000","last_fullsized":"2024-09-10T22:01:35.189839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:11.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'31","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:49.788840+0000","last_change":"2024-09-10T22:00:30.822561+0000","last_active":"2024-09-10T22:01:49.788840+0000","last_peered":"2024-09-10T22:01:49.788840+0000","last_clean":"2024-09-10T22:01:49.788840+0000","last_became_active":"2024-09-10T22:00:30.822201+0000","last_became_peered":"2024-09-10T22:00:30.822201+0000","last_unstale":"2024-09-10T22:01:49.788840+0000","last_undegraded":"2024-09-10T22:01:49.788840+0000","last_fullsized":"2024-09-10T22:01:49.788840+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:41.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57254,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":43,"num_read_kb":77,"num_write":33,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.445101+0000","last_change":"2024-09-10T22:01:35.444988+0000","last_active":"2024-09-10T22:01:35.445101+0000","last_peered":"2024-09-10T22:01:35.445101+0000","last_clean":"2024-09-10T22:01:35.445101+0000","last_became_active":"2024-09-10T22:01:35.444498+0000","last_became_peered":"2024-09-10T22:01:35.444498+0000","last_unstale":"2024-09-10T22:01:35.445101+0000","last_undegraded":"2024-09-10T22:01:35.445101+0000","last_fullsized":"2024-09-10T22:01:35.445101+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:35.017660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877143+0000","last_change":"2024-09-10T22:00:34.503234+0000","last_active":"2024-09-10T22:01:35.877143+0000","last_peered":"2024-09-10T22:01:35.877143+0000","last_clean":"2024-09-10T22:01:35.877143+0000","last_became_active":"2024-09-10T22:00:34.501501+0000","last_became_peered":"2024-09-10T22:00:34.501501+0000","last_unstale":"2024-09-10T22:01:35.877143+0000","last_undegraded":"2024-09-10T22:01:35.877143+0000","last_fullsized":"2024-09-10T22:01:35.877143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:23.772491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443988+0000","last_change":"2024-09-10T22:00:31.082131+0000","last_active":"2024-09-10T22:01:35.443988+0000","last_peered":"2024-09-10T22:01:35.443988+0000","last_clean":"2024-09-10T22:01:35.443988+0000","last_became_active":"2024-09-10T22:00:31.081405+0000","last_became_peered":"2024-09-10T22:00:31.081405+0000","last_unstale":"2024-09-10T22:01:35.443988+0000","last_undegraded":"2024-09-10T22:01:35.443988+0000","last_fullsized":"2024-09-10T22:01:35.443988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:02.400004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.191177+0000","last_change":"2024-09-10T22:00:43.705732+0000","last_active":"2024-09-10T22:01:35.191177+0000","last_peered":"2024-09-10T22:01:35.191177+0000","last_clean":"2024-09-10T22:01:35.191177+0000","last_became_active":"2024-09-10T22:00:43.705274+0000","last_became_peered":"2024-09-10T22:00:43.705274+0000","last_unstale":"2024-09-10T22:01:35.191177+0000","last_undegraded":"2024-09-10T22:01:35.191177+0000","last_fullsized":"2024-09-10T22:01:35.191177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:54.707636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189696+0000","last_change":"2024-09-10T22:00:34.499585+0000","last_active":"2024-09-10T22:01:35.189696+0000","last_peered":"2024-09-10T22:01:35.189696+0000","last_clean":"2024-09-10T22:01:35.189696+0000","last_became_active":"2024-09-10T22:00:34.499261+0000","last_became_peered":"2024-09-10T22:00:34.499261+0000","last_unstale":"2024-09-10T22:01:35.189696+0000","last_undegraded":"2024-09-10T22:01:35.189696+0000","last_fullsized":"2024-09-10T22:01:35.189696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:41.156093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194845+0000","last_change":"2024-09-10T22:00:30.823322+0000","last_active":"2024-09-10T22:01:35.194845+0000","last_peered":"2024-09-10T22:01:35.194845+0000","last_clean":"2024-09-10T22:01:35.194845+0000","last_became_active":"2024-09-10T22:00:30.822987+0000","last_became_peered":"2024-09-10T22:00:30.822987+0000","last_unstale":"2024-09-10T22:01:35.194845+0000","last_undegraded":"2024-09-10T22:01:35.194845+0000","last_fullsized":"2024-09-10T22:01:35.194845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:33.605627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595416+0000","last_change":"2024-09-10T22:00:43.708746+0000","last_active":"2024-09-10T22:01:35.595416+0000","last_peered":"2024-09-10T22:01:35.595416+0000","last_clean":"2024-09-10T22:01:35.595416+0000","last_became_active":"2024-09-10T22:00:43.708283+0000","last_became_peered":"2024-09-10T22:00:43.708283+0000","last_unstale":"2024-09-10T22:01:35.595416+0000","last_undegraded":"2024-09-10T22:01:35.595416+0000","last_fullsized":"2024-09-10T22:01:35.595416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:30.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877356+0000","last_change":"2024-09-10T22:00:34.501472+0000","last_active":"2024-09-10T22:01:35.877356+0000","last_peered":"2024-09-10T22:01:35.877356+0000","last_clean":"2024-09-10T22:01:35.877356+0000","last_became_active":"2024-09-10T22:00:34.501131+0000","last_became_peered":"2024-09-10T22:00:34.501131+0000","last_unstale":"2024-09-10T22:01:35.877356+0000","last_undegraded":"2024-09-10T22:01:35.877356+0000","last_fullsized":"2024-09-10T22:01:35.877356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:11.281413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":79,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:49.780900+0000","last_change":"2024-09-10T22:00:30.828335+0000","last_active":"2024-09-10T22:01:49.780900+0000","last_peered":"2024-09-10T22:01:49.780900+0000","last_clean":"2024-09-10T22:01:49.780900+0000","last_became_active":"2024-09-10T22:00:30.828028+0000","last_became_peered":"2024-09-10T22:00:30.828028+0000","last_unstale":"2024-09-10T22:01:49.780900+0000","last_undegraded":"2024-09-10T22:01:49.780900+0000","last_fullsized":"2024-09-10T22:01:49.780900+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:56.399002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618333+0000","last_change":"2024-09-10T22:00:43.710086+0000","last_active":"2024-09-10T22:01:35.618333+0000","last_peered":"2024-09-10T22:01:35.618333+0000","last_clean":"2024-09-10T22:01:35.618333+0000","last_became_active":"2024-09-10T22:00:43.709518+0000","last_became_peered":"2024-09-10T22:00:43.709518+0000","last_unstale":"2024-09-10T22:01:35.618333+0000","last_undegraded":"2024-09-10T22:01:35.618333+0000","last_fullsized":"2024-09-10T22:01:35.618333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:07.636887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189263+0000","last_change":"2024-09-10T22:00:43.711620+0000","last_active":"2024-09-10T22:01:35.189263+0000","last_peered":"2024-09-10T22:01:35.189263+0000","last_clean":"2024-09-10T22:01:35.189263+0000","last_became_active":"2024-09-10T22:00:43.711224+0000","last_became_peered":"2024-09-10T22:00:43.711224+0000","last_unstale":"2024-09-10T22:01:35.189263+0000","last_undegraded":"2024-09-10T22:01:35.189263+0000","last_fullsized":"2024-09-10T22:01:35.189263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:43.860286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595581+0000","last_change":"2024-09-10T22:00:34.494408+0000","last_active":"2024-09-10T22:01:35.595581+0000","last_peered":"2024-09-10T22:01:35.595581+0000","last_clean":"2024-09-10T22:01:35.595581+0000","last_became_active":"2024-09-10T22:00:34.494066+0000","last_became_peered":"2024-09-10T22:00:34.494066+0000","last_unstale":"2024-09-10T22:01:35.595581+0000","last_undegraded":"2024-09-10T22:01:35.595581+0000","last_fullsized":"2024-09-10T22:01:35.595581+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:49.565755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443303+0000","last_change":"2024-09-10T22:00:30.821842+0000","last_active":"2024-09-10T22:01:35.443303+0000","last_peered":"2024-09-10T22:01:35.443303+0000","last_clean":"2024-09-10T22:01:35.443303+0000","last_became_active":"2024-09-10T22:00:30.821499+0000","last_became_peered":"2024-09-10T22:00:30.821499+0000","last_unstale":"2024-09-10T22:01:35.443303+0000","last_undegraded":"2024-09-10T22:01:35.443303+0000","last_fullsized":"2024-09-10T22:01:35.443303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:19.125052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195492+0000","last_change":"2024-09-10T22:00:43.706689+0000","last_active":"2024-09-10T22:01:35.195492+0000","last_peered":"2024-09-10T22:01:35.195492+0000","last_clean":"2024-09-10T22:01:35.195492+0000","last_became_active":"2024-09-10T22:00:43.705954+0000","last_became_peered":"2024-09-10T22:00:43.705954+0000","last_unstale":"2024-09-10T22:01:35.195492+0000","last_undegraded":"2024-09-10T22:01:35.195492+0000","last_fullsized":"2024-09-10T22:01:35.195492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:30.800710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618231+0000","last_change":"2024-09-10T22:00:34.490853+0000","last_active":"2024-09-10T22:01:35.618231+0000","last_peered":"2024-09-10T22:01:35.618231+0000","last_clean":"2024-09-10T22:01:35.618231+0000","last_became_active":"2024-09-10T22:00:34.489908+0000","last_became_peered":"2024-09-10T22:00:34.489908+0000","last_unstale":"2024-09-10T22:01:35.618231+0000","last_undegraded":"2024-09-10T22:01:35.618231+0000","last_fullsized":"2024-09-10T22:01:35.618231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:57.105004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749247+0000","last_change":"2024-09-10T22:00:30.816497+0000","last_active":"2024-09-10T22:01:35.749247+0000","last_peered":"2024-09-10T22:01:35.749247+0000","last_clean":"2024-09-10T22:01:35.749247+0000","last_became_active":"2024-09-10T22:00:30.816181+0000","last_became_peered":"2024-09-10T22:00:30.816181+0000","last_unstale":"2024-09-10T22:01:35.749247+0000","last_undegraded":"2024-09-10T22:01:35.749247+0000","last_fullsized":"2024-09-10T22:01:35.749247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:12.779113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193229+0000","last_change":"2024-09-10T22:00:30.821402+0000","last_active":"2024-09-10T22:01:35.193229+0000","last_peered":"2024-09-10T22:01:35.193229+0000","last_clean":"2024-09-10T22:01:35.193229+0000","last_became_active":"2024-09-10T22:00:30.821105+0000","last_became_peered":"2024-09-10T22:00:30.821105+0000","last_unstale":"2024-09-10T22:01:35.193229+0000","last_undegraded":"2024-09-10T22:01:35.193229+0000","last_fullsized":"2024-09-10T22:01:35.193229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:36.090579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443700+0000","last_change":"2024-09-10T22:00:34.511911+0000","last_active":"2024-09-10T22:01:35.443700+0000","last_peered":"2024-09-10T22:01:35.443700+0000","last_clean":"2024-09-10T22:01:35.443700+0000","last_became_active":"2024-09-10T22:00:34.511545+0000","last_became_peered":"2024-09-10T22:00:34.511545+0000","last_unstale":"2024-09-10T22:01:35.443700+0000","last_undegraded":"2024-09-10T22:01:35.443700+0000","last_fullsized":"2024-09-10T22:01:35.443700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:12.484733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.750618+0000","last_change":"2024-09-10T22:01:35.750458+0000","last_active":"2024-09-10T22:01:35.750618+0000","last_peered":"2024-09-10T22:01:35.750618+0000","last_clean":"2024-09-10T22:01:35.750618+0000","last_became_active":"2024-09-10T22:01:35.749946+0000","last_became_peered":"2024-09-10T22:01:35.749946+0000","last_unstale":"2024-09-10T22:01:35.750618+0000","last_undegraded":"2024-09-10T22:01:35.750618+0000","last_fullsized":"2024-09-10T22:01:35.750618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.362221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:50.783379+0000","last_change":"2024-09-10T22:00:30.826427+0000","last_active":"2024-09-10T22:01:50.783379+0000","last_peered":"2024-09-10T22:01:50.783379+0000","last_clean":"2024-09-10T22:01:50.783379+0000","last_became_active":"2024-09-10T22:00:30.826038+0000","last_became_peered":"2024-09-10T22:00:30.826038+0000","last_unstale":"2024-09-10T22:01:50.783379+0000","last_undegraded":"2024-09-10T22:01:50.783379+0000","last_fullsized":"2024-09-10T22:01:50.783379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:45.075208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189516+0000","last_change":"2024-09-10T22:00:34.494479+0000","last_active":"2024-09-10T22:01:35.189516+0000","last_peered":"2024-09-10T22:01:35.189516+0000","last_clean":"2024-09-10T22:01:35.189516+0000","last_became_active":"2024-09-10T22:00:34.494087+0000","last_became_peered":"2024-09-10T22:00:34.494087+0000","last_unstale":"2024-09-10T22:01:35.189516+0000","last_undegraded":"2024-09-10T22:01:35.189516+0000","last_fullsized":"2024-09-10T22:01:35.189516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:28.548999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195923+0000","last_change":"2024-09-10T22:00:43.714568+0000","last_active":"2024-09-10T22:01:35.195923+0000","last_peered":"2024-09-10T22:01:35.195923+0000","last_clean":"2024-09-10T22:01:35.195923+0000","last_became_active":"2024-09-10T22:00:43.712923+0000","last_became_peered":"2024-09-10T22:00:43.712923+0000","last_unstale":"2024-09-10T22:01:35.195923+0000","last_undegraded":"2024-09-10T22:01:35.195923+0000","last_fullsized":"2024-09-10T22:01:35.195923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:27.945799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193388+0000","last_change":"2024-09-10T22:00:34.496942+0000","last_active":"2024-09-10T22:01:35.193388+0000","last_peered":"2024-09-10T22:01:35.193388+0000","last_clean":"2024-09-10T22:01:35.193388+0000","last_became_active":"2024-09-10T22:00:34.496475+0000","last_became_peered":"2024-09-10T22:00:34.496475+0000","last_unstale":"2024-09-10T22:01:35.193388+0000","last_undegraded":"2024-09-10T22:01:35.193388+0000","last_fullsized":"2024-09-10T22:01:35.193388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:33.277346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193991+0000","last_change":"2024-09-10T22:00:34.506571+0000","last_active":"2024-09-10T22:01:35.193991+0000","last_peered":"2024-09-10T22:01:35.193991+0000","last_clean":"2024-09-10T22:01:35.193991+0000","last_became_active":"2024-09-10T22:00:34.505718+0000","last_became_peered":"2024-09-10T22:00:34.505718+0000","last_unstale":"2024-09-10T22:01:35.193991+0000","last_undegraded":"2024-09-10T22:01:35.193991+0000","last_fullsized":"2024-09-10T22:01:35.193991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:10.111415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194096+0000","last_change":"2024-09-10T22:00:34.502030+0000","last_active":"2024-09-10T22:01:35.194096+0000","last_peered":"2024-09-10T22:01:35.194096+0000","last_clean":"2024-09-10T22:01:35.194096+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:01:35.194096+0000","last_undegraded":"2024-09-10T22:01:35.194096+0000","last_fullsized":"2024-09-10T22:01:35.194096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:09.532523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189536+0000","last_change":"2024-09-10T22:00:34.513630+0000","last_active":"2024-09-10T22:01:35.189536+0000","last_peered":"2024-09-10T22:01:35.189536+0000","last_clean":"2024-09-10T22:01:35.189536+0000","last_became_active":"2024-09-10T22:00:34.513348+0000","last_became_peered":"2024-09-10T22:00:34.513348+0000","last_unstale":"2024-09-10T22:01:35.189536+0000","last_undegraded":"2024-09-10T22:01:35.189536+0000","last_fullsized":"2024-09-10T22:01:35.189536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:53.942004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749744+0000","last_change":"2024-09-10T22:00:34.508697+0000","last_active":"2024-09-10T22:01:35.749744+0000","last_peered":"2024-09-10T22:01:35.749744+0000","last_clean":"2024-09-10T22:01:35.749744+0000","last_became_active":"2024-09-10T22:00:34.508371+0000","last_became_peered":"2024-09-10T22:00:34.508371+0000","last_unstale":"2024-09-10T22:01:35.749744+0000","last_undegraded":"2024-09-10T22:01:35.749744+0000","last_fullsized":"2024-09-10T22:01:35.749744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:32.843517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.196494+0000","last_change":"2024-09-10T22:00:34.496021+0000","last_active":"2024-09-10T22:01:35.196494+0000","last_peered":"2024-09-10T22:01:35.196494+0000","last_clean":"2024-09-10T22:01:35.196494+0000","last_became_active":"2024-09-10T22:00:34.495663+0000","last_became_peered":"2024-09-10T22:00:34.495663+0000","last_unstale":"2024-09-10T22:01:35.196494+0000","last_undegraded":"2024-09-10T22:01:35.196494+0000","last_fullsized":"2024-09-10T22:01:35.196494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:16.512182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.196183+0000","last_change":"2024-09-10T22:00:34.506589+0000","last_active":"2024-09-10T22:01:35.196183+0000","last_peered":"2024-09-10T22:01:35.196183+0000","last_clean":"2024-09-10T22:01:35.196183+0000","last_became_active":"2024-09-10T22:00:34.506214+0000","last_became_peered":"2024-09-10T22:00:34.506214+0000","last_unstale":"2024-09-10T22:01:35.196183+0000","last_undegraded":"2024-09-10T22:01:35.196183+0000","last_fullsized":"2024-09-10T22:01:35.196183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:47.072458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618891+0000","last_change":"2024-09-10T22:00:34.503471+0000","last_active":"2024-09-10T22:01:35.618891+0000","last_peered":"2024-09-10T22:01:35.618891+0000","last_clean":"2024-09-10T22:01:35.618891+0000","last_became_active":"2024-09-10T22:00:34.501145+0000","last_became_peered":"2024-09-10T22:00:34.501145+0000","last_unstale":"2024-09-10T22:01:35.618891+0000","last_undegraded":"2024-09-10T22:01:35.618891+0000","last_fullsized":"2024-09-10T22:01:35.618891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:24.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443033+0000","last_change":"2024-09-10T22:00:34.490177+0000","last_active":"2024-09-10T22:01:35.443033+0000","last_peered":"2024-09-10T22:01:35.443033+0000","last_clean":"2024-09-10T22:01:35.443033+0000","last_became_active":"2024-09-10T22:00:34.488428+0000","last_became_peered":"2024-09-10T22:00:34.488428+0000","last_unstale":"2024-09-10T22:01:35.443033+0000","last_undegraded":"2024-09-10T22:01:35.443033+0000","last_fullsized":"2024-09-10T22:01:35.443033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:21.111826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618325+0000","last_change":"2024-09-10T22:00:34.499174+0000","last_active":"2024-09-10T22:01:35.618325+0000","last_peered":"2024-09-10T22:01:35.618325+0000","last_clean":"2024-09-10T22:01:35.618325+0000","last_became_active":"2024-09-10T22:00:34.498824+0000","last_became_peered":"2024-09-10T22:00:34.498824+0000","last_unstale":"2024-09-10T22:01:35.618325+0000","last_undegraded":"2024-09-10T22:01:35.618325+0000","last_fullsized":"2024-09-10T22:01:35.618325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:15.946270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877187+0000","last_change":"2024-09-10T22:00:34.505748+0000","last_active":"2024-09-10T22:01:35.877187+0000","last_peered":"2024-09-10T22:01:35.877187+0000","last_clean":"2024-09-10T22:01:35.877187+0000","last_became_active":"2024-09-10T22:00:34.500441+0000","last_became_peered":"2024-09-10T22:00:34.500441+0000","last_unstale":"2024-09-10T22:01:35.877187+0000","last_undegraded":"2024-09-10T22:01:35.877187+0000","last_fullsized":"2024-09-10T22:01:35.877187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:22.177566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749624+0000","last_change":"2024-09-10T22:00:34.502319+0000","last_active":"2024-09-10T22:01:35.749624+0000","last_peered":"2024-09-10T22:01:35.749624+0000","last_clean":"2024-09-10T22:01:35.749624+0000","last_became_active":"2024-09-10T22:00:34.501427+0000","last_became_peered":"2024-09-10T22:00:34.501427+0000","last_unstale":"2024-09-10T22:01:35.749624+0000","last_undegraded":"2024-09-10T22:01:35.749624+0000","last_fullsized":"2024-09-10T22:01:35.749624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:18.463319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193065+0000","last_change":"2024-09-10T22:00:34.497571+0000","last_active":"2024-09-10T22:01:35.193065+0000","last_peered":"2024-09-10T22:01:35.193065+0000","last_clean":"2024-09-10T22:01:35.193065+0000","last_became_active":"2024-09-10T22:00:34.497205+0000","last_became_peered":"2024-09-10T22:00:34.497205+0000","last_unstale":"2024-09-10T22:01:35.193065+0000","last_undegraded":"2024-09-10T22:01:35.193065+0000","last_fullsized":"2024-09-10T22:01:35.193065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:49.924518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.196020+0000","last_change":"2024-09-10T22:00:34.504641+0000","last_active":"2024-09-10T22:01:35.196020+0000","last_peered":"2024-09-10T22:01:35.196020+0000","last_clean":"2024-09-10T22:01:35.196020+0000","last_became_active":"2024-09-10T22:00:34.504183+0000","last_became_peered":"2024-09-10T22:00:34.504183+0000","last_unstale":"2024-09-10T22:01:35.196020+0000","last_undegraded":"2024-09-10T22:01:35.196020+0000","last_fullsized":"2024-09-10T22:01:35.196020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:06.704972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595962+0000","last_change":"2024-09-10T22:00:34.503884+0000","last_active":"2024-09-10T22:01:35.595962+0000","last_peered":"2024-09-10T22:01:35.595962+0000","last_clean":"2024-09-10T22:01:35.595962+0000","last_became_active":"2024-09-10T22:00:34.502950+0000","last_became_peered":"2024-09-10T22:00:34.502950+0000","last_unstale":"2024-09-10T22:01:35.595962+0000","last_undegraded":"2024-09-10T22:01:35.595962+0000","last_fullsized":"2024-09-10T22:01:35.595962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:45.560958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195749+0000","last_change":"2024-09-10T22:00:34.504981+0000","last_active":"2024-09-10T22:01:35.195749+0000","last_peered":"2024-09-10T22:01:35.195749+0000","last_clean":"2024-09-10T22:01:35.195749+0000","last_became_active":"2024-09-10T22:00:34.500474+0000","last_became_peered":"2024-09-10T22:00:34.500474+0000","last_unstale":"2024-09-10T22:01:35.195749+0000","last_undegraded":"2024-09-10T22:01:35.195749+0000","last_fullsized":"2024-09-10T22:01:35.195749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:20.892605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877526+0000","last_change":"2024-09-10T22:00:34.506650+0000","last_active":"2024-09-10T22:01:35.877526+0000","last_peered":"2024-09-10T22:01:35.877526+0000","last_clean":"2024-09-10T22:01:35.877526+0000","last_became_active":"2024-09-10T22:00:34.505698+0000","last_became_peered":"2024-09-10T22:00:34.505698+0000","last_unstale":"2024-09-10T22:01:35.877526+0000","last_undegraded":"2024-09-10T22:01:35.877526+0000","last_fullsized":"2024-09-10T22:01:35.877526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:29.221693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189132+0000","last_change":"2024-09-10T22:00:34.502238+0000","last_active":"2024-09-10T22:01:35.189132+0000","last_peered":"2024-09-10T22:01:35.189132+0000","last_clean":"2024-09-10T22:01:35.189132+0000","last_became_active":"2024-09-10T22:00:34.500049+0000","last_became_peered":"2024-09-10T22:00:34.500049+0000","last_unstale":"2024-09-10T22:01:35.189132+0000","last_undegraded":"2024-09-10T22:01:35.189132+0000","last_fullsized":"2024-09-10T22:01:35.189132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:05.669784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189132+0000","last_change":"2024-09-10T22:00:34.499340+0000","last_active":"2024-09-10T22:01:35.189132+0000","last_peered":"2024-09-10T22:01:35.189132+0000","last_clean":"2024-09-10T22:01:35.189132+0000","last_became_active":"2024-09-10T22:00:34.497655+0000","last_became_peered":"2024-09-10T22:00:34.497655+0000","last_unstale":"2024-09-10T22:01:35.189132+0000","last_undegraded":"2024-09-10T22:01:35.189132+0000","last_fullsized":"2024-09-10T22:01:35.189132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:34.115829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749525+0000","last_change":"2024-09-10T22:00:34.500705+0000","last_active":"2024-09-10T22:01:35.749525+0000","last_peered":"2024-09-10T22:01:35.749525+0000","last_clean":"2024-09-10T22:01:35.749525+0000","last_became_active":"2024-09-10T22:00:34.500071+0000","last_became_peered":"2024-09-10T22:00:34.500071+0000","last_unstale":"2024-09-10T22:01:35.749525+0000","last_undegraded":"2024-09-10T22:01:35.749525+0000","last_fullsized":"2024-09-10T22:01:35.749525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:40.519976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190179+0000","last_change":"2024-09-10T22:00:34.510570+0000","last_active":"2024-09-10T22:01:35.190179+0000","last_peered":"2024-09-10T22:01:35.190179+0000","last_clean":"2024-09-10T22:01:35.190179+0000","last_became_active":"2024-09-10T22:00:34.506922+0000","last_became_peered":"2024-09-10T22:00:34.506922+0000","last_unstale":"2024-09-10T22:01:35.190179+0000","last_undegraded":"2024-09-10T22:01:35.190179+0000","last_fullsized":"2024-09-10T22:01:35.190179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.489031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195114+0000","last_change":"2024-09-10T22:00:34.509985+0000","last_active":"2024-09-10T22:01:35.195114+0000","last_peered":"2024-09-10T22:01:35.195114+0000","last_clean":"2024-09-10T22:01:35.195114+0000","last_became_active":"2024-09-10T22:00:34.509527+0000","last_became_peered":"2024-09-10T22:00:34.509527+0000","last_unstale":"2024-09-10T22:01:35.195114+0000","last_undegraded":"2024-09-10T22:01:35.195114+0000","last_fullsized":"2024-09-10T22:01:35.195114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:49.463334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193894+0000","last_change":"2024-09-10T22:00:34.497986+0000","last_active":"2024-09-10T22:01:35.193894+0000","last_peered":"2024-09-10T22:01:35.193894+0000","last_clean":"2024-09-10T22:01:35.193894+0000","last_became_active":"2024-09-10T22:00:34.497612+0000","last_became_peered":"2024-09-10T22:00:34.497612+0000","last_unstale":"2024-09-10T22:01:35.193894+0000","last_undegraded":"2024-09-10T22:01:35.193894+0000","last_fullsized":"2024-09-10T22:01:35.193894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:15.696988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877514+0000","last_change":"2024-09-10T22:00:34.498482+0000","last_active":"2024-09-10T22:01:35.877514+0000","last_peered":"2024-09-10T22:01:35.877514+0000","last_clean":"2024-09-10T22:01:35.877514+0000","last_became_active":"2024-09-10T22:00:34.498139+0000","last_became_peered":"2024-09-10T22:00:34.498139+0000","last_unstale":"2024-09-10T22:01:35.877514+0000","last_undegraded":"2024-09-10T22:01:35.877514+0000","last_fullsized":"2024-09-10T22:01:35.877514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:23.427253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618225+0000","last_change":"2024-09-10T22:00:34.491325+0000","last_active":"2024-09-10T22:01:35.618225+0000","last_peered":"2024-09-10T22:01:35.618225+0000","last_clean":"2024-09-10T22:01:35.618225+0000","last_became_active":"2024-09-10T22:00:34.491026+0000","last_became_peered":"2024-09-10T22:00:34.491026+0000","last_unstale":"2024-09-10T22:01:35.618225+0000","last_undegraded":"2024-09-10T22:01:35.618225+0000","last_fullsized":"2024-09-10T22:01:35.618225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:07.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194822+0000","last_change":"2024-09-10T22:00:34.503975+0000","last_active":"2024-09-10T22:01:35.194822+0000","last_peered":"2024-09-10T22:01:35.194822+0000","last_clean":"2024-09-10T22:01:35.194822+0000","last_became_active":"2024-09-10T22:00:34.503624+0000","last_became_peered":"2024-09-10T22:00:34.503624+0000","last_unstale":"2024-09-10T22:01:35.194822+0000","last_undegraded":"2024-09-10T22:01:35.194822+0000","last_fullsized":"2024-09-10T22:01:35.194822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:17.678811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.596879+0000","last_change":"2024-09-10T22:01:35.596824+0000","last_active":"2024-09-10T22:01:35.596879+0000","last_peered":"2024-09-10T22:01:35.596879+0000","last_clean":"2024-09-10T22:01:35.596879+0000","last_became_active":"2024-09-10T22:01:35.596550+0000","last_became_peered":"2024-09-10T22:01:35.596550+0000","last_unstale":"2024-09-10T22:01:35.596879+0000","last_undegraded":"2024-09-10T22:01:35.596879+0000","last_fullsized":"2024-09-10T22:01:35.596879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:19.617649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189287+0000","last_change":"2024-09-10T22:00:34.501647+0000","last_active":"2024-09-10T22:01:35.189287+0000","last_peered":"2024-09-10T22:01:35.189287+0000","last_clean":"2024-09-10T22:01:35.189287+0000","last_became_active":"2024-09-10T22:00:34.499216+0000","last_became_peered":"2024-09-10T22:00:34.499216+0000","last_unstale":"2024-09-10T22:01:35.189287+0000","last_undegraded":"2024-09-10T22:01:35.189287+0000","last_fullsized":"2024-09-10T22:01:35.189287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:41:08.395469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190176+0000","last_change":"2024-09-10T22:00:34.513134+0000","last_active":"2024-09-10T22:01:35.190176+0000","last_peered":"2024-09-10T22:01:35.190176+0000","last_clean":"2024-09-10T22:01:35.190176+0000","last_became_active":"2024-09-10T22:00:34.507578+0000","last_became_peered":"2024-09-10T22:00:34.507578+0000","last_unstale":"2024-09-10T22:01:35.190176+0000","last_undegraded":"2024-09-10T22:01:35.190176+0000","last_fullsized":"2024-09-10T22:01:35.190176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:33.387337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189178+0000","last_change":"2024-09-10T22:00:34.507645+0000","last_active":"2024-09-10T22:01:35.189178+0000","last_peered":"2024-09-10T22:01:35.189178+0000","last_clean":"2024-09-10T22:01:35.189178+0000","last_became_active":"2024-09-10T22:00:34.503614+0000","last_became_peered":"2024-09-10T22:00:34.503614+0000","last_unstale":"2024-09-10T22:01:35.189178+0000","last_undegraded":"2024-09-10T22:01:35.189178+0000","last_fullsized":"2024-09-10T22:01:35.189178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:53.033523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189362+0000","last_change":"2024-09-10T22:00:34.501891+0000","last_active":"2024-09-10T22:01:35.189362+0000","last_peered":"2024-09-10T22:01:35.189362+0000","last_clean":"2024-09-10T22:01:35.189362+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:01:35.189362+0000","last_undegraded":"2024-09-10T22:01:35.189362+0000","last_fullsized":"2024-09-10T22:01:35.189362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:30.068461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195722+0000","last_change":"2024-09-10T22:00:34.501475+0000","last_active":"2024-09-10T22:01:35.195722+0000","last_peered":"2024-09-10T22:01:35.195722+0000","last_clean":"2024-09-10T22:01:35.195722+0000","last_became_active":"2024-09-10T22:00:34.500909+0000","last_became_peered":"2024-09-10T22:00:34.500909+0000","last_unstale":"2024-09-10T22:01:35.195722+0000","last_undegraded":"2024-09-10T22:01:35.195722+0000","last_fullsized":"2024-09-10T22:01:35.195722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:39.616773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65240,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":219,"num_read_kb":180,"num_write":244,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":195720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246272,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678046,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":516912,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874275,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1160870,"data_compressed":62485,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037802,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":520435,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234033,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":516605,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430263,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":1113996,"data_compressed":62485,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":512234,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1108796,"data_compressed":62485,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":516308,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":511551,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378710,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":558242,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542237,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":570630,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705762,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":513690,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":59509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":59896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":59381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:01:55.352 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:01:55.737 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:55 smithi090 ceph-mon[30350]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:01:55.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:55 smithi171 ceph-mon[34138]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:01:56.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:55 smithi154 ceph-mon[34495]: from='client.24868 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:01:56.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.345+0000 7fc774400640 1 -- 172.21.15.90:0/3558163864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c102140 msgr2=0x7fc76c10ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:56.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.345+0000 7fc774400640 1 --2- 172.21.15.90:0/3558163864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c102140 0x7fc76c10ee90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc758009900 tx=0x7fc75802a410 comp rx=0 tx=0).stop 2024-09-10T22:01:56.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.346+0000 7fc774400640 1 -- 172.21.15.90:0/3558163864 shutdown_connections 2024-09-10T22:01:56.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.346+0000 7fc774400640 1 --2- 172.21.15.90:0/3558163864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c10f3d0 0x7fc76c1117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.346+0000 7fc774400640 1 --2- 172.21.15.90:0/3558163864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c102140 0x7fc76c10ee90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.346+0000 7fc774400640 1 --2- 172.21.15.90:0/3558163864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc76c101820 0x7fc76c101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.346+0000 7fc774400640 1 -- 172.21.15.90:0/3558163864 >> 172.21.15.90:0/3558163864 conn(0x7fc76c0fd710 msgr2=0x7fc76c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:56.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.346+0000 7fc774400640 1 -- 172.21.15.90:0/3558163864 shutdown_connections 2024-09-10T22:01:56.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.346+0000 7fc774400640 1 -- 172.21.15.90:0/3558163864 wait complete. 2024-09-10T22:01:56.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc774400640 1 Processor -- start 2024-09-10T22:01:56.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc774400640 1 -- start start 2024-09-10T22:01:56.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc774400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c101820 0x7fc76c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:56.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc774400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc76c102140 0x7fc76c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:56.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc774400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c10f3d0 0x7fc76c1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:56.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc774400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c114e20 con 0x7fc76c101820 2024-09-10T22:01:56.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc774400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c114ca0 con 0x7fc76c102140 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc774400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c114fa0 con 0x7fc76c10f3d0 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc771a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c101820 0x7fc76c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc771a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c101820 0x7fc76c19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48422/0 (socket says 172.21.15.90:48422) 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.347+0000 7fc771a00640 1 -- 172.21.15.90:0/3368941960 learned_addr learned my addr 172.21.15.90:0/3368941960 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc771000640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc76c102140 0x7fc76c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc772400640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c10f3d0 0x7fc76c1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc771a00640 1 -- 172.21.15.90:0/3368941960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c10f3d0 msgr2=0x7fc76c1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc771a00640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c10f3d0 0x7fc76c1a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc771a00640 1 -- 172.21.15.90:0/3368941960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc76c102140 msgr2=0x7fc76c19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc771a00640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc76c102140 0x7fc76c19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc771a00640 1 -- 172.21.15.90:0/3368941960 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c1a1440 con 0x7fc76c101820 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc772400640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c10f3d0 0x7fc76c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:01:56.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc771a00640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c101820 0x7fc76c19b390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc76000e9f0 tx=0x7fc76000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:56.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc76a000640 1 -- 172.21.15.90:0/3368941960 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76000cd80 con 0x7fc76c101820 2024-09-10T22:01:56.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.348+0000 7fc76a000640 1 -- 172.21.15.90:0/3368941960 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc7600108c0 con 0x7fc76c101820 2024-09-10T22:01:56.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.349+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c1a1730 con 0x7fc76c101820 2024-09-10T22:01:56.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.349+0000 7fc76a000640 1 -- 172.21.15.90:0/3368941960 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7600187c0 con 0x7fc76c101820 2024-09-10T22:01:56.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.349+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c1a7df0 con 0x7fc76c101820 2024-09-10T22:01:56.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.350+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c101c00 con 0x7fc76c101820 2024-09-10T22:01:56.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.352+0000 7fc76a000640 1 -- 172.21.15.90:0/3368941960 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fc760010430 con 0x7fc76c101820 2024-09-10T22:01:56.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.353+0000 7fc76a000640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc7540b53c0 0x7fc7540b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:56.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.354+0000 7fc76a000640 1 -- 172.21.15.90:0/3368941960 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fc7600e6700 con 0x7fc76c101820 2024-09-10T22:01:56.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.354+0000 7fc771000640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc7540b53c0 0x7fc7540b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:56.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.354+0000 7fc771000640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc7540b53c0 0x7fc7540b7880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc758006010 tx=0x7fc758005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:56.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.357+0000 7fc76a000640 1 -- 172.21.15.90:0/3368941960 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc7600acfe0 con 0x7fc76c101820 2024-09-10T22:01:56.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.539+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc76c110770 con 0x7fc7540b53c0 2024-09-10T22:01:56.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.548+0000 7fc76a000640 1 -- 172.21.15.90:0/3368941960 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340656 (secure 0 0 0) 0x7fc76c110770 con 0x7fc7540b53c0 2024-09-10T22:01:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:01:56.555 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-09-10T22:01:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.553+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc7540b53c0 msgr2=0x7fc7540b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.553+0000 7fc774400640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc7540b53c0 0x7fc7540b7880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc758006010 tx=0x7fc758005a20 comp rx=0 tx=0).stop 2024-09-10T22:01:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.553+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c101820 msgr2=0x7fc76c19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:56.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.553+0000 7fc774400640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c101820 0x7fc76c19b390 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc76000e9f0 tx=0x7fc76000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.554+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 shutdown_connections 2024-09-10T22:01:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.554+0000 7fc774400640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc7540b53c0 0x7fc7540b7880 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.554+0000 7fc774400640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc76c10f3d0 0x7fc76c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.554+0000 7fc774400640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc76c102140 0x7fc76c19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.554+0000 7fc774400640 1 --2- 172.21.15.90:0/3368941960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc76c101820 0x7fc76c19b390 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.554+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 >> 172.21.15.90:0/3368941960 conn(0x7fc76c0fd710 msgr2=0x7fc76c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.554+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 shutdown_connections 2024-09-10T22:01:56.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:56.554+0000 7fc774400640 1 -- 172.21.15.90:0/3368941960 wait complete. 2024-09-10T22:01:56.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:56 smithi090 ceph-mon[30350]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-10T22:01:56.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:56 smithi171 ceph-mon[34138]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-10T22:01:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:56 smithi154 ceph-mon[34495]: pgmap v357: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-09-10T22:01:57.690 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-10T22:01:55.599772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661245,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":336,"num_read_kb":275,"num_write":372,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":348796,"kb_used_data":12012,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124593028,"statfs":{"total":1151940427776,"available":1151583260672,"internally_reserved":0,"allocated":12300288,"data_stored":8143850,"data_compressed":426882,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38315,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":109,"num_write":24,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003425"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:51.782636+0000","last_change":"2024-09-10T22:00:30.824265+0000","last_active":"2024-09-10T22:01:51.782636+0000","last_peered":"2024-09-10T22:01:51.782636+0000","last_clean":"2024-09-10T22:01:51.782636+0000","last_became_active":"2024-09-10T22:00:30.823905+0000","last_became_peered":"2024-09-10T22:00:30.823905+0000","last_unstale":"2024-09-10T22:01:51.782636+0000","last_undegraded":"2024-09-10T22:01:51.782636+0000","last_fullsized":"2024-09-10T22:01:51.782636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:16.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618713+0000","last_change":"2024-09-10T22:00:34.497719+0000","last_active":"2024-09-10T22:01:35.618713+0000","last_peered":"2024-09-10T22:01:35.618713+0000","last_clean":"2024-09-10T22:01:35.618713+0000","last_became_active":"2024-09-10T22:00:34.497380+0000","last_became_peered":"2024-09-10T22:00:34.497380+0000","last_unstale":"2024-09-10T22:01:35.618713+0000","last_undegraded":"2024-09-10T22:01:35.618713+0000","last_fullsized":"2024-09-10T22:01:35.618713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:26.336299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193383+0000","last_change":"2024-09-10T22:00:43.710835+0000","last_active":"2024-09-10T22:01:35.193383+0000","last_peered":"2024-09-10T22:01:35.193383+0000","last_clean":"2024-09-10T22:01:35.193383+0000","last_became_active":"2024-09-10T22:00:43.710363+0000","last_became_peered":"2024-09-10T22:00:43.710363+0000","last_unstale":"2024-09-10T22:01:35.193383+0000","last_undegraded":"2024-09-10T22:01:35.193383+0000","last_fullsized":"2024-09-10T22:01:35.193383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:28.173482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618909+0000","last_change":"2024-09-10T22:00:31.083228+0000","last_active":"2024-09-10T22:01:35.618909+0000","last_peered":"2024-09-10T22:01:35.618909+0000","last_clean":"2024-09-10T22:01:35.618909+0000","last_became_active":"2024-09-10T22:00:31.082911+0000","last_became_peered":"2024-09-10T22:00:31.082911+0000","last_unstale":"2024-09-10T22:01:35.618909+0000","last_undegraded":"2024-09-10T22:01:35.618909+0000","last_fullsized":"2024-09-10T22:01:35.618909+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:28.598628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189606+0000","last_change":"2024-09-10T22:00:34.509939+0000","last_active":"2024-09-10T22:01:35.189606+0000","last_peered":"2024-09-10T22:01:35.189606+0000","last_clean":"2024-09-10T22:01:35.189606+0000","last_became_active":"2024-09-10T22:00:34.509536+0000","last_became_peered":"2024-09-10T22:00:34.509536+0000","last_unstale":"2024-09-10T22:01:35.189606+0000","last_undegraded":"2024-09-10T22:01:35.189606+0000","last_fullsized":"2024-09-10T22:01:35.189606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:27.602378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189753+0000","last_change":"2024-09-10T22:00:43.714662+0000","last_active":"2024-09-10T22:01:35.189753+0000","last_peered":"2024-09-10T22:01:35.189753+0000","last_clean":"2024-09-10T22:01:35.189753+0000","last_became_active":"2024-09-10T22:00:43.714074+0000","last_became_peered":"2024-09-10T22:00:43.714074+0000","last_unstale":"2024-09-10T22:01:35.189753+0000","last_undegraded":"2024-09-10T22:01:35.189753+0000","last_fullsized":"2024-09-10T22:01:35.189753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:41.204519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.210089+0000","last_change":"2024-09-10T22:01:35.209979+0000","last_active":"2024-09-10T22:01:35.210089+0000","last_peered":"2024-09-10T22:01:35.210089+0000","last_clean":"2024-09-10T22:01:35.210089+0000","last_became_active":"2024-09-10T22:01:35.197192+0000","last_became_peered":"2024-09-10T22:01:35.197192+0000","last_unstale":"2024-09-10T22:01:35.210089+0000","last_undegraded":"2024-09-10T22:01:35.210089+0000","last_fullsized":"2024-09-10T22:01:35.210089+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:47.707679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877107+0000","last_change":"2024-09-10T22:00:34.502795+0000","last_active":"2024-09-10T22:01:35.877107+0000","last_peered":"2024-09-10T22:01:35.877107+0000","last_clean":"2024-09-10T22:01:35.877107+0000","last_became_active":"2024-09-10T22:00:34.501112+0000","last_became_peered":"2024-09-10T22:00:34.501112+0000","last_unstale":"2024-09-10T22:01:35.877107+0000","last_undegraded":"2024-09-10T22:01:35.877107+0000","last_fullsized":"2024-09-10T22:01:35.877107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:45.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.619835+0000","last_change":"2024-09-10T22:01:35.619765+0000","last_active":"2024-09-10T22:01:35.619835+0000","last_peered":"2024-09-10T22:01:35.619835+0000","last_clean":"2024-09-10T22:01:35.619835+0000","last_became_active":"2024-09-10T22:01:35.619407+0000","last_became_peered":"2024-09-10T22:01:35.619407+0000","last_unstale":"2024-09-10T22:01:35.619835+0000","last_undegraded":"2024-09-10T22:01:35.619835+0000","last_fullsized":"2024-09-10T22:01:35.619835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:17.064585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189727+0000","last_change":"2024-09-10T22:00:30.818224+0000","last_active":"2024-09-10T22:01:35.189727+0000","last_peered":"2024-09-10T22:01:35.189727+0000","last_clean":"2024-09-10T22:01:35.189727+0000","last_became_active":"2024-09-10T22:00:30.817781+0000","last_became_peered":"2024-09-10T22:00:30.817781+0000","last_unstale":"2024-09-10T22:01:35.189727+0000","last_undegraded":"2024-09-10T22:01:35.189727+0000","last_fullsized":"2024-09-10T22:01:35.189727+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:47.097642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.444107+0000","last_change":"2024-09-10T22:00:34.499281+0000","last_active":"2024-09-10T22:01:35.444107+0000","last_peered":"2024-09-10T22:01:35.444107+0000","last_clean":"2024-09-10T22:01:35.444107+0000","last_became_active":"2024-09-10T22:00:34.498914+0000","last_became_peered":"2024-09-10T22:00:34.498914+0000","last_unstale":"2024-09-10T22:01:35.444107+0000","last_undegraded":"2024-09-10T22:01:35.444107+0000","last_fullsized":"2024-09-10T22:01:35.444107+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:23:47.838346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189862+0000","last_change":"2024-09-10T22:00:43.712066+0000","last_active":"2024-09-10T22:01:35.189862+0000","last_peered":"2024-09-10T22:01:35.189862+0000","last_clean":"2024-09-10T22:01:35.189862+0000","last_became_active":"2024-09-10T22:00:43.711614+0000","last_became_peered":"2024-09-10T22:00:43.711614+0000","last_unstale":"2024-09-10T22:01:35.189862+0000","last_undegraded":"2024-09-10T22:01:35.189862+0000","last_fullsized":"2024-09-10T22:01:35.189862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:42.403367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195148+0000","last_change":"2024-09-10T22:00:34.501943+0000","last_active":"2024-09-10T22:01:35.195148+0000","last_peered":"2024-09-10T22:01:35.195148+0000","last_clean":"2024-09-10T22:01:35.195148+0000","last_became_active":"2024-09-10T22:00:34.501602+0000","last_became_peered":"2024-09-10T22:00:34.501602+0000","last_unstale":"2024-09-10T22:01:35.195148+0000","last_undegraded":"2024-09-10T22:01:35.195148+0000","last_fullsized":"2024-09-10T22:01:35.195148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:53.200974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.200844+0000","last_change":"2024-09-10T22:01:35.200791+0000","last_active":"2024-09-10T22:01:35.200844+0000","last_peered":"2024-09-10T22:01:35.200844+0000","last_clean":"2024-09-10T22:01:35.200844+0000","last_became_active":"2024-09-10T22:01:35.196059+0000","last_became_peered":"2024-09-10T22:01:35.196059+0000","last_unstale":"2024-09-10T22:01:35.200844+0000","last_undegraded":"2024-09-10T22:01:35.200844+0000","last_fullsized":"2024-09-10T22:01:35.200844+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:51.349216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189977+0000","last_change":"2024-09-10T22:00:43.760567+0000","last_active":"2024-09-10T22:01:35.189977+0000","last_peered":"2024-09-10T22:01:35.189977+0000","last_clean":"2024-09-10T22:01:35.189977+0000","last_became_active":"2024-09-10T22:00:43.760064+0000","last_became_peered":"2024-09-10T22:00:43.760064+0000","last_unstale":"2024-09-10T22:01:35.189977+0000","last_undegraded":"2024-09-10T22:01:35.189977+0000","last_fullsized":"2024-09-10T22:01:35.189977+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:55.469525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618453+0000","last_change":"2024-09-10T22:00:34.486225+0000","last_active":"2024-09-10T22:01:35.618453+0000","last_peered":"2024-09-10T22:01:35.618453+0000","last_clean":"2024-09-10T22:01:35.618453+0000","last_became_active":"2024-09-10T22:00:34.485896+0000","last_became_peered":"2024-09-10T22:00:34.485896+0000","last_unstale":"2024-09-10T22:01:35.618453+0000","last_undegraded":"2024-09-10T22:01:35.618453+0000","last_fullsized":"2024-09-10T22:01:35.618453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:00.151927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194272+0000","last_change":"2024-09-10T22:00:30.817958+0000","last_active":"2024-09-10T22:01:35.194272+0000","last_peered":"2024-09-10T22:01:35.194272+0000","last_clean":"2024-09-10T22:01:35.194272+0000","last_became_active":"2024-09-10T22:00:30.817646+0000","last_became_peered":"2024-09-10T22:00:30.817646+0000","last_unstale":"2024-09-10T22:01:35.194272+0000","last_undegraded":"2024-09-10T22:01:35.194272+0000","last_fullsized":"2024-09-10T22:01:35.194272+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:27.722184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190037+0000","last_change":"2024-09-10T22:00:43.716465+0000","last_active":"2024-09-10T22:01:35.190037+0000","last_peered":"2024-09-10T22:01:35.190037+0000","last_clean":"2024-09-10T22:01:35.190037+0000","last_became_active":"2024-09-10T22:00:43.716165+0000","last_became_peered":"2024-09-10T22:00:43.716165+0000","last_unstale":"2024-09-10T22:01:35.190037+0000","last_undegraded":"2024-09-10T22:01:35.190037+0000","last_fullsized":"2024-09-10T22:01:35.190037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:18.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193477+0000","last_change":"2024-09-10T22:00:34.493992+0000","last_active":"2024-09-10T22:01:35.193477+0000","last_peered":"2024-09-10T22:01:35.193477+0000","last_clean":"2024-09-10T22:01:35.193477+0000","last_became_active":"2024-09-10T22:00:34.493670+0000","last_became_peered":"2024-09-10T22:00:34.493670+0000","last_unstale":"2024-09-10T22:01:35.193477+0000","last_undegraded":"2024-09-10T22:01:35.193477+0000","last_fullsized":"2024-09-10T22:01:35.193477+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:04.991819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618513+0000","last_change":"2024-09-10T22:00:30.819830+0000","last_active":"2024-09-10T22:01:35.618513+0000","last_peered":"2024-09-10T22:01:35.618513+0000","last_clean":"2024-09-10T22:01:35.618513+0000","last_became_active":"2024-09-10T22:00:30.819401+0000","last_became_peered":"2024-09-10T22:00:30.819401+0000","last_unstale":"2024-09-10T22:01:35.618513+0000","last_undegraded":"2024-09-10T22:01:35.618513+0000","last_fullsized":"2024-09-10T22:01:35.618513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:37.823107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595833+0000","last_change":"2024-09-10T22:00:43.718500+0000","last_active":"2024-09-10T22:01:35.595833+0000","last_peered":"2024-09-10T22:01:35.595833+0000","last_clean":"2024-09-10T22:01:35.595833+0000","last_became_active":"2024-09-10T22:00:43.717667+0000","last_became_peered":"2024-09-10T22:00:43.717667+0000","last_unstale":"2024-09-10T22:01:35.595833+0000","last_undegraded":"2024-09-10T22:01:35.595833+0000","last_fullsized":"2024-09-10T22:01:35.595833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:23.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189221+0000","last_change":"2024-09-10T22:00:34.494162+0000","last_active":"2024-09-10T22:01:35.189221+0000","last_peered":"2024-09-10T22:01:35.189221+0000","last_clean":"2024-09-10T22:01:35.189221+0000","last_became_active":"2024-09-10T22:00:34.493819+0000","last_became_peered":"2024-09-10T22:00:34.493819+0000","last_unstale":"2024-09-10T22:01:35.189221+0000","last_undegraded":"2024-09-10T22:01:35.189221+0000","last_fullsized":"2024-09-10T22:01:35.189221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:43.264363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877624+0000","last_change":"2024-09-10T22:00:31.081559+0000","last_active":"2024-09-10T22:01:35.877624+0000","last_peered":"2024-09-10T22:01:35.877624+0000","last_clean":"2024-09-10T22:01:35.877624+0000","last_became_active":"2024-09-10T22:00:31.081257+0000","last_became_peered":"2024-09-10T22:00:31.081257+0000","last_unstale":"2024-09-10T22:01:35.877624+0000","last_undegraded":"2024-09-10T22:01:35.877624+0000","last_fullsized":"2024-09-10T22:01:35.877624+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:53.235765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.445506+0000","last_change":"2024-09-10T22:01:35.445171+0000","last_active":"2024-09-10T22:01:35.445506+0000","last_peered":"2024-09-10T22:01:35.445506+0000","last_clean":"2024-09-10T22:01:35.445506+0000","last_became_active":"2024-09-10T22:01:35.444651+0000","last_became_peered":"2024-09-10T22:01:35.444651+0000","last_unstale":"2024-09-10T22:01:35.445506+0000","last_undegraded":"2024-09-10T22:01:35.445506+0000","last_fullsized":"2024-09-10T22:01:35.445506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:36.457151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.751094+0000","last_change":"2024-09-10T22:01:35.750982+0000","last_active":"2024-09-10T22:01:35.751094+0000","last_peered":"2024-09-10T22:01:35.751094+0000","last_clean":"2024-09-10T22:01:35.751094+0000","last_became_active":"2024-09-10T22:01:35.750572+0000","last_became_peered":"2024-09-10T22:01:35.750572+0000","last_unstale":"2024-09-10T22:01:35.751094+0000","last_undegraded":"2024-09-10T22:01:35.751094+0000","last_fullsized":"2024-09-10T22:01:35.751094+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:36.596313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190527+0000","last_change":"2024-09-10T22:00:31.311864+0000","last_active":"2024-09-10T22:01:35.190527+0000","last_peered":"2024-09-10T22:01:35.190527+0000","last_clean":"2024-09-10T22:01:35.190527+0000","last_became_active":"2024-09-10T22:00:31.311611+0000","last_became_peered":"2024-09-10T22:00:31.311611+0000","last_unstale":"2024-09-10T22:01:35.190527+0000","last_undegraded":"2024-09-10T22:01:35.190527+0000","last_fullsized":"2024-09-10T22:01:35.190527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:27.969999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193840+0000","last_change":"2024-09-10T22:00:43.708023+0000","last_active":"2024-09-10T22:01:35.193840+0000","last_peered":"2024-09-10T22:01:35.193840+0000","last_clean":"2024-09-10T22:01:35.193840+0000","last_became_active":"2024-09-10T22:00:43.707086+0000","last_became_peered":"2024-09-10T22:00:43.707086+0000","last_unstale":"2024-09-10T22:01:35.193840+0000","last_undegraded":"2024-09-10T22:01:35.193840+0000","last_fullsized":"2024-09-10T22:01:35.193840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:15.585104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618945+0000","last_change":"2024-09-10T22:00:34.494104+0000","last_active":"2024-09-10T22:01:35.618945+0000","last_peered":"2024-09-10T22:01:35.618945+0000","last_clean":"2024-09-10T22:01:35.618945+0000","last_became_active":"2024-09-10T22:00:34.493404+0000","last_became_peered":"2024-09-10T22:00:34.493404+0000","last_unstale":"2024-09-10T22:01:35.618945+0000","last_undegraded":"2024-09-10T22:01:35.618945+0000","last_fullsized":"2024-09-10T22:01:35.618945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:47.444100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'13","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:48.513188+0000","last_change":"2024-09-10T22:00:30.819427+0000","last_active":"2024-09-10T22:01:48.513188+0000","last_peered":"2024-09-10T22:01:48.513188+0000","last_clean":"2024-09-10T22:01:48.513188+0000","last_became_active":"2024-09-10T22:00:30.819098+0000","last_became_peered":"2024-09-10T22:00:30.819098+0000","last_unstale":"2024-09-10T22:01:48.513188+0000","last_undegraded":"2024-09-10T22:01:48.513188+0000","last_fullsized":"2024-09-10T22:01:48.513188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:23.068155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618087+0000","last_change":"2024-09-10T22:00:43.760602+0000","last_active":"2024-09-10T22:01:35.618087+0000","last_peered":"2024-09-10T22:01:35.618087+0000","last_clean":"2024-09-10T22:01:35.618087+0000","last_became_active":"2024-09-10T22:00:43.760172+0000","last_became_peered":"2024-09-10T22:00:43.760172+0000","last_unstale":"2024-09-10T22:01:35.618087+0000","last_undegraded":"2024-09-10T22:01:35.618087+0000","last_fullsized":"2024-09-10T22:01:35.618087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:40.332101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749946+0000","last_change":"2024-09-10T22:00:34.501632+0000","last_active":"2024-09-10T22:01:35.749946+0000","last_peered":"2024-09-10T22:01:35.749946+0000","last_clean":"2024-09-10T22:01:35.749946+0000","last_became_active":"2024-09-10T22:00:34.500924+0000","last_became_peered":"2024-09-10T22:00:34.500924+0000","last_unstale":"2024-09-10T22:01:35.749946+0000","last_undegraded":"2024-09-10T22:01:35.749946+0000","last_fullsized":"2024-09-10T22:01:35.749946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:28.052570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190019+0000","last_change":"2024-09-10T22:00:31.081391+0000","last_active":"2024-09-10T22:01:35.190019+0000","last_peered":"2024-09-10T22:01:35.190019+0000","last_clean":"2024-09-10T22:01:35.190019+0000","last_became_active":"2024-09-10T22:00:31.081018+0000","last_became_peered":"2024-09-10T22:00:31.081018+0000","last_unstale":"2024-09-10T22:01:35.190019+0000","last_undegraded":"2024-09-10T22:01:35.190019+0000","last_fullsized":"2024-09-10T22:01:35.190019+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:09.450785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193019+0000","last_change":"2024-09-10T22:00:43.711592+0000","last_active":"2024-09-10T22:01:35.193019+0000","last_peered":"2024-09-10T22:01:35.193019+0000","last_clean":"2024-09-10T22:01:35.193019+0000","last_became_active":"2024-09-10T22:00:43.711144+0000","last_became_peered":"2024-09-10T22:00:43.711144+0000","last_unstale":"2024-09-10T22:01:35.193019+0000","last_undegraded":"2024-09-10T22:01:35.193019+0000","last_fullsized":"2024-09-10T22:01:35.193019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:01.818907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.192436+0000","last_change":"2024-09-10T22:00:34.494070+0000","last_active":"2024-09-10T22:01:35.192436+0000","last_peered":"2024-09-10T22:01:35.192436+0000","last_clean":"2024-09-10T22:01:35.192436+0000","last_became_active":"2024-09-10T22:00:34.493324+0000","last_became_peered":"2024-09-10T22:00:34.493324+0000","last_unstale":"2024-09-10T22:01:35.192436+0000","last_undegraded":"2024-09-10T22:01:35.192436+0000","last_fullsized":"2024-09-10T22:01:35.192436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:29.969743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595852+0000","last_change":"2024-09-10T22:00:31.082439+0000","last_active":"2024-09-10T22:01:35.595852+0000","last_peered":"2024-09-10T22:01:35.595852+0000","last_clean":"2024-09-10T22:01:35.595852+0000","last_became_active":"2024-09-10T22:00:31.082025+0000","last_became_peered":"2024-09-10T22:00:31.082025+0000","last_unstale":"2024-09-10T22:01:35.595852+0000","last_undegraded":"2024-09-10T22:01:35.595852+0000","last_fullsized":"2024-09-10T22:01:35.595852+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:21.933419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189264+0000","last_change":"2024-09-10T22:00:43.760997+0000","last_active":"2024-09-10T22:01:35.189264+0000","last_peered":"2024-09-10T22:01:35.189264+0000","last_clean":"2024-09-10T22:01:35.189264+0000","last_became_active":"2024-09-10T22:00:43.760124+0000","last_became_peered":"2024-09-10T22:00:43.760124+0000","last_unstale":"2024-09-10T22:01:35.189264+0000","last_undegraded":"2024-09-10T22:01:35.189264+0000","last_fullsized":"2024-09-10T22:01:35.189264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:29.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.192492+0000","last_change":"2024-09-10T22:00:34.511730+0000","last_active":"2024-09-10T22:01:35.192492+0000","last_peered":"2024-09-10T22:01:35.192492+0000","last_clean":"2024-09-10T22:01:35.192492+0000","last_became_active":"2024-09-10T22:00:34.511440+0000","last_became_peered":"2024-09-10T22:00:34.511440+0000","last_unstale":"2024-09-10T22:01:35.192492+0000","last_undegraded":"2024-09-10T22:01:35.192492+0000","last_fullsized":"2024-09-10T22:01:35.192492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:09.053835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:52.784829+0000","last_change":"2024-09-10T22:00:30.825999+0000","last_active":"2024-09-10T22:01:52.784829+0000","last_peered":"2024-09-10T22:01:52.784829+0000","last_clean":"2024-09-10T22:01:52.784829+0000","last_became_active":"2024-09-10T22:00:30.825334+0000","last_became_peered":"2024-09-10T22:00:30.825334+0000","last_unstale":"2024-09-10T22:01:52.784829+0000","last_undegraded":"2024-09-10T22:01:52.784829+0000","last_fullsized":"2024-09-10T22:01:52.784829+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:14.684222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749605+0000","last_change":"2024-09-10T22:00:43.710517+0000","last_active":"2024-09-10T22:01:35.749605+0000","last_peered":"2024-09-10T22:01:35.749605+0000","last_clean":"2024-09-10T22:01:35.749605+0000","last_became_active":"2024-09-10T22:00:43.709252+0000","last_became_peered":"2024-09-10T22:00:43.709252+0000","last_unstale":"2024-09-10T22:01:35.749605+0000","last_undegraded":"2024-09-10T22:01:35.749605+0000","last_fullsized":"2024-09-10T22:01:35.749605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:21.355152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443126+0000","last_change":"2024-09-10T22:00:34.498168+0000","last_active":"2024-09-10T22:01:35.443126+0000","last_peered":"2024-09-10T22:01:35.443126+0000","last_clean":"2024-09-10T22:01:35.443126+0000","last_became_active":"2024-09-10T22:00:34.497867+0000","last_became_peered":"2024-09-10T22:00:34.497867+0000","last_unstale":"2024-09-10T22:01:35.443126+0000","last_undegraded":"2024-09-10T22:01:35.443126+0000","last_fullsized":"2024-09-10T22:01:35.443126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:56:14.570265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:52.801379+0000","last_change":"2024-09-10T22:00:31.081590+0000","last_active":"2024-09-10T22:01:52.801379+0000","last_peered":"2024-09-10T22:01:52.801379+0000","last_clean":"2024-09-10T22:01:52.801379+0000","last_became_active":"2024-09-10T22:00:31.081250+0000","last_became_peered":"2024-09-10T22:00:31.081250+0000","last_unstale":"2024-09-10T22:01:52.801379+0000","last_undegraded":"2024-09-10T22:01:52.801379+0000","last_fullsized":"2024-09-10T22:01:52.801379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:12.872832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749980+0000","last_change":"2024-09-10T22:00:43.761351+0000","last_active":"2024-09-10T22:01:35.749980+0000","last_peered":"2024-09-10T22:01:35.749980+0000","last_clean":"2024-09-10T22:01:35.749980+0000","last_became_active":"2024-09-10T22:00:43.760903+0000","last_became_peered":"2024-09-10T22:00:43.760903+0000","last_unstale":"2024-09-10T22:01:35.749980+0000","last_undegraded":"2024-09-10T22:01:35.749980+0000","last_fullsized":"2024-09-10T22:01:35.749980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:14.113974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749203+0000","last_change":"2024-09-10T22:00:34.504671+0000","last_active":"2024-09-10T22:01:35.749203+0000","last_peered":"2024-09-10T22:01:35.749203+0000","last_clean":"2024-09-10T22:01:35.749203+0000","last_became_active":"2024-09-10T22:00:34.504368+0000","last_became_peered":"2024-09-10T22:00:34.504368+0000","last_unstale":"2024-09-10T22:01:35.749203+0000","last_undegraded":"2024-09-10T22:01:35.749203+0000","last_fullsized":"2024-09-10T22:01:35.749203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:59.771589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194611+0000","last_change":"2024-09-10T22:00:30.820401+0000","last_active":"2024-09-10T22:01:35.194611+0000","last_peered":"2024-09-10T22:01:35.194611+0000","last_clean":"2024-09-10T22:01:35.194611+0000","last_became_active":"2024-09-10T22:00:30.820194+0000","last_became_peered":"2024-09-10T22:00:30.820194+0000","last_unstale":"2024-09-10T22:01:35.194611+0000","last_undegraded":"2024-09-10T22:01:35.194611+0000","last_fullsized":"2024-09-10T22:01:35.194611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:47.767102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443924+0000","last_change":"2024-09-10T22:00:43.762001+0000","last_active":"2024-09-10T22:01:35.443924+0000","last_peered":"2024-09-10T22:01:35.443924+0000","last_clean":"2024-09-10T22:01:35.443924+0000","last_became_active":"2024-09-10T22:00:43.761323+0000","last_became_peered":"2024-09-10T22:00:43.761323+0000","last_unstale":"2024-09-10T22:01:35.443924+0000","last_undegraded":"2024-09-10T22:01:35.443924+0000","last_fullsized":"2024-09-10T22:01:35.443924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:50.113545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618461+0000","last_change":"2024-09-10T22:00:34.498591+0000","last_active":"2024-09-10T22:01:35.618461+0000","last_peered":"2024-09-10T22:01:35.618461+0000","last_clean":"2024-09-10T22:01:35.618461+0000","last_became_active":"2024-09-10T22:00:34.496449+0000","last_became_peered":"2024-09-10T22:00:34.496449+0000","last_unstale":"2024-09-10T22:01:35.618461+0000","last_undegraded":"2024-09-10T22:01:35.618461+0000","last_fullsized":"2024-09-10T22:01:35.618461+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:54.815268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.191416+0000","last_change":"2024-09-10T22:00:30.817201+0000","last_active":"2024-09-10T22:01:35.191416+0000","last_peered":"2024-09-10T22:01:35.191416+0000","last_clean":"2024-09-10T22:01:35.191416+0000","last_became_active":"2024-09-10T22:00:30.816792+0000","last_became_peered":"2024-09-10T22:00:30.816792+0000","last_unstale":"2024-09-10T22:01:35.191416+0000","last_undegraded":"2024-09-10T22:01:35.191416+0000","last_fullsized":"2024-09-10T22:01:35.191416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:48.239811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194713+0000","last_change":"2024-09-10T21:59:01.830452+0000","last_active":"2024-09-10T22:01:35.194713+0000","last_peered":"2024-09-10T22:01:35.194713+0000","last_clean":"2024-09-10T22:01:35.194713+0000","last_became_active":"2024-09-10T21:59:01.823744+0000","last_became_peered":"2024-09-10T21:59:01.823744+0000","last_unstale":"2024-09-10T22:01:35.194713+0000","last_undegraded":"2024-09-10T22:01:35.194713+0000","last_fullsized":"2024-09-10T22:01:35.194713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_clean_scrub_stamp":"2024-09-10T21:54:48.621299+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:39.573257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194993+0000","last_change":"2024-09-10T22:00:43.713888+0000","last_active":"2024-09-10T22:01:35.194993+0000","last_peered":"2024-09-10T22:01:35.194993+0000","last_clean":"2024-09-10T22:01:35.194993+0000","last_became_active":"2024-09-10T22:00:43.713329+0000","last_became_peered":"2024-09-10T22:00:43.713329+0000","last_unstale":"2024-09-10T22:01:35.194993+0000","last_undegraded":"2024-09-10T22:01:35.194993+0000","last_fullsized":"2024-09-10T22:01:35.194993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:01.062089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195205+0000","last_change":"2024-09-10T22:00:34.502589+0000","last_active":"2024-09-10T22:01:35.195205+0000","last_peered":"2024-09-10T22:01:35.195205+0000","last_clean":"2024-09-10T22:01:35.195205+0000","last_became_active":"2024-09-10T22:00:34.502216+0000","last_became_peered":"2024-09-10T22:00:34.502216+0000","last_unstale":"2024-09-10T22:01:35.195205+0000","last_undegraded":"2024-09-10T22:01:35.195205+0000","last_fullsized":"2024-09-10T22:01:35.195205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:44.959375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:51.784673+0000","last_change":"2024-09-10T22:00:31.311807+0000","last_active":"2024-09-10T22:01:51.784673+0000","last_peered":"2024-09-10T22:01:51.784673+0000","last_clean":"2024-09-10T22:01:51.784673+0000","last_became_active":"2024-09-10T22:00:31.311424+0000","last_became_peered":"2024-09-10T22:00:31.311424+0000","last_unstale":"2024-09-10T22:01:51.784673+0000","last_undegraded":"2024-09-10T22:01:51.784673+0000","last_fullsized":"2024-09-10T22:01:51.784673+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:09.754777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.876960+0000","last_change":"2024-09-10T22:00:43.711882+0000","last_active":"2024-09-10T22:01:35.876960+0000","last_peered":"2024-09-10T22:01:35.876960+0000","last_clean":"2024-09-10T22:01:35.876960+0000","last_became_active":"2024-09-10T22:00:43.711408+0000","last_became_peered":"2024-09-10T22:00:43.711408+0000","last_unstale":"2024-09-10T22:01:35.876960+0000","last_undegraded":"2024-09-10T22:01:35.876960+0000","last_fullsized":"2024-09-10T22:01:35.876960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:31.155723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195251+0000","last_change":"2024-09-10T22:00:34.505653+0000","last_active":"2024-09-10T22:01:35.195251+0000","last_peered":"2024-09-10T22:01:35.195251+0000","last_clean":"2024-09-10T22:01:35.195251+0000","last_became_active":"2024-09-10T22:00:34.505317+0000","last_became_peered":"2024-09-10T22:00:34.505317+0000","last_unstale":"2024-09-10T22:01:35.195251+0000","last_undegraded":"2024-09-10T22:01:35.195251+0000","last_fullsized":"2024-09-10T22:01:35.195251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:35.050892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.444093+0000","last_change":"2024-09-10T22:00:30.817431+0000","last_active":"2024-09-10T22:01:35.444093+0000","last_peered":"2024-09-10T22:01:35.444093+0000","last_clean":"2024-09-10T22:01:35.444093+0000","last_became_active":"2024-09-10T22:00:30.816952+0000","last_became_peered":"2024-09-10T22:00:30.816952+0000","last_unstale":"2024-09-10T22:01:35.444093+0000","last_undegraded":"2024-09-10T22:01:35.444093+0000","last_fullsized":"2024-09-10T22:01:35.444093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:37.881641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189475+0000","last_change":"2024-09-10T22:00:43.720627+0000","last_active":"2024-09-10T22:01:35.189475+0000","last_peered":"2024-09-10T22:01:35.189475+0000","last_clean":"2024-09-10T22:01:35.189475+0000","last_became_active":"2024-09-10T22:00:43.720235+0000","last_became_peered":"2024-09-10T22:00:43.720235+0000","last_unstale":"2024-09-10T22:01:35.189475+0000","last_undegraded":"2024-09-10T22:01:35.189475+0000","last_fullsized":"2024-09-10T22:01:35.189475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:50.549536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195435+0000","last_change":"2024-09-10T22:00:34.501649+0000","last_active":"2024-09-10T22:01:35.195435+0000","last_peered":"2024-09-10T22:01:35.195435+0000","last_clean":"2024-09-10T22:01:35.195435+0000","last_became_active":"2024-09-10T22:00:34.498882+0000","last_became_peered":"2024-09-10T22:00:34.498882+0000","last_unstale":"2024-09-10T22:01:35.195435+0000","last_undegraded":"2024-09-10T22:01:35.195435+0000","last_fullsized":"2024-09-10T22:01:35.195435+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.750071+0000","last_change":"2024-09-10T22:00:30.821943+0000","last_active":"2024-09-10T22:01:35.750071+0000","last_peered":"2024-09-10T22:01:35.750071+0000","last_clean":"2024-09-10T22:01:35.750071+0000","last_became_active":"2024-09-10T22:00:30.821372+0000","last_became_peered":"2024-09-10T22:00:30.821372+0000","last_unstale":"2024-09-10T22:01:35.750071+0000","last_undegraded":"2024-09-10T22:01:35.750071+0000","last_fullsized":"2024-09-10T22:01:35.750071+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:04.360383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.442853+0000","last_change":"2024-09-10T22:00:43.710045+0000","last_active":"2024-09-10T22:01:35.442853+0000","last_peered":"2024-09-10T22:01:35.442853+0000","last_clean":"2024-09-10T22:01:35.442853+0000","last_became_active":"2024-09-10T22:00:43.709416+0000","last_became_peered":"2024-09-10T22:00:43.709416+0000","last_unstale":"2024-09-10T22:01:35.442853+0000","last_undegraded":"2024-09-10T22:01:35.442853+0000","last_fullsized":"2024-09-10T22:01:35.442853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:56.005549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195561+0000","last_change":"2024-09-10T22:00:34.496163+0000","last_active":"2024-09-10T22:01:35.195561+0000","last_peered":"2024-09-10T22:01:35.195561+0000","last_clean":"2024-09-10T22:01:35.195561+0000","last_became_active":"2024-09-10T22:00:34.495814+0000","last_became_peered":"2024-09-10T22:00:34.495814+0000","last_unstale":"2024-09-10T22:01:35.195561+0000","last_undegraded":"2024-09-10T22:01:35.195561+0000","last_fullsized":"2024-09-10T22:01:35.195561+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:05.398154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189944+0000","last_change":"2024-09-10T22:00:30.812702+0000","last_active":"2024-09-10T22:01:35.189944+0000","last_peered":"2024-09-10T22:01:35.189944+0000","last_clean":"2024-09-10T22:01:35.189944+0000","last_became_active":"2024-09-10T22:00:30.812369+0000","last_became_peered":"2024-09-10T22:00:30.812369+0000","last_unstale":"2024-09-10T22:01:35.189944+0000","last_undegraded":"2024-09-10T22:01:35.189944+0000","last_fullsized":"2024-09-10T22:01:35.189944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:52.332876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443239+0000","last_change":"2024-09-10T22:00:43.715573+0000","last_active":"2024-09-10T22:01:35.443239+0000","last_peered":"2024-09-10T22:01:35.443239+0000","last_clean":"2024-09-10T22:01:35.443239+0000","last_became_active":"2024-09-10T22:00:43.714475+0000","last_became_peered":"2024-09-10T22:00:43.714475+0000","last_unstale":"2024-09-10T22:01:35.443239+0000","last_undegraded":"2024-09-10T22:01:35.443239+0000","last_fullsized":"2024-09-10T22:01:35.443239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:55.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443379+0000","last_change":"2024-09-10T22:00:34.505126+0000","last_active":"2024-09-10T22:01:35.443379+0000","last_peered":"2024-09-10T22:01:35.443379+0000","last_clean":"2024-09-10T22:01:35.443379+0000","last_became_active":"2024-09-10T22:00:34.504653+0000","last_became_peered":"2024-09-10T22:00:34.504653+0000","last_unstale":"2024-09-10T22:01:35.443379+0000","last_undegraded":"2024-09-10T22:01:35.443379+0000","last_fullsized":"2024-09-10T22:01:35.443379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:10.202902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.903455+0000","last_change":"2024-09-10T22:01:35.903381+0000","last_active":"2024-09-10T22:01:35.903455+0000","last_peered":"2024-09-10T22:01:35.903455+0000","last_clean":"2024-09-10T22:01:35.903455+0000","last_became_active":"2024-09-10T22:01:35.878262+0000","last_became_peered":"2024-09-10T22:01:35.878262+0000","last_unstale":"2024-09-10T22:01:35.903455+0000","last_undegraded":"2024-09-10T22:01:35.903455+0000","last_fullsized":"2024-09-10T22:01:35.903455+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:45.419449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.750036+0000","last_change":"2024-09-10T22:00:43.720728+0000","last_active":"2024-09-10T22:01:35.750036+0000","last_peered":"2024-09-10T22:01:35.750036+0000","last_clean":"2024-09-10T22:01:35.750036+0000","last_became_active":"2024-09-10T22:00:43.720305+0000","last_became_peered":"2024-09-10T22:00:43.720305+0000","last_unstale":"2024-09-10T22:01:35.750036+0000","last_undegraded":"2024-09-10T22:01:35.750036+0000","last_fullsized":"2024-09-10T22:01:35.750036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:54.986822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194548+0000","last_change":"2024-09-10T22:00:34.501571+0000","last_active":"2024-09-10T22:01:35.194548+0000","last_peered":"2024-09-10T22:01:35.194548+0000","last_clean":"2024-09-10T22:01:35.194548+0000","last_became_active":"2024-09-10T22:00:34.501211+0000","last_became_peered":"2024-09-10T22:00:34.501211+0000","last_unstale":"2024-09-10T22:01:35.194548+0000","last_undegraded":"2024-09-10T22:01:35.194548+0000","last_fullsized":"2024-09-10T22:01:35.194548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:05.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:50.798500+0000","last_change":"2024-09-10T22:00:30.824296+0000","last_active":"2024-09-10T22:01:50.798500+0000","last_peered":"2024-09-10T22:01:50.798500+0000","last_clean":"2024-09-10T22:01:50.798500+0000","last_became_active":"2024-09-10T22:00:30.823886+0000","last_became_peered":"2024-09-10T22:00:30.823886+0000","last_unstale":"2024-09-10T22:01:50.798500+0000","last_undegraded":"2024-09-10T22:01:50.798500+0000","last_fullsized":"2024-09-10T22:01:50.798500+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:26.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595913+0000","last_change":"2024-09-10T22:00:43.717580+0000","last_active":"2024-09-10T22:01:35.595913+0000","last_peered":"2024-09-10T22:01:35.595913+0000","last_clean":"2024-09-10T22:01:35.595913+0000","last_became_active":"2024-09-10T22:00:43.708377+0000","last_became_peered":"2024-09-10T22:00:43.708377+0000","last_unstale":"2024-09-10T22:01:35.595913+0000","last_undegraded":"2024-09-10T22:01:35.595913+0000","last_fullsized":"2024-09-10T22:01:35.595913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:54.682292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195209+0000","last_change":"2024-09-10T22:00:34.514348+0000","last_active":"2024-09-10T22:01:35.195209+0000","last_peered":"2024-09-10T22:01:35.195209+0000","last_clean":"2024-09-10T22:01:35.195209+0000","last_became_active":"2024-09-10T22:00:34.513797+0000","last_became_peered":"2024-09-10T22:00:34.513797+0000","last_unstale":"2024-09-10T22:01:35.195209+0000","last_undegraded":"2024-09-10T22:01:35.195209+0000","last_fullsized":"2024-09-10T22:01:35.195209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:12.797643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443518+0000","last_change":"2024-09-10T22:00:30.826486+0000","last_active":"2024-09-10T22:01:35.443518+0000","last_peered":"2024-09-10T22:01:35.443518+0000","last_clean":"2024-09-10T22:01:35.443518+0000","last_became_active":"2024-09-10T22:00:30.826111+0000","last_became_peered":"2024-09-10T22:00:30.826111+0000","last_unstale":"2024-09-10T22:01:35.443518+0000","last_undegraded":"2024-09-10T22:01:35.443518+0000","last_fullsized":"2024-09-10T22:01:35.443518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:45.609925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193020+0000","last_change":"2024-09-10T22:00:43.709432+0000","last_active":"2024-09-10T22:01:35.193020+0000","last_peered":"2024-09-10T22:01:35.193020+0000","last_clean":"2024-09-10T22:01:35.193020+0000","last_became_active":"2024-09-10T22:00:43.708627+0000","last_became_peered":"2024-09-10T22:00:43.708627+0000","last_unstale":"2024-09-10T22:01:35.193020+0000","last_undegraded":"2024-09-10T22:01:35.193020+0000","last_fullsized":"2024-09-10T22:01:35.193020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:12.799976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749410+0000","last_change":"2024-09-10T22:00:34.499203+0000","last_active":"2024-09-10T22:01:35.749410+0000","last_peered":"2024-09-10T22:01:35.749410+0000","last_clean":"2024-09-10T22:01:35.749410+0000","last_became_active":"2024-09-10T22:00:34.497877+0000","last_became_peered":"2024-09-10T22:00:34.497877+0000","last_unstale":"2024-09-10T22:01:35.749410+0000","last_undegraded":"2024-09-10T22:01:35.749410+0000","last_fullsized":"2024-09-10T22:01:35.749410+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:41.897603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195380+0000","last_change":"2024-09-10T22:00:30.820852+0000","last_active":"2024-09-10T22:01:35.195380+0000","last_peered":"2024-09-10T22:01:35.195380+0000","last_clean":"2024-09-10T22:01:35.195380+0000","last_became_active":"2024-09-10T22:00:30.820023+0000","last_became_peered":"2024-09-10T22:00:30.820023+0000","last_unstale":"2024-09-10T22:01:35.195380+0000","last_undegraded":"2024-09-10T22:01:35.195380+0000","last_fullsized":"2024-09-10T22:01:35.195380+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:54.245142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443707+0000","last_change":"2024-09-10T22:00:43.716323+0000","last_active":"2024-09-10T22:01:35.443707+0000","last_peered":"2024-09-10T22:01:35.443707+0000","last_clean":"2024-09-10T22:01:35.443707+0000","last_became_active":"2024-09-10T22:00:43.715827+0000","last_became_peered":"2024-09-10T22:00:43.715827+0000","last_unstale":"2024-09-10T22:01:35.443707+0000","last_undegraded":"2024-09-10T22:01:35.443707+0000","last_fullsized":"2024-09-10T22:01:35.443707+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:10.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189839+0000","last_change":"2024-09-10T22:00:34.500114+0000","last_active":"2024-09-10T22:01:35.189839+0000","last_peered":"2024-09-10T22:01:35.189839+0000","last_clean":"2024-09-10T22:01:35.189839+0000","last_became_active":"2024-09-10T22:00:34.499334+0000","last_became_peered":"2024-09-10T22:00:34.499334+0000","last_unstale":"2024-09-10T22:01:35.189839+0000","last_undegraded":"2024-09-10T22:01:35.189839+0000","last_fullsized":"2024-09-10T22:01:35.189839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:11.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":127,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:54.799028+0000","last_change":"2024-09-10T22:00:30.822561+0000","last_active":"2024-09-10T22:01:54.799028+0000","last_peered":"2024-09-10T22:01:54.799028+0000","last_clean":"2024-09-10T22:01:54.799028+0000","last_became_active":"2024-09-10T22:00:30.822201+0000","last_became_peered":"2024-09-10T22:00:30.822201+0000","last_unstale":"2024-09-10T22:01:54.799028+0000","last_undegraded":"2024-09-10T22:01:54.799028+0000","last_fullsized":"2024-09-10T22:01:54.799028+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:41.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62657,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":88,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.445101+0000","last_change":"2024-09-10T22:01:35.444988+0000","last_active":"2024-09-10T22:01:35.445101+0000","last_peered":"2024-09-10T22:01:35.445101+0000","last_clean":"2024-09-10T22:01:35.445101+0000","last_became_active":"2024-09-10T22:01:35.444498+0000","last_became_peered":"2024-09-10T22:01:35.444498+0000","last_unstale":"2024-09-10T22:01:35.445101+0000","last_undegraded":"2024-09-10T22:01:35.445101+0000","last_fullsized":"2024-09-10T22:01:35.445101+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:35.017660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877143+0000","last_change":"2024-09-10T22:00:34.503234+0000","last_active":"2024-09-10T22:01:35.877143+0000","last_peered":"2024-09-10T22:01:35.877143+0000","last_clean":"2024-09-10T22:01:35.877143+0000","last_became_active":"2024-09-10T22:00:34.501501+0000","last_became_peered":"2024-09-10T22:00:34.501501+0000","last_unstale":"2024-09-10T22:01:35.877143+0000","last_undegraded":"2024-09-10T22:01:35.877143+0000","last_fullsized":"2024-09-10T22:01:35.877143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:23.772491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443988+0000","last_change":"2024-09-10T22:00:31.082131+0000","last_active":"2024-09-10T22:01:35.443988+0000","last_peered":"2024-09-10T22:01:35.443988+0000","last_clean":"2024-09-10T22:01:35.443988+0000","last_became_active":"2024-09-10T22:00:31.081405+0000","last_became_peered":"2024-09-10T22:00:31.081405+0000","last_unstale":"2024-09-10T22:01:35.443988+0000","last_undegraded":"2024-09-10T22:01:35.443988+0000","last_fullsized":"2024-09-10T22:01:35.443988+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:02.400004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.191177+0000","last_change":"2024-09-10T22:00:43.705732+0000","last_active":"2024-09-10T22:01:35.191177+0000","last_peered":"2024-09-10T22:01:35.191177+0000","last_clean":"2024-09-10T22:01:35.191177+0000","last_became_active":"2024-09-10T22:00:43.705274+0000","last_became_peered":"2024-09-10T22:00:43.705274+0000","last_unstale":"2024-09-10T22:01:35.191177+0000","last_undegraded":"2024-09-10T22:01:35.191177+0000","last_fullsized":"2024-09-10T22:01:35.191177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:54.707636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189696+0000","last_change":"2024-09-10T22:00:34.499585+0000","last_active":"2024-09-10T22:01:35.189696+0000","last_peered":"2024-09-10T22:01:35.189696+0000","last_clean":"2024-09-10T22:01:35.189696+0000","last_became_active":"2024-09-10T22:00:34.499261+0000","last_became_peered":"2024-09-10T22:00:34.499261+0000","last_unstale":"2024-09-10T22:01:35.189696+0000","last_undegraded":"2024-09-10T22:01:35.189696+0000","last_fullsized":"2024-09-10T22:01:35.189696+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:41.156093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194845+0000","last_change":"2024-09-10T22:00:30.823322+0000","last_active":"2024-09-10T22:01:35.194845+0000","last_peered":"2024-09-10T22:01:35.194845+0000","last_clean":"2024-09-10T22:01:35.194845+0000","last_became_active":"2024-09-10T22:00:30.822987+0000","last_became_peered":"2024-09-10T22:00:30.822987+0000","last_unstale":"2024-09-10T22:01:35.194845+0000","last_undegraded":"2024-09-10T22:01:35.194845+0000","last_fullsized":"2024-09-10T22:01:35.194845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:33.605627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595416+0000","last_change":"2024-09-10T22:00:43.708746+0000","last_active":"2024-09-10T22:01:35.595416+0000","last_peered":"2024-09-10T22:01:35.595416+0000","last_clean":"2024-09-10T22:01:35.595416+0000","last_became_active":"2024-09-10T22:00:43.708283+0000","last_became_peered":"2024-09-10T22:00:43.708283+0000","last_unstale":"2024-09-10T22:01:35.595416+0000","last_undegraded":"2024-09-10T22:01:35.595416+0000","last_fullsized":"2024-09-10T22:01:35.595416+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:30.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877356+0000","last_change":"2024-09-10T22:00:34.501472+0000","last_active":"2024-09-10T22:01:35.877356+0000","last_peered":"2024-09-10T22:01:35.877356+0000","last_clean":"2024-09-10T22:01:35.877356+0000","last_became_active":"2024-09-10T22:00:34.501131+0000","last_became_peered":"2024-09-10T22:00:34.501131+0000","last_unstale":"2024-09-10T22:01:35.877356+0000","last_undegraded":"2024-09-10T22:01:35.877356+0000","last_fullsized":"2024-09-10T22:01:35.877356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:11.281413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":89,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:54.789667+0000","last_change":"2024-09-10T22:00:30.828335+0000","last_active":"2024-09-10T22:01:54.789667+0000","last_peered":"2024-09-10T22:01:54.789667+0000","last_clean":"2024-09-10T22:01:54.789667+0000","last_became_active":"2024-09-10T22:00:30.828028+0000","last_became_peered":"2024-09-10T22:00:30.828028+0000","last_unstale":"2024-09-10T22:01:54.789667+0000","last_undegraded":"2024-09-10T22:01:54.789667+0000","last_fullsized":"2024-09-10T22:01:54.789667+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:56.399002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618333+0000","last_change":"2024-09-10T22:00:43.710086+0000","last_active":"2024-09-10T22:01:35.618333+0000","last_peered":"2024-09-10T22:01:35.618333+0000","last_clean":"2024-09-10T22:01:35.618333+0000","last_became_active":"2024-09-10T22:00:43.709518+0000","last_became_peered":"2024-09-10T22:00:43.709518+0000","last_unstale":"2024-09-10T22:01:35.618333+0000","last_undegraded":"2024-09-10T22:01:35.618333+0000","last_fullsized":"2024-09-10T22:01:35.618333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:07.636887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189263+0000","last_change":"2024-09-10T22:00:43.711620+0000","last_active":"2024-09-10T22:01:35.189263+0000","last_peered":"2024-09-10T22:01:35.189263+0000","last_clean":"2024-09-10T22:01:35.189263+0000","last_became_active":"2024-09-10T22:00:43.711224+0000","last_became_peered":"2024-09-10T22:00:43.711224+0000","last_unstale":"2024-09-10T22:01:35.189263+0000","last_undegraded":"2024-09-10T22:01:35.189263+0000","last_fullsized":"2024-09-10T22:01:35.189263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:43.860286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595581+0000","last_change":"2024-09-10T22:00:34.494408+0000","last_active":"2024-09-10T22:01:35.595581+0000","last_peered":"2024-09-10T22:01:35.595581+0000","last_clean":"2024-09-10T22:01:35.595581+0000","last_became_active":"2024-09-10T22:00:34.494066+0000","last_became_peered":"2024-09-10T22:00:34.494066+0000","last_unstale":"2024-09-10T22:01:35.595581+0000","last_undegraded":"2024-09-10T22:01:35.595581+0000","last_fullsized":"2024-09-10T22:01:35.595581+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:49.565755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443303+0000","last_change":"2024-09-10T22:00:30.821842+0000","last_active":"2024-09-10T22:01:35.443303+0000","last_peered":"2024-09-10T22:01:35.443303+0000","last_clean":"2024-09-10T22:01:35.443303+0000","last_became_active":"2024-09-10T22:00:30.821499+0000","last_became_peered":"2024-09-10T22:00:30.821499+0000","last_unstale":"2024-09-10T22:01:35.443303+0000","last_undegraded":"2024-09-10T22:01:35.443303+0000","last_fullsized":"2024-09-10T22:01:35.443303+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:19.125052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195492+0000","last_change":"2024-09-10T22:00:43.706689+0000","last_active":"2024-09-10T22:01:35.195492+0000","last_peered":"2024-09-10T22:01:35.195492+0000","last_clean":"2024-09-10T22:01:35.195492+0000","last_became_active":"2024-09-10T22:00:43.705954+0000","last_became_peered":"2024-09-10T22:00:43.705954+0000","last_unstale":"2024-09-10T22:01:35.195492+0000","last_undegraded":"2024-09-10T22:01:35.195492+0000","last_fullsized":"2024-09-10T22:01:35.195492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:30.800710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618231+0000","last_change":"2024-09-10T22:00:34.490853+0000","last_active":"2024-09-10T22:01:35.618231+0000","last_peered":"2024-09-10T22:01:35.618231+0000","last_clean":"2024-09-10T22:01:35.618231+0000","last_became_active":"2024-09-10T22:00:34.489908+0000","last_became_peered":"2024-09-10T22:00:34.489908+0000","last_unstale":"2024-09-10T22:01:35.618231+0000","last_undegraded":"2024-09-10T22:01:35.618231+0000","last_fullsized":"2024-09-10T22:01:35.618231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:57.105004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749247+0000","last_change":"2024-09-10T22:00:30.816497+0000","last_active":"2024-09-10T22:01:35.749247+0000","last_peered":"2024-09-10T22:01:35.749247+0000","last_clean":"2024-09-10T22:01:35.749247+0000","last_became_active":"2024-09-10T22:00:30.816181+0000","last_became_peered":"2024-09-10T22:00:30.816181+0000","last_unstale":"2024-09-10T22:01:35.749247+0000","last_undegraded":"2024-09-10T22:01:35.749247+0000","last_fullsized":"2024-09-10T22:01:35.749247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:12.779113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193229+0000","last_change":"2024-09-10T22:00:30.821402+0000","last_active":"2024-09-10T22:01:35.193229+0000","last_peered":"2024-09-10T22:01:35.193229+0000","last_clean":"2024-09-10T22:01:35.193229+0000","last_became_active":"2024-09-10T22:00:30.821105+0000","last_became_peered":"2024-09-10T22:00:30.821105+0000","last_unstale":"2024-09-10T22:01:35.193229+0000","last_undegraded":"2024-09-10T22:01:35.193229+0000","last_fullsized":"2024-09-10T22:01:35.193229+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:36.090579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443700+0000","last_change":"2024-09-10T22:00:34.511911+0000","last_active":"2024-09-10T22:01:35.443700+0000","last_peered":"2024-09-10T22:01:35.443700+0000","last_clean":"2024-09-10T22:01:35.443700+0000","last_became_active":"2024-09-10T22:00:34.511545+0000","last_became_peered":"2024-09-10T22:00:34.511545+0000","last_unstale":"2024-09-10T22:01:35.443700+0000","last_undegraded":"2024-09-10T22:01:35.443700+0000","last_fullsized":"2024-09-10T22:01:35.443700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:12.484733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.750618+0000","last_change":"2024-09-10T22:01:35.750458+0000","last_active":"2024-09-10T22:01:35.750618+0000","last_peered":"2024-09-10T22:01:35.750618+0000","last_clean":"2024-09-10T22:01:35.750618+0000","last_became_active":"2024-09-10T22:01:35.749946+0000","last_became_peered":"2024-09-10T22:01:35.749946+0000","last_unstale":"2024-09-10T22:01:35.750618+0000","last_undegraded":"2024-09-10T22:01:35.750618+0000","last_fullsized":"2024-09-10T22:01:35.750618+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.362221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:50.783379+0000","last_change":"2024-09-10T22:00:30.826427+0000","last_active":"2024-09-10T22:01:50.783379+0000","last_peered":"2024-09-10T22:01:50.783379+0000","last_clean":"2024-09-10T22:01:50.783379+0000","last_became_active":"2024-09-10T22:00:30.826038+0000","last_became_peered":"2024-09-10T22:00:30.826038+0000","last_unstale":"2024-09-10T22:01:50.783379+0000","last_undegraded":"2024-09-10T22:01:50.783379+0000","last_fullsized":"2024-09-10T22:01:50.783379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:45.075208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189516+0000","last_change":"2024-09-10T22:00:34.494479+0000","last_active":"2024-09-10T22:01:35.189516+0000","last_peered":"2024-09-10T22:01:35.189516+0000","last_clean":"2024-09-10T22:01:35.189516+0000","last_became_active":"2024-09-10T22:00:34.494087+0000","last_became_peered":"2024-09-10T22:00:34.494087+0000","last_unstale":"2024-09-10T22:01:35.189516+0000","last_undegraded":"2024-09-10T22:01:35.189516+0000","last_fullsized":"2024-09-10T22:01:35.189516+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:28.548999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195923+0000","last_change":"2024-09-10T22:00:43.714568+0000","last_active":"2024-09-10T22:01:35.195923+0000","last_peered":"2024-09-10T22:01:35.195923+0000","last_clean":"2024-09-10T22:01:35.195923+0000","last_became_active":"2024-09-10T22:00:43.712923+0000","last_became_peered":"2024-09-10T22:00:43.712923+0000","last_unstale":"2024-09-10T22:01:35.195923+0000","last_undegraded":"2024-09-10T22:01:35.195923+0000","last_fullsized":"2024-09-10T22:01:35.195923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:27.945799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193388+0000","last_change":"2024-09-10T22:00:34.496942+0000","last_active":"2024-09-10T22:01:35.193388+0000","last_peered":"2024-09-10T22:01:35.193388+0000","last_clean":"2024-09-10T22:01:35.193388+0000","last_became_active":"2024-09-10T22:00:34.496475+0000","last_became_peered":"2024-09-10T22:00:34.496475+0000","last_unstale":"2024-09-10T22:01:35.193388+0000","last_undegraded":"2024-09-10T22:01:35.193388+0000","last_fullsized":"2024-09-10T22:01:35.193388+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:33.277346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193991+0000","last_change":"2024-09-10T22:00:34.506571+0000","last_active":"2024-09-10T22:01:35.193991+0000","last_peered":"2024-09-10T22:01:35.193991+0000","last_clean":"2024-09-10T22:01:35.193991+0000","last_became_active":"2024-09-10T22:00:34.505718+0000","last_became_peered":"2024-09-10T22:00:34.505718+0000","last_unstale":"2024-09-10T22:01:35.193991+0000","last_undegraded":"2024-09-10T22:01:35.193991+0000","last_fullsized":"2024-09-10T22:01:35.193991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:10.111415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194096+0000","last_change":"2024-09-10T22:00:34.502030+0000","last_active":"2024-09-10T22:01:35.194096+0000","last_peered":"2024-09-10T22:01:35.194096+0000","last_clean":"2024-09-10T22:01:35.194096+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:01:35.194096+0000","last_undegraded":"2024-09-10T22:01:35.194096+0000","last_fullsized":"2024-09-10T22:01:35.194096+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:09.532523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189536+0000","last_change":"2024-09-10T22:00:34.513630+0000","last_active":"2024-09-10T22:01:35.189536+0000","last_peered":"2024-09-10T22:01:35.189536+0000","last_clean":"2024-09-10T22:01:35.189536+0000","last_became_active":"2024-09-10T22:00:34.513348+0000","last_became_peered":"2024-09-10T22:00:34.513348+0000","last_unstale":"2024-09-10T22:01:35.189536+0000","last_undegraded":"2024-09-10T22:01:35.189536+0000","last_fullsized":"2024-09-10T22:01:35.189536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:53.942004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749744+0000","last_change":"2024-09-10T22:00:34.508697+0000","last_active":"2024-09-10T22:01:35.749744+0000","last_peered":"2024-09-10T22:01:35.749744+0000","last_clean":"2024-09-10T22:01:35.749744+0000","last_became_active":"2024-09-10T22:00:34.508371+0000","last_became_peered":"2024-09-10T22:00:34.508371+0000","last_unstale":"2024-09-10T22:01:35.749744+0000","last_undegraded":"2024-09-10T22:01:35.749744+0000","last_fullsized":"2024-09-10T22:01:35.749744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:32.843517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.196494+0000","last_change":"2024-09-10T22:00:34.496021+0000","last_active":"2024-09-10T22:01:35.196494+0000","last_peered":"2024-09-10T22:01:35.196494+0000","last_clean":"2024-09-10T22:01:35.196494+0000","last_became_active":"2024-09-10T22:00:34.495663+0000","last_became_peered":"2024-09-10T22:00:34.495663+0000","last_unstale":"2024-09-10T22:01:35.196494+0000","last_undegraded":"2024-09-10T22:01:35.196494+0000","last_fullsized":"2024-09-10T22:01:35.196494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:16.512182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.196183+0000","last_change":"2024-09-10T22:00:34.506589+0000","last_active":"2024-09-10T22:01:35.196183+0000","last_peered":"2024-09-10T22:01:35.196183+0000","last_clean":"2024-09-10T22:01:35.196183+0000","last_became_active":"2024-09-10T22:00:34.506214+0000","last_became_peered":"2024-09-10T22:00:34.506214+0000","last_unstale":"2024-09-10T22:01:35.196183+0000","last_undegraded":"2024-09-10T22:01:35.196183+0000","last_fullsized":"2024-09-10T22:01:35.196183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:47.072458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618891+0000","last_change":"2024-09-10T22:00:34.503471+0000","last_active":"2024-09-10T22:01:35.618891+0000","last_peered":"2024-09-10T22:01:35.618891+0000","last_clean":"2024-09-10T22:01:35.618891+0000","last_became_active":"2024-09-10T22:00:34.501145+0000","last_became_peered":"2024-09-10T22:00:34.501145+0000","last_unstale":"2024-09-10T22:01:35.618891+0000","last_undegraded":"2024-09-10T22:01:35.618891+0000","last_fullsized":"2024-09-10T22:01:35.618891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:24.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.443033+0000","last_change":"2024-09-10T22:00:34.490177+0000","last_active":"2024-09-10T22:01:35.443033+0000","last_peered":"2024-09-10T22:01:35.443033+0000","last_clean":"2024-09-10T22:01:35.443033+0000","last_became_active":"2024-09-10T22:00:34.488428+0000","last_became_peered":"2024-09-10T22:00:34.488428+0000","last_unstale":"2024-09-10T22:01:35.443033+0000","last_undegraded":"2024-09-10T22:01:35.443033+0000","last_fullsized":"2024-09-10T22:01:35.443033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:21.111826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618325+0000","last_change":"2024-09-10T22:00:34.499174+0000","last_active":"2024-09-10T22:01:35.618325+0000","last_peered":"2024-09-10T22:01:35.618325+0000","last_clean":"2024-09-10T22:01:35.618325+0000","last_became_active":"2024-09-10T22:00:34.498824+0000","last_became_peered":"2024-09-10T22:00:34.498824+0000","last_unstale":"2024-09-10T22:01:35.618325+0000","last_undegraded":"2024-09-10T22:01:35.618325+0000","last_fullsized":"2024-09-10T22:01:35.618325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:15.946270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877187+0000","last_change":"2024-09-10T22:00:34.505748+0000","last_active":"2024-09-10T22:01:35.877187+0000","last_peered":"2024-09-10T22:01:35.877187+0000","last_clean":"2024-09-10T22:01:35.877187+0000","last_became_active":"2024-09-10T22:00:34.500441+0000","last_became_peered":"2024-09-10T22:00:34.500441+0000","last_unstale":"2024-09-10T22:01:35.877187+0000","last_undegraded":"2024-09-10T22:01:35.877187+0000","last_fullsized":"2024-09-10T22:01:35.877187+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:22.177566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749624+0000","last_change":"2024-09-10T22:00:34.502319+0000","last_active":"2024-09-10T22:01:35.749624+0000","last_peered":"2024-09-10T22:01:35.749624+0000","last_clean":"2024-09-10T22:01:35.749624+0000","last_became_active":"2024-09-10T22:00:34.501427+0000","last_became_peered":"2024-09-10T22:00:34.501427+0000","last_unstale":"2024-09-10T22:01:35.749624+0000","last_undegraded":"2024-09-10T22:01:35.749624+0000","last_fullsized":"2024-09-10T22:01:35.749624+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:18.463319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193065+0000","last_change":"2024-09-10T22:00:34.497571+0000","last_active":"2024-09-10T22:01:35.193065+0000","last_peered":"2024-09-10T22:01:35.193065+0000","last_clean":"2024-09-10T22:01:35.193065+0000","last_became_active":"2024-09-10T22:00:34.497205+0000","last_became_peered":"2024-09-10T22:00:34.497205+0000","last_unstale":"2024-09-10T22:01:35.193065+0000","last_undegraded":"2024-09-10T22:01:35.193065+0000","last_fullsized":"2024-09-10T22:01:35.193065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:49.924518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.196020+0000","last_change":"2024-09-10T22:00:34.504641+0000","last_active":"2024-09-10T22:01:35.196020+0000","last_peered":"2024-09-10T22:01:35.196020+0000","last_clean":"2024-09-10T22:01:35.196020+0000","last_became_active":"2024-09-10T22:00:34.504183+0000","last_became_peered":"2024-09-10T22:00:34.504183+0000","last_unstale":"2024-09-10T22:01:35.196020+0000","last_undegraded":"2024-09-10T22:01:35.196020+0000","last_fullsized":"2024-09-10T22:01:35.196020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:06.704972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.595962+0000","last_change":"2024-09-10T22:00:34.503884+0000","last_active":"2024-09-10T22:01:35.595962+0000","last_peered":"2024-09-10T22:01:35.595962+0000","last_clean":"2024-09-10T22:01:35.595962+0000","last_became_active":"2024-09-10T22:00:34.502950+0000","last_became_peered":"2024-09-10T22:00:34.502950+0000","last_unstale":"2024-09-10T22:01:35.595962+0000","last_undegraded":"2024-09-10T22:01:35.595962+0000","last_fullsized":"2024-09-10T22:01:35.595962+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:45.560958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195749+0000","last_change":"2024-09-10T22:00:34.504981+0000","last_active":"2024-09-10T22:01:35.195749+0000","last_peered":"2024-09-10T22:01:35.195749+0000","last_clean":"2024-09-10T22:01:35.195749+0000","last_became_active":"2024-09-10T22:00:34.500474+0000","last_became_peered":"2024-09-10T22:00:34.500474+0000","last_unstale":"2024-09-10T22:01:35.195749+0000","last_undegraded":"2024-09-10T22:01:35.195749+0000","last_fullsized":"2024-09-10T22:01:35.195749+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:20.892605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877526+0000","last_change":"2024-09-10T22:00:34.506650+0000","last_active":"2024-09-10T22:01:35.877526+0000","last_peered":"2024-09-10T22:01:35.877526+0000","last_clean":"2024-09-10T22:01:35.877526+0000","last_became_active":"2024-09-10T22:00:34.505698+0000","last_became_peered":"2024-09-10T22:00:34.505698+0000","last_unstale":"2024-09-10T22:01:35.877526+0000","last_undegraded":"2024-09-10T22:01:35.877526+0000","last_fullsized":"2024-09-10T22:01:35.877526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:29.221693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189132+0000","last_change":"2024-09-10T22:00:34.502238+0000","last_active":"2024-09-10T22:01:35.189132+0000","last_peered":"2024-09-10T22:01:35.189132+0000","last_clean":"2024-09-10T22:01:35.189132+0000","last_became_active":"2024-09-10T22:00:34.500049+0000","last_became_peered":"2024-09-10T22:00:34.500049+0000","last_unstale":"2024-09-10T22:01:35.189132+0000","last_undegraded":"2024-09-10T22:01:35.189132+0000","last_fullsized":"2024-09-10T22:01:35.189132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:05.669784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189132+0000","last_change":"2024-09-10T22:00:34.499340+0000","last_active":"2024-09-10T22:01:35.189132+0000","last_peered":"2024-09-10T22:01:35.189132+0000","last_clean":"2024-09-10T22:01:35.189132+0000","last_became_active":"2024-09-10T22:00:34.497655+0000","last_became_peered":"2024-09-10T22:00:34.497655+0000","last_unstale":"2024-09-10T22:01:35.189132+0000","last_undegraded":"2024-09-10T22:01:35.189132+0000","last_fullsized":"2024-09-10T22:01:35.189132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:34.115829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.749525+0000","last_change":"2024-09-10T22:00:34.500705+0000","last_active":"2024-09-10T22:01:35.749525+0000","last_peered":"2024-09-10T22:01:35.749525+0000","last_clean":"2024-09-10T22:01:35.749525+0000","last_became_active":"2024-09-10T22:00:34.500071+0000","last_became_peered":"2024-09-10T22:00:34.500071+0000","last_unstale":"2024-09-10T22:01:35.749525+0000","last_undegraded":"2024-09-10T22:01:35.749525+0000","last_fullsized":"2024-09-10T22:01:35.749525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:40.519976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190179+0000","last_change":"2024-09-10T22:00:34.510570+0000","last_active":"2024-09-10T22:01:35.190179+0000","last_peered":"2024-09-10T22:01:35.190179+0000","last_clean":"2024-09-10T22:01:35.190179+0000","last_became_active":"2024-09-10T22:00:34.506922+0000","last_became_peered":"2024-09-10T22:00:34.506922+0000","last_unstale":"2024-09-10T22:01:35.190179+0000","last_undegraded":"2024-09-10T22:01:35.190179+0000","last_fullsized":"2024-09-10T22:01:35.190179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.489031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195114+0000","last_change":"2024-09-10T22:00:34.509985+0000","last_active":"2024-09-10T22:01:35.195114+0000","last_peered":"2024-09-10T22:01:35.195114+0000","last_clean":"2024-09-10T22:01:35.195114+0000","last_became_active":"2024-09-10T22:00:34.509527+0000","last_became_peered":"2024-09-10T22:00:34.509527+0000","last_unstale":"2024-09-10T22:01:35.195114+0000","last_undegraded":"2024-09-10T22:01:35.195114+0000","last_fullsized":"2024-09-10T22:01:35.195114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:49.463334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.193894+0000","last_change":"2024-09-10T22:00:34.497986+0000","last_active":"2024-09-10T22:01:35.193894+0000","last_peered":"2024-09-10T22:01:35.193894+0000","last_clean":"2024-09-10T22:01:35.193894+0000","last_became_active":"2024-09-10T22:00:34.497612+0000","last_became_peered":"2024-09-10T22:00:34.497612+0000","last_unstale":"2024-09-10T22:01:35.193894+0000","last_undegraded":"2024-09-10T22:01:35.193894+0000","last_fullsized":"2024-09-10T22:01:35.193894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:15.696988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.877514+0000","last_change":"2024-09-10T22:00:34.498482+0000","last_active":"2024-09-10T22:01:35.877514+0000","last_peered":"2024-09-10T22:01:35.877514+0000","last_clean":"2024-09-10T22:01:35.877514+0000","last_became_active":"2024-09-10T22:00:34.498139+0000","last_became_peered":"2024-09-10T22:00:34.498139+0000","last_unstale":"2024-09-10T22:01:35.877514+0000","last_undegraded":"2024-09-10T22:01:35.877514+0000","last_fullsized":"2024-09-10T22:01:35.877514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:23.427253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.618225+0000","last_change":"2024-09-10T22:00:34.491325+0000","last_active":"2024-09-10T22:01:35.618225+0000","last_peered":"2024-09-10T22:01:35.618225+0000","last_clean":"2024-09-10T22:01:35.618225+0000","last_became_active":"2024-09-10T22:00:34.491026+0000","last_became_peered":"2024-09-10T22:00:34.491026+0000","last_unstale":"2024-09-10T22:01:35.618225+0000","last_undegraded":"2024-09-10T22:01:35.618225+0000","last_fullsized":"2024-09-10T22:01:35.618225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:07.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.194822+0000","last_change":"2024-09-10T22:00:34.503975+0000","last_active":"2024-09-10T22:01:35.194822+0000","last_peered":"2024-09-10T22:01:35.194822+0000","last_clean":"2024-09-10T22:01:35.194822+0000","last_became_active":"2024-09-10T22:00:34.503624+0000","last_became_peered":"2024-09-10T22:00:34.503624+0000","last_unstale":"2024-09-10T22:01:35.194822+0000","last_undegraded":"2024-09-10T22:01:35.194822+0000","last_fullsized":"2024-09-10T22:01:35.194822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:17.678811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.596879+0000","last_change":"2024-09-10T22:01:35.596824+0000","last_active":"2024-09-10T22:01:35.596879+0000","last_peered":"2024-09-10T22:01:35.596879+0000","last_clean":"2024-09-10T22:01:35.596879+0000","last_became_active":"2024-09-10T22:01:35.596550+0000","last_became_peered":"2024-09-10T22:01:35.596550+0000","last_unstale":"2024-09-10T22:01:35.596879+0000","last_undegraded":"2024-09-10T22:01:35.596879+0000","last_fullsized":"2024-09-10T22:01:35.596879+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:19.617649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189287+0000","last_change":"2024-09-10T22:00:34.501647+0000","last_active":"2024-09-10T22:01:35.189287+0000","last_peered":"2024-09-10T22:01:35.189287+0000","last_clean":"2024-09-10T22:01:35.189287+0000","last_became_active":"2024-09-10T22:00:34.499216+0000","last_became_peered":"2024-09-10T22:00:34.499216+0000","last_unstale":"2024-09-10T22:01:35.189287+0000","last_undegraded":"2024-09-10T22:01:35.189287+0000","last_fullsized":"2024-09-10T22:01:35.189287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:41:08.395469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.190176+0000","last_change":"2024-09-10T22:00:34.513134+0000","last_active":"2024-09-10T22:01:35.190176+0000","last_peered":"2024-09-10T22:01:35.190176+0000","last_clean":"2024-09-10T22:01:35.190176+0000","last_became_active":"2024-09-10T22:00:34.507578+0000","last_became_peered":"2024-09-10T22:00:34.507578+0000","last_unstale":"2024-09-10T22:01:35.190176+0000","last_undegraded":"2024-09-10T22:01:35.190176+0000","last_fullsized":"2024-09-10T22:01:35.190176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:33.387337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189178+0000","last_change":"2024-09-10T22:00:34.507645+0000","last_active":"2024-09-10T22:01:35.189178+0000","last_peered":"2024-09-10T22:01:35.189178+0000","last_clean":"2024-09-10T22:01:35.189178+0000","last_became_active":"2024-09-10T22:00:34.503614+0000","last_became_peered":"2024-09-10T22:00:34.503614+0000","last_unstale":"2024-09-10T22:01:35.189178+0000","last_undegraded":"2024-09-10T22:01:35.189178+0000","last_fullsized":"2024-09-10T22:01:35.189178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:53.033523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.189362+0000","last_change":"2024-09-10T22:00:34.501891+0000","last_active":"2024-09-10T22:01:35.189362+0000","last_peered":"2024-09-10T22:01:35.189362+0000","last_clean":"2024-09-10T22:01:35.189362+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:01:35.189362+0000","last_undegraded":"2024-09-10T22:01:35.189362+0000","last_fullsized":"2024-09-10T22:01:35.189362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:30.068461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-10T22:01:35.195722+0000","last_change":"2024-09-10T22:00:34.501475+0000","last_active":"2024-09-10T22:01:35.195722+0000","last_peered":"2024-09-10T22:01:35.195722+0000","last_clean":"2024-09-10T22:01:35.195722+0000","last_became_active":"2024-09-10T22:00:34.500909+0000","last_became_peered":"2024-09-10T22:00:34.500909+0000","last_unstale":"2024-09-10T22:01:35.195722+0000","last_undegraded":"2024-09-10T22:01:35.195722+0000","last_fullsized":"2024-09-10T22:01:35.195722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:39.616773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":255,"num_read_kb":206,"num_write":247,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246272,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678047,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":516912,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874276,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717024,"statfs":{"total":95995035648,"available":95966232576,"internally_reserved":0,"allocated":1130496,"data_stored":1166273,"data_compressed":62485,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037802,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":520435,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234033,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":516605,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":1113996,"data_compressed":62485,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593791,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":512234,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1108796,"data_compressed":62485,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":516308,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":511551,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378711,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":571017,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542238,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1056768,"data_stored":576033,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705763,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":513690,"data_compressed":26603,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:01:57.695 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:01:57.696 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T22:01:57.696 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T22:01:57.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:57 smithi171 ceph-mon[34138]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:01:58.068 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:01:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:57 smithi090 ceph-mon[30350]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:01:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:57 smithi154 ceph-mon[34495]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:01:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.603+0000 7fb4cf200640 1 -- 172.21.15.90:0/3917437091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c8104f90 msgr2=0x7fb4c8105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.603+0000 7fb4cf200640 1 --2- 172.21.15.90:0/3917437091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c8104f90 0x7fb4c8105370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8009900 tx=0x7fb4b802a410 comp rx=0 tx=0).stop 2024-09-10T22:01:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.603+0000 7fb4cf200640 1 -- 172.21.15.90:0/3917437091 shutdown_connections 2024-09-10T22:01:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.603+0000 7fb4cf200640 1 --2- 172.21.15.90:0/3917437091 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4c810b320 0x7fb4c8111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.603+0000 7fb4cf200640 1 --2- 172.21.15.90:0/3917437091 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4c810a9a0 0x7fb4c810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.603+0000 7fb4cf200640 1 --2- 172.21.15.90:0/3917437091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c8104f90 0x7fb4c8105370 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.603+0000 7fb4cf200640 1 -- 172.21.15.90:0/3917437091 >> 172.21.15.90:0/3917437091 conn(0x7fb4c8100a00 msgr2=0x7fb4c8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.603+0000 7fb4cf200640 1 -- 172.21.15.90:0/3917437091 shutdown_connections 2024-09-10T22:01:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.603+0000 7fb4cf200640 1 -- 172.21.15.90:0/3917437091 wait complete. 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.604+0000 7fb4cf200640 1 Processor -- start 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.604+0000 7fb4cf200640 1 -- start start 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.604+0000 7fb4cf200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4c8104f90 0x7fb4c819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.604+0000 7fb4cf200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c810a9a0 0x7fb4c819b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.604+0000 7fb4cf200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4c810b320 0x7fb4c81a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.604+0000 7fb4cf200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8114e50 con 0x7fb4c810a9a0 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.604+0000 7fb4cf200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8114cd0 con 0x7fb4c8104f90 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.604+0000 7fb4cf200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c8114fd0 con 0x7fb4c810b320 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.604+0000 7fb4c7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c810a9a0 0x7fb4c819b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c810a9a0 0x7fb4c819b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51508/0 (socket says 172.21.15.90:51508) 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c7400640 1 -- 172.21.15.90:0/4236979201 learned_addr learned my addr 172.21.15.90:0/4236979201 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c7e00640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4c8104f90 0x7fb4c819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4cd200640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4c810b320 0x7fb4c81a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c7400640 1 -- 172.21.15.90:0/4236979201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4c810b320 msgr2=0x7fb4c81a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c7400640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4c810b320 0x7fb4c81a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c7400640 1 -- 172.21.15.90:0/4236979201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4c8104f90 msgr2=0x7fb4c819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c7400640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4c8104f90 0x7fb4c819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c7400640 1 -- 172.21.15.90:0/4236979201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c81a14b0 con 0x7fb4c810a9a0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c7400640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c810a9a0 0x7fb4c819b940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e9f0 tx=0x7fb4b400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c4c00640 1 -- 172.21.15.90:0/4236979201 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400cd80 con 0x7fb4c810a9a0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c4c00640 1 -- 172.21.15.90:0/4236979201 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4b40054b0 con 0x7fb4c810a9a0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.605+0000 7fb4c4c00640 1 -- 172.21.15.90:0/4236979201 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4010430 con 0x7fb4c810a9a0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.606+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c81a17a0 con 0x7fb4c810a9a0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.606+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c81a7e60 con 0x7fb4c810a9a0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.607+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c8105370 con 0x7fb4c810a9a0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.609+0000 7fb4c4c00640 1 -- 172.21.15.90:0/4236979201 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb4b4005000 con 0x7fb4c810a9a0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.611+0000 7fb4c4c00640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4ac0b53c0 0x7fb4ac0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.611+0000 7fb4c4c00640 1 -- 172.21.15.90:0/4236979201 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fb4b40e68a0 con 0x7fb4c810a9a0 2024-09-10T22:01:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.612+0000 7fb4c7e00640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4ac0b53c0 0x7fb4ac0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:01:58.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.612+0000 7fb4c7e00640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4ac0b53c0 0x7fb4ac0b7880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8006010 tx=0x7fb4b8005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:01:58.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.613+0000 7fb4c4c00640 1 -- 172.21.15.90:0/4236979201 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb4b4056220 con 0x7fb4c810a9a0 2024-09-10T22:01:58.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:58 smithi090 ceph-mon[30350]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-09-10T22:01:58.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.877+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fb4c802e5c0 con 0x7fb4c810a9a0 2024-09-10T22:01:58.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.880+0000 7fb4c4c00640 1 -- 172.21.15.90:0/4236979201 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v33) ==== 170+0+59 (secure 0 0 0) 0x7fb4b40ad180 con 0x7fb4c810a9a0 2024-09-10T22:01:58.881 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-09-10T22:01:58.881 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDWweBmuBhhNBAANHmkWW4orJ3xBLpr2tKLaQ== 2024-09-10T22:01:58.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4ac0b53c0 msgr2=0x7fb4ac0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:58.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4ac0b53c0 0x7fb4ac0b7880 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8006010 tx=0x7fb4b8005e80 comp rx=0 tx=0).stop 2024-09-10T22:01:58.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c810a9a0 msgr2=0x7fb4c819b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:01:58.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c810a9a0 0x7fb4c819b940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400e9f0 tx=0x7fb4b400edf0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 shutdown_connections 2024-09-10T22:01:58.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb4ac0b53c0 0x7fb4ac0b7880 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb4c810b320 0x7fb4c81a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb4c810a9a0 0x7fb4c819b940 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 --2- 172.21.15.90:0/4236979201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb4c8104f90 0x7fb4c819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:01:58.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 >> 172.21.15.90:0/4236979201 conn(0x7fb4c8100a00 msgr2=0x7fb4c8102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:01:58.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 shutdown_connections 2024-09-10T22:01:58.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:01:58.882+0000 7fb4cf200640 1 -- 172.21.15.90:0/4236979201 wait complete. 2024-09-10T22:01:58.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:58 smithi171 ceph-mon[34138]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-09-10T22:01:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:58 smithi154 ceph-mon[34495]: pgmap v358: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.4 KiB/s wr, 10 op/s 2024-09-10T22:01:59.669 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T22:01:59.669 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T22:01:59.670 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T22:01:59.751 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-10T22:01:59.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:59 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4236979201' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:01:59.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:01:59 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4236979201' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:02:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:59 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4236979201' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:02:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:01:59 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4236979201' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:02:00.159 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:59 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4236979201' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T22:02:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:01:59 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4236979201' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T22:02:00.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.790+0000 7f4e5c400640 1 -- 172.21.15.90:0/2018452043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54101220 msgr2=0x7f4e54101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.790+0000 7f4e5c400640 1 --2- 172.21.15.90:0/2018452043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54101220 0x7f4e54101600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4e48009ca0 tx=0x7f4e4802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.790+0000 7f4e5c400640 1 -- 172.21.15.90:0/2018452043 shutdown_connections 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.790+0000 7f4e5c400640 1 --2- 172.21.15.90:0/2018452043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e54105ef0 0x7f4e5410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.790+0000 7f4e5c400640 1 --2- 172.21.15.90:0/2018452043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e54105570 0x7f4e541059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.790+0000 7f4e5c400640 1 --2- 172.21.15.90:0/2018452043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54101220 0x7f4e54101600 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.790+0000 7f4e5c400640 1 -- 172.21.15.90:0/2018452043 >> 172.21.15.90:0/2018452043 conn(0x7f4e54078100 msgr2=0x7f4e540ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.790+0000 7f4e5c400640 1 -- 172.21.15.90:0/2018452043 shutdown_connections 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.790+0000 7f4e5c400640 1 -- 172.21.15.90:0/2018452043 wait complete. 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.791+0000 7f4e5c400640 1 Processor -- start 2024-09-10T22:02:00.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.791+0000 7f4e5c400640 1 -- start start 2024-09-10T22:02:00.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.791+0000 7f4e5c400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e54101220 0x7f4e5419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:00.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.791+0000 7f4e5c400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e54105570 0x7f4e5419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:00.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.791+0000 7f4e5c400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54105ef0 0x7f4e541a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:00.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.791+0000 7f4e5c400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54112cf0 con 0x7f4e54105ef0 2024-09-10T22:02:00.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.791+0000 7f4e5c400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54112b70 con 0x7f4e54101220 2024-09-10T22:02:00.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.791+0000 7f4e5c400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54112e70 con 0x7f4e54105570 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e5a400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54105ef0 0x7f4e541a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e5a400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54105ef0 0x7f4e541a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51528/0 (socket says 172.21.15.90:51528) 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e5a400640 1 -- 172.21.15.90:0/1664358711 learned_addr learned my addr 172.21.15.90:0/1664358711 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e59a00640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e54101220 0x7f4e5419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e59000640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e54105570 0x7f4e5419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e5a400640 1 -- 172.21.15.90:0/1664358711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e54105570 msgr2=0x7f4e5419b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e5a400640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e54105570 0x7f4e5419b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e5a400640 1 -- 172.21.15.90:0/1664358711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e54101220 msgr2=0x7f4e5419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e5a400640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e54101220 0x7f4e5419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.792+0000 7f4e5a400640 1 -- 172.21.15.90:0/1664358711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e541a14c0 con 0x7f4e54105ef0 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.793+0000 7f4e5a400640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54105ef0 0x7f4e541a0cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00d750 tx=0x7f4e4c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.793+0000 7f4e52000640 1 -- 172.21.15.90:0/1664358711 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c017070 con 0x7f4e54105ef0 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.793+0000 7f4e52000640 1 -- 172.21.15.90:0/1664358711 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e4c0054b0 con 0x7f4e54105ef0 2024-09-10T22:02:00.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.793+0000 7f4e52000640 1 -- 172.21.15.90:0/1664358711 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c004b90 con 0x7f4e54105ef0 2024-09-10T22:02:00.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.793+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e541a17b0 con 0x7f4e54105ef0 2024-09-10T22:02:00.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.793+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541a7e70 con 0x7f4e54105ef0 2024-09-10T22:02:00.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.794+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e54101600 con 0x7f4e54105ef0 2024-09-10T22:02:00.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.797+0000 7f4e52000640 1 -- 172.21.15.90:0/1664358711 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4e4c004d30 con 0x7f4e54105ef0 2024-09-10T22:02:00.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.798+0000 7f4e52000640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e3c0b5440 0x7f4e3c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:00.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.798+0000 7f4e52000640 1 -- 172.21.15.90:0/1664358711 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f4e4c0e5b20 con 0x7f4e54105ef0 2024-09-10T22:02:00.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.798+0000 7f4e59a00640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e3c0b5440 0x7f4e3c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:00.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.799+0000 7f4e59a00640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e3c0b5440 0x7f4e3c0b7900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4e48006010 tx=0x7f4e48031000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:00.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.802+0000 7f4e52000640 1 -- 172.21.15.90:0/1664358711 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4e4c0abdb0 con 0x7f4e54105ef0 2024-09-10T22:02:00.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.986+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f4e541a82d0 con 0x7f4e54105ef0 2024-09-10T22:02:00.988 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:00 smithi090 ceph-mon[30350]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-10T22:02:00.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.987+0000 7f4e52000640 1 -- 172.21.15.90:0/1664358711 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v51) ==== 86+0+133 (secure 0 0 0) 0x7f4e4c0b07e0 con 0x7f4e54105ef0 2024-09-10T22:02:00.989 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e3c0b5440 msgr2=0x7f4e3c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e3c0b5440 0x7f4e3c0b7900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4e48006010 tx=0x7f4e48031000 comp rx=0 tx=0).stop 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54105ef0 msgr2=0x7f4e541a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54105ef0 0x7f4e541a0cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c00d750 tx=0x7f4e4c00dc10 comp rx=0 tx=0).stop 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 shutdown_connections 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e3c0b5440 0x7f4e3c0b7900 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e54105ef0 0x7f4e541a0cf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e54105570 0x7f4e5419b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 --2- 172.21.15.90:0/1664358711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e54101220 0x7f4e5419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 >> 172.21.15.90:0/1664358711 conn(0x7f4e54078100 msgr2=0x7f4e54107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 shutdown_connections 2024-09-10T22:02:00.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:00.989+0000 7f4e5c400640 1 -- 172.21.15.90:0/1664358711 wait complete. 2024-09-10T22:02:00.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:00 smithi171 ceph-mon[34138]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-10T22:02:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:00 smithi154 ceph-mon[34495]: pgmap v359: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-09-10T22:02:01.738 INFO:teuthology.orchestra.run.smithi090.stdout:[{"version":51,"timestamp":"2024-09-10T22:00:23.667180+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-10T22:02:01.738 INFO:tasks.ceph_manager:config epoch is 51 2024-09-10T22:02:01.738 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T22:02:01.738 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T22:02:01.739 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T22:02:01.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:01 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1664358711' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:02:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:01 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1664358711' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:02:02.108 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:02.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:01 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1664358711' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T22:02:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.755+0000 7fe7e7e00640 1 -- 172.21.15.90:0/3184434601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108620 msgr2=0x7fe7e8108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.755+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/3184434601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108620 0x7fe7e8108a60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009900 tx=0x7fe7d402a410 comp rx=0 tx=0).stop 2024-09-10T22:02:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.755+0000 7fe7e7e00640 1 -- 172.21.15.90:0/3184434601 shutdown_connections 2024-09-10T22:02:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.755+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/3184434601 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7e8108fa0 0x7fe7e810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.755+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/3184434601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108620 0x7fe7e8108a60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.755+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/3184434601 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7e8102c10 0x7fe7e8102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.756+0000 7fe7e7e00640 1 -- 172.21.15.90:0/3184434601 >> 172.21.15.90:0/3184434601 conn(0x7fe7e80fe6c0 msgr2=0x7fe7e8100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.756+0000 7fe7e7e00640 1 -- 172.21.15.90:0/3184434601 shutdown_connections 2024-09-10T22:02:02.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.756+0000 7fe7e7e00640 1 -- 172.21.15.90:0/3184434601 wait complete. 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.757+0000 7fe7e7e00640 1 Processor -- start 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.757+0000 7fe7e7e00640 1 -- start start 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.757+0000 7fe7e7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7e8102c10 0x7fe7e80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.757+0000 7fe7e7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7e8108620 0x7fe7e80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.757+0000 7fe7e7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108fa0 0x7fe7e8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.757+0000 7fe7e7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e8112b20 con 0x7fe7e8108fa0 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.757+0000 7fe7e7e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e81129a0 con 0x7fe7e8102c10 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.757+0000 7fe7e7e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e8112ca0 con 0x7fe7e8108620 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108fa0 0x7fe7e8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108fa0 0x7fe7e8075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51552/0 (socket says 172.21.15.90:51552) 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e7400640 1 -- 172.21.15.90:0/1353673226 learned_addr learned my addr 172.21.15.90:0/1353673226 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e6000640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7e8108620 0x7fe7e80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e6a00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7e8102c10 0x7fe7e80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:02.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e7400640 1 -- 172.21.15.90:0/1353673226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7e8108620 msgr2=0x7fe7e80759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e7400640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7e8108620 0x7fe7e80759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e7400640 1 -- 172.21.15.90:0/1353673226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7e8102c10 msgr2=0x7fe7e80754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e7400640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7e8102c10 0x7fe7e80754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e7400640 1 -- 172.21.15.90:0/1353673226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e81a7370 con 0x7fe7e8108fa0 2024-09-10T22:02:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e6a00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7e8102c10 0x7fe7e80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:02.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.758+0000 7fe7e7400640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108fa0 0x7fe7e8075f20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000d750 tx=0x7fe7e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.759+0000 7fe7eca00640 1 -- 172.21.15.90:0/1353673226 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0017070 con 0x7fe7e8108fa0 2024-09-10T22:02:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.759+0000 7fe7eca00640 1 -- 172.21.15.90:0/1353673226 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7e00054b0 con 0x7fe7e8108fa0 2024-09-10T22:02:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.759+0000 7fe7eca00640 1 -- 172.21.15.90:0/1353673226 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0004b90 con 0x7fe7e8108fa0 2024-09-10T22:02:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.759+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e81a7660 con 0x7fe7e8108fa0 2024-09-10T22:02:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.759+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e81a7b20 con 0x7fe7e8108fa0 2024-09-10T22:02:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.760+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7e8102ff0 con 0x7fe7e8108fa0 2024-09-10T22:02:02.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.762+0000 7fe7eca00640 1 -- 172.21.15.90:0/1353673226 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fe7e0004d30 con 0x7fe7e8108fa0 2024-09-10T22:02:02.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.763+0000 7fe7eca00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe7cc0b5460 0x7fe7cc0b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:02.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.764+0000 7fe7eca00640 1 -- 172.21.15.90:0/1353673226 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fe7e00e5c10 con 0x7fe7e8108fa0 2024-09-10T22:02:02.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.764+0000 7fe7e6a00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe7cc0b5460 0x7fe7cc0b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:02.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.764+0000 7fe7e6a00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe7cc0b5460 0x7fe7cc0b7920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8005fd0 tx=0x7fe7d8009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:02.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.767+0000 7fe7eca00640 1 -- 172.21.15.90:0/1353673226 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe7e00ac4f0 con 0x7fe7e8108fa0 2024-09-10T22:02:02.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:02 smithi171 ceph-mon[34138]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-10T22:02:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:02.998+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe7e802e5c0 con 0x7fe7e8108fa0 2024-09-10T22:02:02.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:02 smithi090 ceph-mon[30350]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-10T22:02:03.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.003+0000 7fe7eca00640 1 -- 172.21.15.90:0/1353673226 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+294230 (secure 0 0 0) 0x7fe7e00abdf0 con 0x7fe7e8108fa0 2024-09-10T22:02:03.009 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:03.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe7cc0b5460 msgr2=0x7fe7cc0b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:03.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe7cc0b5460 0x7fe7cc0b7920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8005fd0 tx=0x7fe7d8009600 comp rx=0 tx=0).stop 2024-09-10T22:02:03.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108fa0 msgr2=0x7fe7e8075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:03.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108fa0 0x7fe7e8075f20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe7e000d750 tx=0x7fe7e000dc10 comp rx=0 tx=0).stop 2024-09-10T22:02:03.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 shutdown_connections 2024-09-10T22:02:03.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe7cc0b5460 0x7fe7cc0b7920 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:03.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7e8108fa0 0x7fe7e8075f20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:03.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7e8108620 0x7fe7e80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:03.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 --2- 172.21.15.90:0/1353673226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7e8102c10 0x7fe7e80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:03.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 >> 172.21.15.90:0/1353673226 conn(0x7fe7e80fe6c0 msgr2=0x7fe7e8100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:03.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 shutdown_connections 2024-09-10T22:02:03.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:03.008+0000 7fe7e7e00640 1 -- 172.21.15.90:0/1353673226 wait complete. 2024-09-10T22:02:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:02 smithi154 ceph-mon[34495]: pgmap v360: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-10T22:02:03.772 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":21,"flags":0,"active_gid":14223,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3093492804},{"type":"v1","addr":"172.21.15.90:6801","nonce":3093492804}]},"active_addr":"172.21.15.90:6801/3093492804","active_change":"2024-09-10T21:52:33.477512+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24166,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24196,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.90:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":639385352}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":4213121460}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":333946189}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2933192978}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":3684682243}]}]} 2024-09-10T22:02:03.779 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T22:02:03.779 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T22:02:03.780 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:02:03.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:03 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1353673226' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:02:04.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:03 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1353673226' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:02:04.149 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:03 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1353673226' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T22:02:04.811 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:04 smithi090 ceph-mon[30350]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T22:02:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.809+0000 7f52cd400640 1 -- 172.21.15.90:0/2954943076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810a9a0 msgr2=0x7f52c810ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.809+0000 7f52cd400640 1 --2- 172.21.15.90:0/2954943076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810a9a0 0x7f52c810ade0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009900 tx=0x7f52b402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.810+0000 7f52cd400640 1 -- 172.21.15.90:0/2954943076 shutdown_connections 2024-09-10T22:02:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.810+0000 7f52cd400640 1 --2- 172.21.15.90:0/2954943076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c810b320 0x7f52c8111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.810+0000 7f52cd400640 1 --2- 172.21.15.90:0/2954943076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810a9a0 0x7f52c810ade0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.810+0000 7f52cd400640 1 --2- 172.21.15.90:0/2954943076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c8104f90 0x7f52c8105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.810+0000 7f52cd400640 1 -- 172.21.15.90:0/2954943076 >> 172.21.15.90:0/2954943076 conn(0x7f52c8100a00 msgr2=0x7f52c8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.810+0000 7f52cd400640 1 -- 172.21.15.90:0/2954943076 shutdown_connections 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.810+0000 7f52cd400640 1 -- 172.21.15.90:0/2954943076 wait complete. 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52cd400640 1 Processor -- start 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52cd400640 1 -- start start 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52cd400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c8104f90 0x7f52c819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52cd400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c810a9a0 0x7f52c819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52cd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810b320 0x7f52c81a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52cd400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8114e50 con 0x7f52c810b320 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52cd400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8114cd0 con 0x7f52c810a9a0 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52cd400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8114fd0 con 0x7f52c8104f90 2024-09-10T22:02:04.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52c7200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810b320 0x7f52c81a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52c7200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810b320 0x7f52c81a0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51568/0 (socket says 172.21.15.90:51568) 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.811+0000 7f52c7200640 1 -- 172.21.15.90:0/1449299619 learned_addr learned my addr 172.21.15.90:0/1449299619 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52c6800640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c8104f90 0x7f52c819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52c5e00640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c810a9a0 0x7f52c819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52c7200640 1 -- 172.21.15.90:0/1449299619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c8104f90 msgr2=0x7f52c819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52c7200640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c8104f90 0x7f52c819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52c7200640 1 -- 172.21.15.90:0/1449299619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c810a9a0 msgr2=0x7f52c819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52c7200640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c810a9a0 0x7f52c819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52c7200640 1 -- 172.21.15.90:0/1449299619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c81a1410 con 0x7f52c810b320 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52c7200640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810b320 0x7f52c81a0c40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f52c000eaa0 tx=0x7f52c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52bf400640 1 -- 172.21.15.90:0/1449299619 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c000cdb0 con 0x7f52c810b320 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.812+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c81a1700 con 0x7f52c810b320 2024-09-10T22:02:04.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.813+0000 7f52bf400640 1 -- 172.21.15.90:0/1449299619 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52c0005020 con 0x7f52c810b320 2024-09-10T22:02:04.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.813+0000 7f52bf400640 1 -- 172.21.15.90:0/1449299619 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00116d0 con 0x7f52c810b320 2024-09-10T22:02:04.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.813+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c81a7f90 con 0x7f52c810b320 2024-09-10T22:02:04.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.814+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c8106240 con 0x7f52c810b320 2024-09-10T22:02:04.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.816+0000 7f52bf400640 1 -- 172.21.15.90:0/1449299619 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f52c000f040 con 0x7f52c810b320 2024-09-10T22:02:04.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.817+0000 7f52bf400640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52ac0b53c0 0x7f52ac0b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:04.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.817+0000 7f52bf400640 1 -- 172.21.15.90:0/1449299619 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f52c00e6150 con 0x7f52c810b320 2024-09-10T22:02:04.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.817+0000 7f52c6800640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52ac0b53c0 0x7f52ac0b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:04.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.818+0000 7f52c6800640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52ac0b53c0 0x7f52ac0b7880 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52b8005e10 tx=0x7f52b8005da0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:04.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:04.821+0000 7f52bf400640 1 -- 172.21.15.90:0/1449299619 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f52c0016030 con 0x7f52c810b320 2024-09-10T22:02:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:04 smithi171 ceph-mon[34138]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T22:02:05.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.005+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52c810d100 con 0x7f52c810b320 2024-09-10T22:02:05.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.006+0000 7f52bf400640 1 -- 172.21.15.90:0/1449299619 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25040 (secure 0 0 0) 0x7f52c00aca30 con 0x7f52c810b320 2024-09-10T22:02:05.008 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:05.008 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:01:35.182168+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.008+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52ac0b53c0 msgr2=0x7f52ac0b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.008+0000 7f52cd400640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52ac0b53c0 0x7f52ac0b7880 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f52b8005e10 tx=0x7f52b8005da0 comp rx=0 tx=0).stop 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810b320 msgr2=0x7f52c81a0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810b320 0x7f52c81a0c40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f52c000eaa0 tx=0x7f52c000ef60 comp rx=0 tx=0).stop 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 shutdown_connections 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52ac0b53c0 0x7f52ac0b7880 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c810b320 0x7f52c81a0c40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c810a9a0 0x7f52c819b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 --2- 172.21.15.90:0/1449299619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c8104f90 0x7f52c819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 >> 172.21.15.90:0/1449299619 conn(0x7f52c8100a00 msgr2=0x7f52c8102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:05.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 shutdown_connections 2024-09-10T22:02:05.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:05.009+0000 7f52cd400640 1 -- 172.21.15.90:0/1449299619 wait complete. 2024-09-10T22:02:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:04 smithi154 ceph-mon[34495]: pgmap v361: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T22:02:05.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:05 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1449299619' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:02:05.923 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T22:02:05.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:02:05.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:05 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1449299619' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:02:06.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:05 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1449299619' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:02:06.291 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.976+0000 7fbabf400640 1 -- 172.21.15.90:0/4024867831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 msgr2=0x7fbab81097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.976+0000 7fbabf400640 1 --2- 172.21.15.90:0/4024867831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 0x7fbab81097b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009900 tx=0x7fbaa802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.976+0000 7fbabf400640 1 -- 172.21.15.90:0/4024867831 shutdown_connections 2024-09-10T22:02:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.976+0000 7fbabf400640 1 --2- 172.21.15.90:0/4024867831 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab810f760 0x7fbab8115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.976+0000 7fbabf400640 1 --2- 172.21.15.90:0/4024867831 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbab810ede0 0x7fbab810f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:06.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.976+0000 7fbabf400640 1 --2- 172.21.15.90:0/4024867831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 0x7fbab81097b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.976+0000 7fbabf400640 1 -- 172.21.15.90:0/4024867831 >> 172.21.15.90:0/4024867831 conn(0x7fbab8078a40 msgr2=0x7fbab807ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.976+0000 7fbabf400640 1 -- 172.21.15.90:0/4024867831 shutdown_connections 2024-09-10T22:02:06.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.976+0000 7fbabf400640 1 -- 172.21.15.90:0/4024867831 wait complete. 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.977+0000 7fbabf400640 1 Processor -- start 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.977+0000 7fbabf400640 1 -- start start 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 0x7fbab819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabf400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbab810ede0 0x7fbab819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabf400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab810f760 0x7fbab81a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabf400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8119270 con 0x7fbab81093d0 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabf400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81190f0 con 0x7fbab810f760 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabf400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81193f0 con 0x7fbab810ede0 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 0x7fbab819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbab7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbab810ede0 0x7fbab819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:06.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 0x7fbab819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51588/0 (socket says 172.21.15.90:51588) 2024-09-10T22:02:06.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabca00640 1 -- 172.21.15.90:0/2604378532 learned_addr learned my addr 172.21.15.90:0/2604378532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:06.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabd400640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab810f760 0x7fbab81a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:06.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabca00640 1 -- 172.21.15.90:0/2604378532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbab810ede0 msgr2=0x7fbab819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:06.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabca00640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbab810ede0 0x7fbab819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:06.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabca00640 1 -- 172.21.15.90:0/2604378532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab810f760 msgr2=0x7fbab81a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:06.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabca00640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab810f760 0x7fbab81a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:06.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.978+0000 7fbabca00640 1 -- 172.21.15.90:0/2604378532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab81a5850 con 0x7fbab81093d0 2024-09-10T22:02:06.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.979+0000 7fbabca00640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 0x7fbab819f7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbaa802a910 tx=0x7fbaa8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:06.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.979+0000 7fbab5600640 1 -- 172.21.15.90:0/2604378532 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa803f070 con 0x7fbab81093d0 2024-09-10T22:02:06.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.979+0000 7fbab5600640 1 -- 172.21.15.90:0/2604378532 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbaa8003e40 con 0x7fbab81093d0 2024-09-10T22:02:06.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.979+0000 7fbab5600640 1 -- 172.21.15.90:0/2604378532 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8035030 con 0x7fbab81093d0 2024-09-10T22:02:06.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.979+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab81a5ae0 con 0x7fbab81093d0 2024-09-10T22:02:06.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.979+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab81ac200 con 0x7fbab81093d0 2024-09-10T22:02:06.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.980+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab81097b0 con 0x7fbab81093d0 2024-09-10T22:02:06.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.982+0000 7fbab5600640 1 -- 172.21.15.90:0/2604378532 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fbaa803b870 con 0x7fbab81093d0 2024-09-10T22:02:06.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.985+0000 7fbab5600640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fba9c0b5440 0x7fba9c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.985+0000 7fbab5600640 1 -- 172.21.15.90:0/2604378532 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fbaa8106ff0 con 0x7fbab81093d0 2024-09-10T22:02:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.985+0000 7fbab7e00640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fba9c0b5440 0x7fba9c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:06.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.986+0000 7fbab7e00640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fba9c0b5440 0x7fba9c0b7900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbab810e9d0 tx=0x7fbaa000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:06.987 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:06 smithi090 ceph-mon[30350]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T22:02:06.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:06.987+0000 7fbab5600640 1 -- 172.21.15.90:0/2604378532 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbaa80cd8f0 con 0x7fbab81093d0 2024-09-10T22:02:06.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:06 smithi171 ceph-mon[34138]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T22:02:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:06 smithi154 ceph-mon[34495]: pgmap v362: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T22:02:07.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.175+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbab81a08d0 con 0x7fbab81093d0 2024-09-10T22:02:07.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.177+0000 7fbab5600640 1 -- 172.21.15.90:0/2604378532 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25040 (secure 0 0 0) 0x7fbaa80cd1f0 con 0x7fbab81093d0 2024-09-10T22:02:07.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:07.179 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":88,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:01:35.182168+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:02:07.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fba9c0b5440 msgr2=0x7fba9c0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:07.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fba9c0b5440 0x7fba9c0b7900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbab810e9d0 tx=0x7fbaa000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:02:07.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 msgr2=0x7fbab819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:07.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 0x7fbab819f7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbaa802a910 tx=0x7fbaa8003c20 comp rx=0 tx=0).stop 2024-09-10T22:02:07.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 shutdown_connections 2024-09-10T22:02:07.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fba9c0b5440 0x7fba9c0b7900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:07.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbab810f760 0x7fbab81a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:07.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbab810ede0 0x7fbab819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:07.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 --2- 172.21.15.90:0/2604378532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab81093d0 0x7fbab819f7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:07.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.179+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 >> 172.21.15.90:0/2604378532 conn(0x7fbab8078a40 msgr2=0x7fbab8079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:07.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.180+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 shutdown_connections 2024-09-10T22:02:07.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:07.180+0000 7fbabf400640 1 -- 172.21.15.90:0/2604378532 wait complete. 2024-09-10T22:02:07.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:07 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2604378532' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:02:07.999 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T22:02:08.000 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T22:02:08.001 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T22:02:08.001 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T22:02:08.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T22:02:08.003 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T22:02:08.004 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T22:02:08.004 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T22:02:08.076 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T22:02:08.077 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T22:02:08.077 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T22:02:08.077 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T22:02:08.080 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:07 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2604378532' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:02:08.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:07 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2604378532' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:02:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:08 smithi090 ceph-mon[30350]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T22:02:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:08 smithi154 ceph-mon[34495]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T22:02:09.178 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.181 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.181 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.188 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.197 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.203 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.219 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.224 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.231 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:08 smithi171 ceph-mon[34138]: pgmap v363: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T22:02:09.251 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:10.837 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:10 smithi090 ceph-mon[30350]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:02:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:10 smithi154 ceph-mon[34495]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:02:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:10 smithi171 ceph-mon[34138]: pgmap v364: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:02:12.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.019+0000 7f763fe00640 1 -- 172.21.15.90:0/3176034515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7640077ff0 msgr2=0x7f76401132a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.019+0000 7f763fe00640 1 --2- 172.21.15.90:0/3176034515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7640077ff0 0x7f76401132a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7628009900 tx=0x7f762802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.020+0000 7f763fe00640 1 -- 172.21.15.90:0/3176034515 shutdown_connections 2024-09-10T22:02:12.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.020+0000 7f763fe00640 1 --2- 172.21.15.90:0/3176034515 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76401137e0 0x7f7640115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.020+0000 7f763fe00640 1 --2- 172.21.15.90:0/3176034515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7640077ff0 0x7f76401132a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.020+0000 7f763fe00640 1 --2- 172.21.15.90:0/3176034515 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76400776d0 0x7f7640077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.020+0000 7f763fe00640 1 -- 172.21.15.90:0/3176034515 >> 172.21.15.90:0/3176034515 conn(0x7f7640100840 msgr2=0x7f7640102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:12.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.020+0000 7f763fe00640 1 -- 172.21.15.90:0/3176034515 shutdown_connections 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.020+0000 7f763fe00640 1 -- 172.21.15.90:0/3176034515 wait complete. 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.021+0000 7f763fe00640 1 Processor -- start 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.021+0000 7f763fe00640 1 -- start start 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76400776d0 0x7f764019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7640077ff0 0x7f764019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76401137e0 0x7f76401a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7640119070 con 0x7f76400776d0 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7640118ef0 con 0x7f76401137e0 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f76401191f0 con 0x7f7640077ff0 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76400776d0 0x7f764019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76400776d0 0x7f764019f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60994/0 (socket says 172.21.15.90:60994) 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763ea00640 1 -- 172.21.15.90:0/1497789427 learned_addr learned my addr 172.21.15.90:0/1497789427 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f7637e00640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7640077ff0 0x7f764019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.022+0000 7f763f400640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76401137e0 0x7f76401a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f763ea00640 1 -- 172.21.15.90:0/1497789427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7640077ff0 msgr2=0x7f764019fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f763ea00640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7640077ff0 0x7f764019fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f763ea00640 1 -- 172.21.15.90:0/1497789427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76401137e0 msgr2=0x7f76401a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f763ea00640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76401137e0 0x7f76401a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f763ea00640 1 -- 172.21.15.90:0/1497789427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76401a5570 con 0x7f76400776d0 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f763f400640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f76401137e0 0x7f76401a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f763ea00640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76400776d0 0x7f764019f4c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f763000d950 tx=0x7f763000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f7637400640 1 -- 172.21.15.90:0/1497789427 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630017070 con 0x7f76400776d0 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f7637400640 1 -- 172.21.15.90:0/1497789427 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7630009c30 con 0x7f76400776d0 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f7637400640 1 -- 172.21.15.90:0/1497789427 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763000f620 con 0x7f76400776d0 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.023+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76401a5860 con 0x7f76400776d0 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.024+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76401abf20 con 0x7f76400776d0 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.025+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7640115ba0 con 0x7f76400776d0 2024-09-10T22:02:12.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.027+0000 7f7637400640 1 -- 172.21.15.90:0/1497789427 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f7630010460 con 0x7f76400776d0 2024-09-10T22:02:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.029+0000 7f7637400640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76240b5460 0x7f76240b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.029+0000 7f7637400640 1 -- 172.21.15.90:0/1497789427 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f76300e6200 con 0x7f76400776d0 2024-09-10T22:02:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.029+0000 7f7637e00640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76240b5460 0x7f76240b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.029+0000 7f7637400640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] conn(0x7f76240c0720 0x7f76240c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.029+0000 7f7637400640 1 -- 172.21.15.90:0/1497789427 --> [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7630010ac0 con 0x7f76240c0720 2024-09-10T22:02:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.029+0000 7f7637400640 1 -- 172.21.15.90:0/1497789427 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f76300e6650 con 0x7f76400776d0 2024-09-10T22:02:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.029+0000 7f763f400640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] conn(0x7f76240c0720 0x7f76240c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.030+0000 7f7637e00640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76240b5460 0x7f76240b7920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76280057c0 tx=0x7f76280058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.030+0000 7f763f400640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] conn(0x7f76240c0720 0x7f76240c2bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.031+0000 7f7637400640 1 -- 172.21.15.90:0/1497789427 <== osd.3 v2:172.21.15.90:6810/995208383 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7630010ac0 con 0x7f76240c0720 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.049+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 --> [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7640078be0 con 0x7f76240c0720 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.050+0000 7f7637400640 1 -- 172.21.15.90:0/1497789427 <== osd.3 v2:172.21.15.90:6810/995208383 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7640078be0 con 0x7f76240c0720 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.050+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] conn(0x7f76240c0720 msgr2=0x7f76240c2bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.050+0000 7f763fe00640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] conn(0x7f76240c0720 0x7f76240c2bc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.050+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76240b5460 msgr2=0x7f76240b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.050+0000 7f763fe00640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f76240b5460 0x7f76240b7920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f76280057c0 tx=0x7f76280058d0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.050+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76400776d0 msgr2=0x7f764019f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.050+0000 7f763fe00640 1 --2- 172.21.15.90:0/1497789427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76400776d0 0x7f764019f4c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f763000d950 tx=0x7f763000de10 comp rx=0 tx=0).stop 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.050+0000 7f763f400640 1 -- 172.21.15.90:0/1497789427 reap_dead start 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.052+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 shutdown_connections 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.052+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 >> 172.21.15.90:0/1497789427 conn(0x7f7640100840 msgr2=0x7f7640102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:12.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.052+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 shutdown_connections 2024-09-10T22:02:12.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.052+0000 7f763fe00640 1 -- 172.21.15.90:0/1497789427 wait complete. 2024-09-10T22:02:12.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.708+0000 7f5679c00640 1 -- 172.21.15.90:0/575494429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5674102dc0 msgr2=0x7f56741031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.708+0000 7f5679c00640 1 --2- 172.21.15.90:0/575494429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5674102dc0 0x7f56741031a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566402a410 comp rx=0 tx=0).stop 2024-09-10T22:02:12.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.708+0000 7f5679c00640 1 -- 172.21.15.90:0/575494429 shutdown_connections 2024-09-10T22:02:12.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.708+0000 7f5679c00640 1 --2- 172.21.15.90:0/575494429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5674109150 0x7f567410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.708+0000 7f5679c00640 1 --2- 172.21.15.90:0/575494429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56741087d0 0x7f5674108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.708+0000 7f5679c00640 1 --2- 172.21.15.90:0/575494429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5674102dc0 0x7f56741031a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.708+0000 7f5679c00640 1 -- 172.21.15.90:0/575494429 >> 172.21.15.90:0/575494429 conn(0x7f56740fe830 msgr2=0x7f5674100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.708+0000 7f5679c00640 1 -- 172.21.15.90:0/575494429 shutdown_connections 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.708+0000 7f5679c00640 1 -- 172.21.15.90:0/575494429 wait complete. 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.709+0000 7f5679c00640 1 Processor -- start 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.709+0000 7f5679c00640 1 -- start start 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.709+0000 7f5679c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5674102dc0 0x7f567419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.709+0000 7f5679c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56741087d0 0x7f567419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.709+0000 7f5679c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5674109150 0x7f56741a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.709+0000 7f5679c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5674112d10 con 0x7f5674109150 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.709+0000 7f5679c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5674112b90 con 0x7f5674102dc0 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.709+0000 7f5679c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5674112e90 con 0x7f56741087d0 2024-09-10T22:02:12.718 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5673400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5674109150 0x7f56741a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5672a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5674102dc0 0x7f567419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5673400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5674109150 0x7f56741a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32786/0 (socket says 172.21.15.90:32786) 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5673400640 1 -- 172.21.15.90:0/1625111240 learned_addr learned my addr 172.21.15.90:0/1625111240 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5672000640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56741087d0 0x7f567419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5673400640 1 -- 172.21.15.90:0/1625111240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56741087d0 msgr2=0x7f567419b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5673400640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f56741087d0 0x7f567419b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5673400640 1 -- 172.21.15.90:0/1625111240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5674102dc0 msgr2=0x7f567419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5673400640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5674102dc0 0x7f567419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5673400640 1 -- 172.21.15.90:0/1625111240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56741a1450 con 0x7f5674109150 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.710+0000 7f5673400640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5674109150 0x7f56741a0c80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f566c00cce0 tx=0x7f566c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.711+0000 7f566b400640 1 -- 172.21.15.90:0/1625111240 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c00f450 con 0x7f5674109150 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.711+0000 7f566b400640 1 -- 172.21.15.90:0/1625111240 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f566c00fa80 con 0x7f5674109150 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.711+0000 7f566b400640 1 -- 172.21.15.90:0/1625111240 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c018ad0 con 0x7f5674109150 2024-09-10T22:02:12.719 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.711+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741a1740 con 0x7f5674109150 2024-09-10T22:02:12.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.711+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56741a7e00 con 0x7f5674109150 2024-09-10T22:02:12.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.712+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f56740638a0 con 0x7f5674109150 2024-09-10T22:02:12.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.714+0000 7f566b400640 1 -- 172.21.15.90:0/1625111240 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f566c00f5f0 con 0x7f5674109150 2024-09-10T22:02:12.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.715+0000 7f566b400640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56580b5440 0x7f56580b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.716+0000 7f566b400640 1 -- 172.21.15.90:0/1625111240 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f566c0e6bf0 con 0x7f5674109150 2024-09-10T22:02:12.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.716+0000 7f5672a00640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56580b5440 0x7f56580b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.716+0000 7f566b400640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] conn(0x7f56580c0700 0x7f56580c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:12.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.716+0000 7f566b400640 1 -- 172.21.15.90:0/1625111240 --> [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f566c012ec0 con 0x7f56580c0700 2024-09-10T22:02:12.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.716+0000 7f566b400640 1 -- 172.21.15.90:0/1625111240 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f566c0e7040 con 0x7f5674109150 2024-09-10T22:02:12.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.717+0000 7f5672a00640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56580b5440 0x7f56580b7900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:12.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.717+0000 7f5672000640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] conn(0x7f56580c0700 0x7f56580c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:12.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.718+0000 7f5672000640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] conn(0x7f56580c0700 0x7f56580c2ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:12.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.719+0000 7f566b400640 1 -- 172.21.15.90:0/1625111240 <== osd.0 v2:172.21.15.90:6802/405981593 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f566c012ec0 con 0x7f56580c0700 2024-09-10T22:02:12.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.733+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 --> [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5674063d20 con 0x7f56580c0700 2024-09-10T22:02:12.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f566b400640 1 -- 172.21.15.90:0/1625111240 <== osd.0 v2:172.21.15.90:6802/405981593 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5674063d20 con 0x7f56580c0700 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] conn(0x7f56580c0700 msgr2=0x7f56580c2ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] conn(0x7f56580c0700 0x7f56580c2ba0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56580b5440 msgr2=0x7f56580b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56580b5440 0x7f56580b7900 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664005e80 comp rx=0 tx=0).stop 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5674109150 msgr2=0x7f56741a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 --2- 172.21.15.90:0/1625111240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5674109150 0x7f56741a0c80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f566c00cce0 tx=0x7f566c00a620 comp rx=0 tx=0).stop 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5673400640 1 -- 172.21.15.90:0/1625111240 reap_dead start 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 shutdown_connections 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 >> 172.21.15.90:0/1625111240 conn(0x7f56740fe830 msgr2=0x7f5674104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 shutdown_connections 2024-09-10T22:02:12.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.734+0000 7f5679c00640 1 -- 172.21.15.90:0/1625111240 wait complete. 2024-09-10T22:02:12.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:12 smithi090 ceph-mon[30350]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:02:13.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.950+0000 7f17b0600640 1 -- 172.21.15.90:0/274830219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a8104fa0 msgr2=0x7f17a8105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.950+0000 7f17b0600640 1 --2- 172.21.15.90:0/274830219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a8104fa0 0x7f17a8105380 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f179c009900 tx=0x7f179c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.951+0000 7f17b0600640 1 -- 172.21.15.90:0/274830219 shutdown_connections 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.951+0000 7f17b0600640 1 --2- 172.21.15.90:0/274830219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a810b330 0x7f17a8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.951+0000 7f17b0600640 1 --2- 172.21.15.90:0/274830219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17a810a9b0 0x7f17a810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.951+0000 7f17b0600640 1 --2- 172.21.15.90:0/274830219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a8104fa0 0x7f17a8105380 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.951+0000 7f17b0600640 1 -- 172.21.15.90:0/274830219 >> 172.21.15.90:0/274830219 conn(0x7f17a8100a50 msgr2=0x7f17a8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.951+0000 7f17b0600640 1 -- 172.21.15.90:0/274830219 shutdown_connections 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.951+0000 7f17b0600640 1 -- 172.21.15.90:0/274830219 wait complete. 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17b0600640 1 Processor -- start 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17b0600640 1 -- start start 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17b0600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17a8104fa0 0x7f17a819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17b0600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a810a9b0 0x7f17a819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17b0600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a810b330 0x7f17a81a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17b0600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8114df0 con 0x7f17a810b330 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17b0600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8114c70 con 0x7f17a8104fa0 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17b0600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8114f70 con 0x7f17a810a9b0 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17ae600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a810b330 0x7f17a81a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17ae600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a810b330 0x7f17a81a0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32810/0 (socket says 172.21.15.90:32810) 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.952+0000 7f17ae600640 1 -- 172.21.15.90:0/759258646 learned_addr learned my addr 172.21.15.90:0/759258646 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17ad200640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a810a9b0 0x7f17a819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17adc00640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17a8104fa0 0x7f17a819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17ae600640 1 -- 172.21.15.90:0/759258646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a810a9b0 msgr2=0x7f17a819b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17ae600640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17a810a9b0 0x7f17a819b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17ae600640 1 -- 172.21.15.90:0/759258646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17a8104fa0 msgr2=0x7f17a819b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17ae600640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17a8104fa0 0x7f17a819b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17ae600640 1 -- 172.21.15.90:0/759258646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a81a1350 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17ae600640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a810b330 0x7f17a81a0b80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f17a000cce0 tx=0x7f17a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17a6000640 1 -- 172.21.15.90:0/759258646 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a000f450 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17a6000640 1 -- 172.21.15.90:0/759258646 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17a000fa80 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a81a1640 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.953+0000 7f17a6000640 1 -- 172.21.15.90:0/759258646 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0018ad0 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.954+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a81a7d00 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.954+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f17a80638e0 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.957+0000 7f17a6000640 1 -- 172.21.15.90:0/759258646 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f17a000f5f0 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.958+0000 7f17a6000640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17900b53c0 0x7f17900b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.958+0000 7f17a6000640 1 -- 172.21.15.90:0/759258646 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f17a00e6da0 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.958+0000 7f17adc00640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17900b53c0 0x7f17900b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.959+0000 7f17a6000640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] conn(0x7f17900c0680 0x7f17900c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.959+0000 7f17a6000640 1 -- 172.21.15.90:0/759258646 --> [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17a0012ec0 con 0x7f17900c0680 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.959+0000 7f17a6000640 1 -- 172.21.15.90:0/759258646 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f17a00e7150 con 0x7f17a810b330 2024-09-10T22:02:13.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.959+0000 7f17ad200640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] conn(0x7f17900c0680 0x7f17900c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.959+0000 7f17adc00640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17900b53c0 0x7f17900b7880 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f179c000c00 tx=0x7f179c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:13.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.960+0000 7f17ad200640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] conn(0x7f17900c0680 0x7f17900c2b20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:13.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.961+0000 7f17a6000640 1 -- 172.21.15.90:0/759258646 <== osd.10 v2:172.21.15.154:6824/1803626214 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17a0012ec0 con 0x7f17900c0680 2024-09-10T22:02:13.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.975+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 --> [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f17a8063d60 con 0x7f17900c0680 2024-09-10T22:02:13.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.975+0000 7f17a6000640 1 -- 172.21.15.90:0/759258646 <== osd.10 v2:172.21.15.154:6824/1803626214 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f17a8063d60 con 0x7f17900c0680 2024-09-10T22:02:13.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 >> [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] conn(0x7f17900c0680 msgr2=0x7f17900c2b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] conn(0x7f17900c0680 0x7f17900c2b20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17900b53c0 msgr2=0x7f17900b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17900b53c0 0x7f17900b7880 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f179c000c00 tx=0x7f179c005a20 comp rx=0 tx=0).stop 2024-09-10T22:02:13.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a810b330 msgr2=0x7f17a81a0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 --2- 172.21.15.90:0/759258646 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17a810b330 0x7f17a81a0b80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f17a000cce0 tx=0x7f17a000a620 comp rx=0 tx=0).stop 2024-09-10T22:02:13.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17ae600640 1 -- 172.21.15.90:0/759258646 reap_dead start 2024-09-10T22:02:13.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 shutdown_connections 2024-09-10T22:02:13.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 >> 172.21.15.90:0/759258646 conn(0x7f17a8100a50 msgr2=0x7f17a81012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:13.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 shutdown_connections 2024-09-10T22:02:13.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:12.976+0000 7f17b0600640 1 -- 172.21.15.90:0/759258646 wait complete. 2024-09-10T22:02:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:12 smithi154 ceph-mon[34495]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:02:13.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:12 smithi171 ceph-mon[34138]: pgmap v365: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:02:13.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.745+0000 7fbba0e00640 1 -- 172.21.15.90:0/1107872394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb9c108fa0 msgr2=0x7fbb9c10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.745+0000 7fbba0e00640 1 --2- 172.21.15.90:0/1107872394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb9c108fa0 0x7fbb9c10f7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbb94009900 tx=0x7fbb9402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.745+0000 7fbba0e00640 1 -- 172.21.15.90:0/1107872394 shutdown_connections 2024-09-10T22:02:13.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.745+0000 7fbba0e00640 1 --2- 172.21.15.90:0/1107872394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb9c108fa0 0x7fbb9c10f7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.745+0000 7fbba0e00640 1 --2- 172.21.15.90:0/1107872394 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9c108620 0x7fbb9c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.745+0000 7fbba0e00640 1 --2- 172.21.15.90:0/1107872394 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c102c10 0x7fbb9c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.745+0000 7fbba0e00640 1 -- 172.21.15.90:0/1107872394 >> 172.21.15.90:0/1107872394 conn(0x7fbb9c0fe6c0 msgr2=0x7fbb9c100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:13.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.745+0000 7fbba0e00640 1 -- 172.21.15.90:0/1107872394 shutdown_connections 2024-09-10T22:02:13.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.745+0000 7fbba0e00640 1 -- 172.21.15.90:0/1107872394 wait complete. 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.746+0000 7fbba0e00640 1 Processor -- start 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.746+0000 7fbba0e00640 1 -- start start 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.746+0000 7fbba0e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9c102c10 0x7fbb9c19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.746+0000 7fbba0e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c108620 0x7fbb9c19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.746+0000 7fbba0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb9c108fa0 0x7fbb9c1a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.746+0000 7fbba0e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c112b20 con 0x7fbb9c108fa0 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.746+0000 7fbba0e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c1129a0 con 0x7fbb9c108620 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.746+0000 7fbba0e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c112ca0 con 0x7fbb9c102c10 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb9c108fa0 0x7fbb9c1a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb9c108fa0 0x7fbb9c1a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32832/0 (socket says 172.21.15.90:32832) 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9be00640 1 -- 172.21.15.90:0/1901109219 learned_addr learned my addr 172.21.15.90:0/1901109219 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9aa00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c108620 0x7fbb9c19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9b400640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9c102c10 0x7fbb9c19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9be00640 1 -- 172.21.15.90:0/1901109219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9c102c10 msgr2=0x7fbb9c19b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9be00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbb9c102c10 0x7fbb9c19b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9be00640 1 -- 172.21.15.90:0/1901109219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c108620 msgr2=0x7fbb9c19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9be00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbb9c108620 0x7fbb9c19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.747+0000 7fbb9be00640 1 -- 172.21.15.90:0/1901109219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c1a11c0 con 0x7fbb9c108fa0 2024-09-10T22:02:13.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.748+0000 7fbb9be00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb9c108fa0 0x7fbb9c1a09f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbb94004770 tx=0x7fbb94003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:13.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.748+0000 7fbb93e00640 1 -- 172.21.15.90:0/1901109219 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9403f070 con 0x7fbb9c108fa0 2024-09-10T22:02:13.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.748+0000 7fbb93e00640 1 -- 172.21.15.90:0/1901109219 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbb9403bd00 con 0x7fbb9c108fa0 2024-09-10T22:02:13.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.748+0000 7fbb93e00640 1 -- 172.21.15.90:0/1901109219 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb94035030 con 0x7fbb9c108fa0 2024-09-10T22:02:13.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.748+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c1a1450 con 0x7fbb9c108fa0 2024-09-10T22:02:13.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.748+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c1a7b70 con 0x7fbb9c108fa0 2024-09-10T22:02:13.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.749+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb9c075da0 con 0x7fbb9c108fa0 2024-09-10T22:02:13.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.751+0000 7fbb93e00640 1 -- 172.21.15.90:0/1901109219 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fbb94031bd0 con 0x7fbb9c108fa0 2024-09-10T22:02:13.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.753+0000 7fbb93e00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbb800b54e0 0x7fbb800b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.753+0000 7fbb93e00640 1 -- 172.21.15.90:0/1901109219 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fbb941070e0 con 0x7fbb9c108fa0 2024-09-10T22:02:13.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.753+0000 7fbb9b400640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbb800b54e0 0x7fbb800b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.753+0000 7fbb93e00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] conn(0x7fbb800c07a0 0x7fbb800c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:13.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.753+0000 7fbb93e00640 1 -- 172.21.15.90:0/1901109219 --> [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb800c3310 con 0x7fbb800c07a0 2024-09-10T22:02:13.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.753+0000 7fbb93e00640 1 -- 172.21.15.90:0/1901109219 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fbb94107530 con 0x7fbb9c108fa0 2024-09-10T22:02:13.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.754+0000 7fbb9b400640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbb800b54e0 0x7fbb800b79a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c005fd0 tx=0x7fbb8c009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:13.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.768+0000 7fbb9aa00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] conn(0x7fbb800c07a0 0x7fbb800c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:13.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.768+0000 7fbb9aa00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] conn(0x7fbb800c07a0 0x7fbb800c2c40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:13.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.769+0000 7fbb93e00640 1 -- 172.21.15.90:0/1901109219 <== osd.4 v2:172.21.15.154:6808/1188268169 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb800c3310 con 0x7fbb800c07a0 2024-09-10T22:02:13.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.784+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 --> [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb9c103ec0 con 0x7fbb800c07a0 2024-09-10T22:02:13.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.785+0000 7fbb93e00640 1 -- 172.21.15.90:0/1901109219 <== osd.4 v2:172.21.15.154:6808/1188268169 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbb9c103ec0 con 0x7fbb800c07a0 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.785+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 >> [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] conn(0x7fbb800c07a0 msgr2=0x7fbb800c2c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.785+0000 7fbba0e00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] conn(0x7fbb800c07a0 0x7fbb800c2c40 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.786+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbb800b54e0 msgr2=0x7fbb800b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.786+0000 7fbba0e00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbb800b54e0 0x7fbb800b79a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c005fd0 tx=0x7fbb8c009680 comp rx=0 tx=0).stop 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.786+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb9c108fa0 msgr2=0x7fbb9c1a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.786+0000 7fbba0e00640 1 --2- 172.21.15.90:0/1901109219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb9c108fa0 0x7fbb9c1a09f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbb94004770 tx=0x7fbb94003cd0 comp rx=0 tx=0).stop 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.786+0000 7fbb9be00640 1 -- 172.21.15.90:0/1901109219 reap_dead start 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.786+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 shutdown_connections 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.786+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 >> 172.21.15.90:0/1901109219 conn(0x7fbb9c0fe6c0 msgr2=0x7fbb9c075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.786+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 shutdown_connections 2024-09-10T22:02:13.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:13.786+0000 7fbba0e00640 1 -- 172.21.15.90:0/1901109219 wait complete. 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.015+0000 7f1e83000640 1 -- 172.21.15.90:0/1847778154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e7c103b10 msgr2=0x7f1e7c103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.015+0000 7f1e83000640 1 --2- 172.21.15.90:0/1847778154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e7c103b10 0x7f1e7c103ef0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009900 tx=0x7f1e7002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.015+0000 7f1e83000640 1 -- 172.21.15.90:0/1847778154 shutdown_connections 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.015+0000 7f1e83000640 1 --2- 172.21.15.90:0/1847778154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c10cf70 0x7f1e7c10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.015+0000 7f1e83000640 1 --2- 172.21.15.90:0/1847778154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e7c104430 0x7f1e7c10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.015+0000 7f1e83000640 1 --2- 172.21.15.90:0/1847778154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e7c103b10 0x7f1e7c103ef0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.015+0000 7f1e83000640 1 -- 172.21.15.90:0/1847778154 >> 172.21.15.90:0/1847778154 conn(0x7f1e7c0fd5b0 msgr2=0x7f1e7c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.015+0000 7f1e83000640 1 -- 172.21.15.90:0/1847778154 shutdown_connections 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.015+0000 7f1e83000640 1 -- 172.21.15.90:0/1847778154 wait complete. 2024-09-10T22:02:14.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.016+0000 7f1e83000640 1 Processor -- start 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.016+0000 7f1e83000640 1 -- start start 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e83000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c103b10 0x7f1e7c198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e83000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e7c104430 0x7f1e7c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e83000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e7c10cf70 0x7f1e7c19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e83000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c112ae0 con 0x7f1e7c10cf70 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e83000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c112960 con 0x7f1e7c103b10 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e83000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7c112c60 con 0x7f1e7c104430 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e82600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e7c10cf70 0x7f1e7c19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e82600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e7c10cf70 0x7f1e7c19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32844/0 (socket says 172.21.15.90:32844) 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e82600640 1 -- 172.21.15.90:0/1760161818 learned_addr learned my addr 172.21.15.90:0/1760161818 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e81200640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e7c104430 0x7f1e7c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e81c00640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c103b10 0x7f1e7c198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e82600640 1 -- 172.21.15.90:0/1760161818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e7c104430 msgr2=0x7f1e7c199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e82600640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1e7c104430 0x7f1e7c199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e82600640 1 -- 172.21.15.90:0/1760161818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c103b10 msgr2=0x7f1e7c198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e82600640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c103b10 0x7f1e7c198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e82600640 1 -- 172.21.15.90:0/1760161818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7c19f000 con 0x7f1e7c10cf70 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.017+0000 7f1e81c00640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1e7c103b10 0x7f1e7c198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.018+0000 7f1e82600640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e7c10cf70 0x7f1e7c19e830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009fd0 tx=0x7f1e7400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.018+0000 7f1e7a000640 1 -- 172.21.15.90:0/1760161818 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74017070 con 0x7f1e7c10cf70 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.018+0000 7f1e7a000640 1 -- 172.21.15.90:0/1760161818 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1e74003e00 con 0x7f1e7c10cf70 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.018+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7c19f2f0 con 0x7f1e7c10cf70 2024-09-10T22:02:14.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.018+0000 7f1e7a000640 1 -- 172.21.15.90:0/1760161818 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e74010460 con 0x7f1e7c10cf70 2024-09-10T22:02:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.018+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7c1a59b0 con 0x7f1e7c10cf70 2024-09-10T22:02:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.019+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1e7c0ff510 con 0x7f1e7c10cf70 2024-09-10T22:02:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.021+0000 7f1e7a000640 1 -- 172.21.15.90:0/1760161818 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f1e74003970 con 0x7f1e7c10cf70 2024-09-10T22:02:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.022+0000 7f1e7a000640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1e640b54e0 0x7f1e640b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.023+0000 7f1e7a000640 1 -- 172.21.15.90:0/1760161818 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f1e740e6390 con 0x7f1e7c10cf70 2024-09-10T22:02:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.023+0000 7f1e81c00640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1e640b54e0 0x7f1e640b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.023+0000 7f1e7a000640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] conn(0x7f1e640c07a0 0x7f1e640c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.023+0000 7f1e7a000640 1 -- 172.21.15.90:0/1760161818 --> [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e74015e80 con 0x7f1e640c07a0 2024-09-10T22:02:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.023+0000 7f1e7a000640 1 -- 172.21.15.90:0/1760161818 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1e740e67e0 con 0x7f1e7c10cf70 2024-09-10T22:02:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.023+0000 7f1e81c00640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1e640b54e0 0x7f1e640b79a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e70006010 tx=0x7f1e70005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.024+0000 7f1e81200640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] conn(0x7f1e640c07a0 0x7f1e640c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.024+0000 7f1e81200640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] conn(0x7f1e640c07a0 0x7f1e640c2c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.026+0000 7f1e7a000640 1 -- 172.21.15.90:0/1760161818 <== osd.1 v2:172.21.15.154:6800/2687850784 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1e74015e80 con 0x7f1e640c07a0 2024-09-10T22:02:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.039+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 --> [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1e7c06b1b0 con 0x7f1e640c07a0 2024-09-10T22:02:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.040+0000 7f1e7a000640 1 -- 172.21.15.90:0/1760161818 <== osd.1 v2:172.21.15.154:6800/2687850784 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1e7c06b1b0 con 0x7f1e640c07a0 2024-09-10T22:02:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 >> [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] conn(0x7f1e640c07a0 msgr2=0x7f1e640c2c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] conn(0x7f1e640c07a0 0x7f1e640c2c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1e640b54e0 msgr2=0x7f1e640b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1e640b54e0 0x7f1e640b79a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e70006010 tx=0x7f1e70005a20 comp rx=0 tx=0).stop 2024-09-10T22:02:14.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e7c10cf70 msgr2=0x7f1e7c19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 --2- 172.21.15.90:0/1760161818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1e7c10cf70 0x7f1e7c19e830 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009fd0 tx=0x7f1e7400b260 comp rx=0 tx=0).stop 2024-09-10T22:02:14.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e82600640 1 -- 172.21.15.90:0/1760161818 reap_dead start 2024-09-10T22:02:14.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 shutdown_connections 2024-09-10T22:02:14.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 >> 172.21.15.90:0/1760161818 conn(0x7f1e7c0fd5b0 msgr2=0x7f1e7c10f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:14.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 shutdown_connections 2024-09-10T22:02:14.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.041+0000 7f1e83000640 1 -- 172.21.15.90:0/1760161818 wait complete. 2024-09-10T22:02:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.790+0000 7f0352e00640 1 -- 172.21.15.90:0/1117213274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f034c1045e0 msgr2=0x7f034c10cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.790+0000 7f0352e00640 1 --2- 172.21.15.90:0/1117213274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f034c1045e0 0x7f034c10cad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0334009900 tx=0x7f033402a410 comp rx=0 tx=0).stop 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.790+0000 7f0352e00640 1 -- 172.21.15.90:0/1117213274 shutdown_connections 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.790+0000 7f0352e00640 1 --2- 172.21.15.90:0/1117213274 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f034c10d120 0x7f034c10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.790+0000 7f0352e00640 1 --2- 172.21.15.90:0/1117213274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f034c1045e0 0x7f034c10cad0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.790+0000 7f0352e00640 1 --2- 172.21.15.90:0/1117213274 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034c103cc0 0x7f034c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.790+0000 7f0352e00640 1 -- 172.21.15.90:0/1117213274 >> 172.21.15.90:0/1117213274 conn(0x7f034c0fd700 msgr2=0x7f034c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.790+0000 7f0352e00640 1 -- 172.21.15.90:0/1117213274 shutdown_connections 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.790+0000 7f0352e00640 1 -- 172.21.15.90:0/1117213274 wait complete. 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.791+0000 7f0352e00640 1 Processor -- start 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.791+0000 7f0352e00640 1 -- start start 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.791+0000 7f0352e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034c103cc0 0x7f034c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.791+0000 7f0352e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f034c1045e0 0x7f034c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.791+0000 7f0352e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f034c10d120 0x7f034c1a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.791+0000 7f0352e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f034c112c90 con 0x7f034c1045e0 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.791+0000 7f0352e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f034c112b10 con 0x7f034c103cc0 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.791+0000 7f0352e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f034c112e10 con 0x7f034c10d120 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.791+0000 7f0343e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f034c1045e0 0x7f034c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0343e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f034c1045e0 0x7f034c19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32858/0 (socket says 172.21.15.90:32858) 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0343e00640 1 -- 172.21.15.90:0/2899376289 learned_addr learned my addr 172.21.15.90:0/2899376289 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f034be00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034c103cc0 0x7f034c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0350e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f034c10d120 0x7f034c1a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0343e00640 1 -- 172.21.15.90:0/2899376289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f034c10d120 msgr2=0x7f034c1a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0343e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f034c10d120 0x7f034c1a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0343e00640 1 -- 172.21.15.90:0/2899376289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034c103cc0 msgr2=0x7f034c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0343e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f034c103cc0 0x7f034c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0343e00640 1 -- 172.21.15.90:0/2899376289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c1a1510 con 0x7f034c1045e0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0350e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f034c10d120 0x7f034c1a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.792+0000 7f0343e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f034c1045e0 0x7f034c19b9a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f033402a930 tx=0x7f0334003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.793+0000 7f0349600640 1 -- 172.21.15.90:0/2899376289 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033403f070 con 0x7f034c1045e0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.793+0000 7f0349600640 1 -- 172.21.15.90:0/2899376289 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f033403baf0 con 0x7f034c1045e0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.793+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c1a17a0 con 0x7f034c1045e0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.793+0000 7f0349600640 1 -- 172.21.15.90:0/2899376289 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0334035030 con 0x7f034c1045e0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.793+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c1a7ec0 con 0x7f034c1045e0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.794+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f034c1033b0 con 0x7f034c1045e0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.796+0000 7f0349600640 1 -- 172.21.15.90:0/2899376289 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f033403b660 con 0x7f034c1045e0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.797+0000 7f0349600640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03300b5440 0x7f03300b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.797+0000 7f0349600640 1 -- 172.21.15.90:0/2899376289 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f0334106f80 con 0x7f034c1045e0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.798+0000 7f034be00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03300b5440 0x7f03300b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.798+0000 7f0349600640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] conn(0x7f03300c0700 0x7f03300c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.798+0000 7f0349600640 1 -- 172.21.15.90:0/2899376289 --> [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0334046e50 con 0x7f03300c0700 2024-09-10T22:02:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.798+0000 7f0349600640 1 -- 172.21.15.90:0/2899376289 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0334107330 con 0x7f034c1045e0 2024-09-10T22:02:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.798+0000 7f0350e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] conn(0x7f03300c0700 0x7f03300c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.798+0000 7f034be00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03300b5440 0x7f03300b7900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f033c005d00 tx=0x7f033c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.799+0000 7f0350e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] conn(0x7f03300c0700 0x7f03300c2ba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.799+0000 7f0349600640 1 -- 172.21.15.90:0/2899376289 <== osd.6 v2:172.21.15.90:6818/3973504003 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0334046e50 con 0x7f03300c0700 2024-09-10T22:02:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.814+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 --> [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f034c06bc10 con 0x7f03300c0700 2024-09-10T22:02:14.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.814+0000 7f0349600640 1 -- 172.21.15.90:0/2899376289 <== osd.6 v2:172.21.15.90:6818/3973504003 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f034c06bc10 con 0x7f03300c0700 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.814+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] conn(0x7f03300c0700 msgr2=0x7f03300c2ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.814+0000 7f0352e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] conn(0x7f03300c0700 0x7f03300c2ba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.814+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03300b5440 msgr2=0x7f03300b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.814+0000 7f0352e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03300b5440 0x7f03300b7900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f033c005d00 tx=0x7f033c00b410 comp rx=0 tx=0).stop 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.814+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f034c1045e0 msgr2=0x7f034c19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.814+0000 7f0352e00640 1 --2- 172.21.15.90:0/2899376289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f034c1045e0 0x7f034c19b9a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f033402a930 tx=0x7f0334003b90 comp rx=0 tx=0).stop 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.815+0000 7f0350e00640 1 -- 172.21.15.90:0/2899376289 reap_dead start 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.815+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 shutdown_connections 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.815+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 >> 172.21.15.90:0/2899376289 conn(0x7f034c0fd700 msgr2=0x7f034c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.815+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 shutdown_connections 2024-09-10T22:02:14.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:14.815+0000 7f0352e00640 1 -- 172.21.15.90:0/2899376289 wait complete. 2024-09-10T22:02:14.979 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:14 smithi090 ceph-mon[30350]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-10T22:02:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:14 smithi154 ceph-mon[34495]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-10T22:02:15.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:14 smithi171 ceph-mon[34138]: pgmap v366: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.946+0000 7f8cff000640 1 -- 172.21.15.90:0/1260445820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf810b180 msgr2=0x7f8cf8111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.946+0000 7f8cff000640 1 --2- 172.21.15.90:0/1260445820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf810b180 0x7f8cf8111980 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0009ca0 tx=0x7f8cf002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.946+0000 7f8cff000640 1 -- 172.21.15.90:0/1260445820 shutdown_connections 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.946+0000 7f8cff000640 1 --2- 172.21.15.90:0/1260445820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf810b180 0x7f8cf8111980 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.946+0000 7f8cff000640 1 --2- 172.21.15.90:0/1260445820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf810a800 0x7f8cf810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.946+0000 7f8cff000640 1 --2- 172.21.15.90:0/1260445820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cf8104df0 0x7f8cf81051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.946+0000 7f8cff000640 1 -- 172.21.15.90:0/1260445820 >> 172.21.15.90:0/1260445820 conn(0x7f8cf8100840 msgr2=0x7f8cf8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.946+0000 7f8cff000640 1 -- 172.21.15.90:0/1260445820 shutdown_connections 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.946+0000 7f8cff000640 1 -- 172.21.15.90:0/1260445820 wait complete. 2024-09-10T22:02:15.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.947+0000 7f8cff000640 1 Processor -- start 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cff000640 1 -- start start 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cff000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf8104df0 0x7f8cf819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cff000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf810a800 0x7f8cf819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cff000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cf810b180 0x7f8cf81a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cff000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8114d00 con 0x7f8cf8104df0 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cff000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8114b80 con 0x7f8cf810a800 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cff000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf8114e80 con 0x7f8cf810b180 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cfdc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf8104df0 0x7f8cf819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cfdc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf8104df0 0x7f8cf819b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60950/0 (socket says 172.21.15.90:60950) 2024-09-10T22:02:15.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cfdc00640 1 -- 172.21.15.90:0/1425213795 learned_addr learned my addr 172.21.15.90:0/1425213795 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.948+0000 7f8cfd200640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf810a800 0x7f8cf819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cfe600640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cf810b180 0x7f8cf81a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cfdc00640 1 -- 172.21.15.90:0/1425213795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cf810b180 msgr2=0x7f8cf81a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cfdc00640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8cf810b180 0x7f8cf81a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cfdc00640 1 -- 172.21.15.90:0/1425213795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf810a800 msgr2=0x7f8cf819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cfdc00640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8cf810a800 0x7f8cf819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cfdc00640 1 -- 172.21.15.90:0/1425213795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf81a1200 con 0x7f8cf8104df0 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cfdc00640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf8104df0 0x7f8cf819b150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00e9f0 tx=0x7f8cec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cf6000640 1 -- 172.21.15.90:0/1425213795 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00cd80 con 0x7f8cf8104df0 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cf6000640 1 -- 172.21.15.90:0/1425213795 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8cec0054b0 con 0x7f8cf8104df0 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cf6000640 1 -- 172.21.15.90:0/1425213795 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec010430 con 0x7f8cf8104df0 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.949+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf81a14f0 con 0x7f8cf8104df0 2024-09-10T22:02:15.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.950+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf81a7bb0 con 0x7f8cf8104df0 2024-09-10T22:02:15.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.951+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8cf80638e0 con 0x7f8cf8104df0 2024-09-10T22:02:15.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.953+0000 7f8cf6000640 1 -- 172.21.15.90:0/1425213795 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f8cec005020 con 0x7f8cf8104df0 2024-09-10T22:02:15.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.954+0000 7f8cf6000640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8ce00b55b0 0x7f8ce00b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:15.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.954+0000 7f8cf6000640 1 -- 172.21.15.90:0/1425213795 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f8cec0e65b0 con 0x7f8cf8104df0 2024-09-10T22:02:15.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.954+0000 7f8cfd200640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8ce00b55b0 0x7f8ce00b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:15.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.955+0000 7f8cf6000640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] conn(0x7f8ce00c0870 0x7f8ce00c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:15.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.955+0000 7f8cf6000640 1 -- 172.21.15.90:0/1425213795 --> [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8cec019e70 con 0x7f8ce00c0870 2024-09-10T22:02:15.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.955+0000 7f8cf6000640 1 -- 172.21.15.90:0/1425213795 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f8cec0ed050 con 0x7f8cf8104df0 2024-09-10T22:02:15.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.955+0000 7f8cfd200640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8ce00b55b0 0x7f8ce00b7a70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8103210 tx=0x7f8ce400b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:15.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.955+0000 7f8cfe600640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] conn(0x7f8ce00c0870 0x7f8ce00c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:15.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.956+0000 7f8cfe600640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] conn(0x7f8ce00c0870 0x7f8ce00c2d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:15.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.957+0000 7f8cf6000640 1 -- 172.21.15.90:0/1425213795 <== osd.5 v2:172.21.15.171:6808/2109988699 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8cec019e70 con 0x7f8ce00c0870 2024-09-10T22:02:15.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.971+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 --> [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8cf8063ce0 con 0x7f8ce00c0870 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.972+0000 7f8cf6000640 1 -- 172.21.15.90:0/1425213795 <== osd.5 v2:172.21.15.171:6808/2109988699 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8cf8063ce0 con 0x7f8ce00c0870 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 >> [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] conn(0x7f8ce00c0870 msgr2=0x7f8ce00c2d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] conn(0x7f8ce00c0870 0x7f8ce00c2d10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8ce00b55b0 msgr2=0x7f8ce00b7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8ce00b55b0 0x7f8ce00b7a70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8cf8103210 tx=0x7f8ce400b410 comp rx=0 tx=0).stop 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf8104df0 msgr2=0x7f8cf819b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 --2- 172.21.15.90:0/1425213795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cf8104df0 0x7f8cf819b150 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00e9f0 tx=0x7f8cec00edf0 comp rx=0 tx=0).stop 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cfe600640 1 -- 172.21.15.90:0/1425213795 reap_dead start 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 shutdown_connections 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 >> 172.21.15.90:0/1425213795 conn(0x7f8cf8100840 msgr2=0x7f8cf8101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:15.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 shutdown_connections 2024-09-10T22:02:15.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:10.973+0000 7f8cff000640 1 -- 172.21.15.90:0/1425213795 wait complete. 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.533+0000 7f9a09600640 1 -- 172.21.15.90:0/1254488174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a0410a9d0 msgr2=0x7f9a0410ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.533+0000 7f9a09600640 1 --2- 172.21.15.90:0/1254488174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a0410a9d0 0x7f9a0410ae10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009900 tx=0x7f99ec02a410 comp rx=0 tx=0).stop 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.533+0000 7f9a09600640 1 -- 172.21.15.90:0/1254488174 shutdown_connections 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.533+0000 7f9a09600640 1 --2- 172.21.15.90:0/1254488174 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a0410b350 0x7f9a04111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.533+0000 7f9a09600640 1 --2- 172.21.15.90:0/1254488174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a0410a9d0 0x7f9a0410ae10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.533+0000 7f9a09600640 1 --2- 172.21.15.90:0/1254488174 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a04104fc0 0x7f9a041053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.533+0000 7f9a09600640 1 -- 172.21.15.90:0/1254488174 >> 172.21.15.90:0/1254488174 conn(0x7f9a041009f0 msgr2=0x7f9a04102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.533+0000 7f9a09600640 1 -- 172.21.15.90:0/1254488174 shutdown_connections 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.533+0000 7f9a09600640 1 -- 172.21.15.90:0/1254488174 wait complete. 2024-09-10T22:02:16.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.534+0000 7f9a09600640 1 Processor -- start 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.534+0000 7f9a09600640 1 -- start start 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.534+0000 7f9a09600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a04104fc0 0x7f9a0419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.534+0000 7f9a09600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a0410a9d0 0x7f9a0419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.534+0000 7f9a09600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a0410b350 0x7f9a041a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.534+0000 7f9a09600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04114e00 con 0x7f9a0410a9d0 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.534+0000 7f9a09600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04114c80 con 0x7f9a0410b350 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.534+0000 7f9a09600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04114f80 con 0x7f9a04104fc0 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f99fbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a0410a9d0 0x7f9a0419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f99fbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a0410a9d0 0x7f9a0419b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60962/0 (socket says 172.21.15.90:60962) 2024-09-10T22:02:16.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f99fbe00640 1 -- 172.21.15.90:0/701484973 learned_addr learned my addr 172.21.15.90:0/701484973 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f9a02a00640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a04104fc0 0x7f9a0419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f9a03400640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a0410b350 0x7f9a041a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f99fbe00640 1 -- 172.21.15.90:0/701484973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a04104fc0 msgr2=0x7f9a0419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f99fbe00640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a04104fc0 0x7f9a0419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f99fbe00640 1 -- 172.21.15.90:0/701484973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a0410b350 msgr2=0x7f9a041a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f99fbe00640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9a0410b350 0x7f9a041a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f99fbe00640 1 -- 172.21.15.90:0/701484973 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a041a14a0 con 0x7f9a0410a9d0 2024-09-10T22:02:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f9a02a00640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9a04104fc0 0x7f9a0419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.535+0000 7f99fbe00640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a0410a9d0 0x7f9a0419b930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009230 tx=0x7f99ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.536+0000 7f99fb400640 1 -- 172.21.15.90:0/701484973 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec03f030 con 0x7f9a0410a9d0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.536+0000 7f99fb400640 1 -- 172.21.15.90:0/701484973 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99ec003a80 con 0x7f9a0410a9d0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.536+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a041a1730 con 0x7f9a0410a9d0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.536+0000 7f99fb400640 1 -- 172.21.15.90:0/701484973 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec035070 con 0x7f9a0410a9d0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.536+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a041a7e50 con 0x7f9a0410a9d0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.537+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a040638e0 con 0x7f9a0410a9d0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.539+0000 7f99fb400640 1 -- 172.21.15.90:0/701484973 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f99ec03bc30 con 0x7f9a0410a9d0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.540+0000 7f99fb400640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99e80b5440 0x7f99e80b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.540+0000 7f99fb400640 1 -- 172.21.15.90:0/701484973 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f99ec107110 con 0x7f9a0410a9d0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.541+0000 7f9a02a00640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99e80b5440 0x7f99e80b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.541+0000 7f99fb400640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] conn(0x7f99e80c0700 0x7f99e80c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.541+0000 7f99fb400640 1 -- 172.21.15.90:0/701484973 --> [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99e80c3270 con 0x7f99e80c0700 2024-09-10T22:02:16.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.541+0000 7f99fb400640 1 -- 172.21.15.90:0/701484973 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f99ec1074c0 con 0x7f9a0410a9d0 2024-09-10T22:02:16.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.541+0000 7f9a03400640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] conn(0x7f99e80c0700 0x7f99e80c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.541+0000 7f9a02a00640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99e80b5440 0x7f99e80b7900 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99f40058c0 tx=0x7f99f4005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:16.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.542+0000 7f9a03400640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] conn(0x7f99e80c0700 0x7f99e80c2ba0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:16.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.543+0000 7f99fb400640 1 -- 172.21.15.90:0/701484973 <== osd.11 v2:172.21.15.171:6824/1226533587 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99e80c3270 con 0x7f99e80c0700 2024-09-10T22:02:16.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.557+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 --> [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a04063ce0 con 0x7f99e80c0700 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.557+0000 7f99fb400640 1 -- 172.21.15.90:0/701484973 <== osd.11 v2:172.21.15.171:6824/1226533587 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9a04063ce0 con 0x7f99e80c0700 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 >> [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] conn(0x7f99e80c0700 msgr2=0x7f99e80c2ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] conn(0x7f99e80c0700 0x7f99e80c2ba0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99e80b5440 msgr2=0x7f99e80b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99e80b5440 0x7f99e80b7900 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99f40058c0 tx=0x7f99f4005850 comp rx=0 tx=0).stop 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a0410a9d0 msgr2=0x7f9a0419b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 --2- 172.21.15.90:0/701484973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a0410a9d0 0x7f9a0419b930 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009230 tx=0x7f99ec004270 comp rx=0 tx=0).stop 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a03400640 1 -- 172.21.15.90:0/701484973 reap_dead start 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 shutdown_connections 2024-09-10T22:02:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 >> 172.21.15.90:0/701484973 conn(0x7f9a041009f0 msgr2=0x7f9a04102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:16.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 shutdown_connections 2024-09-10T22:02:16.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:11.558+0000 7f9a09600640 1 -- 172.21.15.90:0/701484973 wait complete. 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.519+0000 7f82b0200640 1 -- 172.21.15.90:0/2802694730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82a8102da0 msgr2=0x7f82a8103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.519+0000 7f82b0200640 1 --2- 172.21.15.90:0/2802694730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82a8102da0 0x7f82a8103180 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f829c009900 tx=0x7f829c02a410 comp rx=0 tx=0).stop 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.519+0000 7f82b0200640 1 -- 172.21.15.90:0/2802694730 shutdown_connections 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.519+0000 7f82b0200640 1 --2- 172.21.15.90:0/2802694730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a8109130 0x7f82a810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.519+0000 7f82b0200640 1 --2- 172.21.15.90:0/2802694730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a81087b0 0x7f82a8108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.519+0000 7f82b0200640 1 --2- 172.21.15.90:0/2802694730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82a8102da0 0x7f82a8103180 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.519+0000 7f82b0200640 1 -- 172.21.15.90:0/2802694730 >> 172.21.15.90:0/2802694730 conn(0x7f82a80fe810 msgr2=0x7f82a8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.519+0000 7f82b0200640 1 -- 172.21.15.90:0/2802694730 shutdown_connections 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.520+0000 7f82b0200640 1 -- 172.21.15.90:0/2802694730 wait complete. 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82b0200640 1 Processor -- start 2024-09-10T22:02:16.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82b0200640 1 -- start start 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82b0200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82a8102da0 0x7f82a819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82b0200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a81087b0 0x7f82a819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82b0200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a8109130 0x7f82a81a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82b0200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f82a8112cf0 con 0x7f82a8102da0 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82b0200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f82a8112b70 con 0x7f82a8109130 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82b0200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f82a8112e70 con 0x7f82a81087b0 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82ad800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82a8102da0 0x7f82a819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82ad800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82a8102da0 0x7f82a819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32868/0 (socket says 172.21.15.90:32868) 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.521+0000 7f82ad800640 1 -- 172.21.15.90:0/449158436 learned_addr learned my addr 172.21.15.90:0/449158436 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.522+0000 7f82ace00640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a81087b0 0x7f82a819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.522+0000 7f82ae200640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a8109130 0x7f82a81a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.522+0000 7f82ad800640 1 -- 172.21.15.90:0/449158436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a81087b0 msgr2=0x7f82a819b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.522+0000 7f82ad800640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f82a81087b0 0x7f82a819b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.522+0000 7f82ad800640 1 -- 172.21.15.90:0/449158436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a8109130 msgr2=0x7f82a81a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.522+0000 7f82ad800640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f82a8109130 0x7f82a81a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.522+0000 7f82ad800640 1 -- 172.21.15.90:0/449158436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a81a14c0 con 0x7f82a8102da0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.522+0000 7f82ad800640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82a8102da0 0x7f82a819b410 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f829c02a930 tx=0x7f829c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.523+0000 7f82a6000640 1 -- 172.21.15.90:0/449158436 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c03f070 con 0x7f82a8102da0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.523+0000 7f82a6000640 1 -- 172.21.15.90:0/449158436 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f829c03baf0 con 0x7f82a8102da0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.523+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a81a1750 con 0x7f82a8102da0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.523+0000 7f82a6000640 1 -- 172.21.15.90:0/449158436 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c035030 con 0x7f82a8102da0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.523+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a81a7e70 con 0x7f82a8102da0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.524+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f82a80638e0 con 0x7f82a8102da0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.527+0000 7f82a6000640 1 -- 172.21.15.90:0/449158436 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f829c03b660 con 0x7f82a8102da0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.528+0000 7f82a6000640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82900b53c0 0x7f82900b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.528+0000 7f82a6000640 1 -- 172.21.15.90:0/449158436 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f829c106ec0 con 0x7f82a8102da0 2024-09-10T22:02:16.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.528+0000 7f82ace00640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82900b53c0 0x7f82900b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.528+0000 7f82a6000640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] conn(0x7f82900c0680 0x7f82900c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:16.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.528+0000 7f82a6000640 1 -- 172.21.15.90:0/449158436 --> [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f829c02bec0 con 0x7f82900c0680 2024-09-10T22:02:16.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.528+0000 7f82a6000640 1 -- 172.21.15.90:0/449158436 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f829c107270 con 0x7f82a8102da0 2024-09-10T22:02:16.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.529+0000 7f82ace00640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82900b53c0 0x7f82900b7880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f82a8108550 tx=0x7f8294005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:16.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.529+0000 7f82ae200640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] conn(0x7f82900c0680 0x7f82900c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:16.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.529+0000 7f82ae200640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] conn(0x7f82900c0680 0x7f82900c2b20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:16.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.530+0000 7f82a6000640 1 -- 172.21.15.90:0/449158436 <== osd.2 v2:172.21.15.171:6800/862625037 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f829c02bec0 con 0x7f82900c0680 2024-09-10T22:02:16.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.548+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 --> [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f82a8063d60 con 0x7f82900c0680 2024-09-10T22:02:16.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.548+0000 7f82a6000640 1 -- 172.21.15.90:0/449158436 <== osd.2 v2:172.21.15.171:6800/862625037 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f82a8063d60 con 0x7f82900c0680 2024-09-10T22:02:16.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 >> [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] conn(0x7f82900c0680 msgr2=0x7f82900c2b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] conn(0x7f82900c0680 0x7f82900c2b20 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:16.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82900b53c0 msgr2=0x7f82900b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82900b53c0 0x7f82900b7880 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f82a8108550 tx=0x7f8294005f90 comp rx=0 tx=0).stop 2024-09-10T22:02:16.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82a8102da0 msgr2=0x7f82a819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:16.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 --2- 172.21.15.90:0/449158436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82a8102da0 0x7f82a819b410 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f829c02a930 tx=0x7f829c003b90 comp rx=0 tx=0).stop 2024-09-10T22:02:16.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82ae200640 1 -- 172.21.15.90:0/449158436 reap_dead start 2024-09-10T22:02:16.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 shutdown_connections 2024-09-10T22:02:16.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 >> 172.21.15.90:0/449158436 conn(0x7f82a80fe810 msgr2=0x7f82a8104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:16.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 shutdown_connections 2024-09-10T22:02:16.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:16.549+0000 7f82b0200640 1 -- 172.21.15.90:0/449158436 wait complete. 2024-09-10T22:02:16.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:16 smithi171 ceph-mon[34138]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T22:02:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:16 smithi090 ceph-mon[30350]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T22:02:16.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:16 smithi154 ceph-mon[34495]: pgmap v367: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T22:02:17.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.149+0000 7f20eb000640 1 -- 172.21.15.90:0/3624212933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20e4104df0 msgr2=0x7f20e41051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.149+0000 7f20eb000640 1 --2- 172.21.15.90:0/3624212933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20e4104df0 0x7f20e41051d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009ca0 tx=0x7f20d402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.149+0000 7f20eb000640 1 -- 172.21.15.90:0/3624212933 shutdown_connections 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.149+0000 7f20eb000640 1 --2- 172.21.15.90:0/3624212933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20e410b180 0x7f20e4111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.149+0000 7f20eb000640 1 --2- 172.21.15.90:0/3624212933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e410a800 0x7f20e410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.149+0000 7f20eb000640 1 --2- 172.21.15.90:0/3624212933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20e4104df0 0x7f20e41051d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.149+0000 7f20eb000640 1 -- 172.21.15.90:0/3624212933 >> 172.21.15.90:0/3624212933 conn(0x7f20e4100840 msgr2=0x7f20e4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.149+0000 7f20eb000640 1 -- 172.21.15.90:0/3624212933 shutdown_connections 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.149+0000 7f20eb000640 1 -- 172.21.15.90:0/3624212933 wait complete. 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.150+0000 7f20eb000640 1 Processor -- start 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.150+0000 7f20eb000640 1 -- start start 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20eb000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e4104df0 0x7f20e419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:17.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20eb000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20e410a800 0x7f20e419b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20eb000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20e410b180 0x7f20e41a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20eb000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4114d00 con 0x7f20e410b180 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20eb000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4114b80 con 0x7f20e4104df0 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20eb000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4114e80 con 0x7f20e410a800 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20ea600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20e410b180 0x7f20e41a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20ea600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20e410b180 0x7f20e41a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32886/0 (socket says 172.21.15.90:32886) 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20ea600640 1 -- 172.21.15.90:0/812810706 learned_addr learned my addr 172.21.15.90:0/812810706 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20e9200640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20e410a800 0x7f20e419b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20e9c00640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e4104df0 0x7f20e419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.151+0000 7f20ea600640 1 -- 172.21.15.90:0/812810706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20e410a800 msgr2=0x7f20e419b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20ea600640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f20e410a800 0x7f20e419b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:17.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20ea600640 1 -- 172.21.15.90:0/812810706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e4104df0 msgr2=0x7f20e419b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:17.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20ea600640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f20e4104df0 0x7f20e419b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:17.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20ea600640 1 -- 172.21.15.90:0/812810706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e41a1140 con 0x7f20e410b180 2024-09-10T22:02:17.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20ea600640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20e410b180 0x7f20e41a0970 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00d750 tx=0x7f20dc00db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:17.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20e2000640 1 -- 172.21.15.90:0/812810706 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc017070 con 0x7f20e410b180 2024-09-10T22:02:17.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e41a1430 con 0x7f20e410b180 2024-09-10T22:02:17.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20e2000640 1 -- 172.21.15.90:0/812810706 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20dc0097c0 con 0x7f20e410b180 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20e2000640 1 -- 172.21.15.90:0/812810706 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc020060 con 0x7f20e410b180 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.152+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e41a7af0 con 0x7f20e410b180 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.153+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20e4108110 con 0x7f20e410b180 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.156+0000 7f20e2000640 1 -- 172.21.15.90:0/812810706 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f20dc023020 con 0x7f20e410b180 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.157+0000 7f20e2000640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f20c80b5460 0x7f20c80b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.158+0000 7f20e2000640 1 -- 172.21.15.90:0/812810706 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f20dc0e5f70 con 0x7f20e410b180 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.158+0000 7f20e9c00640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f20c80b5460 0x7f20c80b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.158+0000 7f20e2000640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] conn(0x7f20c80c0720 0x7f20c80c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.158+0000 7f20e2000640 1 -- 172.21.15.90:0/812810706 --> [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20dc012e50 con 0x7f20c80c0720 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.158+0000 7f20e2000640 1 -- 172.21.15.90:0/812810706 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f20dc0e6320 con 0x7f20e410b180 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.158+0000 7f20e9200640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] conn(0x7f20c80c0720 0x7f20c80c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:17.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.159+0000 7f20e9c00640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f20c80b5460 0x7f20c80b7920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f20d4000f80 tx=0x7f20d4031570 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.159+0000 7f20e9200640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] conn(0x7f20c80c0720 0x7f20c80c2bc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.160+0000 7f20e2000640 1 -- 172.21.15.90:0/812810706 <== osd.9 v2:172.21.15.90:6826/1146146212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20dc012e50 con 0x7f20c80c0720 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.174+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 --> [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20e41060a0 con 0x7f20c80c0720 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.174+0000 7f20e2000640 1 -- 172.21.15.90:0/812810706 <== osd.9 v2:172.21.15.90:6826/1146146212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20e41060a0 con 0x7f20c80c0720 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] conn(0x7f20c80c0720 msgr2=0x7f20c80c2bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] conn(0x7f20c80c0720 0x7f20c80c2bc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f20c80b5460 msgr2=0x7f20c80b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f20c80b5460 0x7f20c80b7920 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f20d4000f80 tx=0x7f20d4031570 comp rx=0 tx=0).stop 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20e410b180 msgr2=0x7f20e41a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 --2- 172.21.15.90:0/812810706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20e410b180 0x7f20e41a0970 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00d750 tx=0x7f20dc00db50 comp rx=0 tx=0).stop 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20ea600640 1 -- 172.21.15.90:0/812810706 reap_dead start 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 shutdown_connections 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 >> 172.21.15.90:0/812810706 conn(0x7f20e4100840 msgr2=0x7f20e4103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 shutdown_connections 2024-09-10T22:02:17.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:17.175+0000 7f20eb000640 1 -- 172.21.15.90:0/812810706 wait complete. 2024-09-10T22:02:18.583 INFO:teuthology.orchestra.run.smithi090.stdout:38654705767 2024-09-10T22:02:18.583 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T22:02:18.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:18 smithi154 ceph-mon[34495]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-10T22:02:18.934 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:18 smithi090 ceph-mon[30350]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-10T22:02:18.970 INFO:teuthology.orchestra.run.smithi090.stdout:257698037807 2024-09-10T22:02:18.970 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T22:02:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:18 smithi171 ceph-mon[34138]: pgmap v368: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-10T22:02:19.348 INFO:teuthology.orchestra.run.smithi090.stdout:111669149781 2024-09-10T22:02:19.349 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T22:02:19.769 INFO:teuthology.orchestra.run.smithi090.stdout:304942678051 2024-09-10T22:02:19.770 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T22:02:20.864 INFO:teuthology.orchestra.run.smithi090.stdout:81604378716 2024-09-10T22:02:20.864 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T22:02:20.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:20 smithi154 ceph-mon[34495]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-10T22:02:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:20 smithi171 ceph-mon[34138]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-10T22:02:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:20 smithi090 ceph-mon[30350]: pgmap v369: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-10T22:02:21.310 INFO:teuthology.orchestra.run.smithi090.stdout:158913790025 2024-09-10T22:02:21.311 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T22:02:21.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.525+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1030461875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d810f250 msgr2=0x7fb5d8111640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:21.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.525+0000 7fb5d7e00640 1 --2- 172.21.15.90:0/1030461875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d810f250 0x7fb5d8111640 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009ca0 tx=0x7fb5d002a950 comp rx=0 tx=0).stop 2024-09-10T22:02:21.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.526+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1030461875 shutdown_connections 2024-09-10T22:02:21.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.526+0000 7fb5d7e00640 1 --2- 172.21.15.90:0/1030461875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d810f250 0x7fb5d8111640 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:21.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.526+0000 7fb5d7e00640 1 --2- 172.21.15.90:0/1030461875 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5d8101fc0 0x7fb5d810ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:21.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.526+0000 7fb5d7e00640 1 --2- 172.21.15.90:0/1030461875 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5d81016a0 0x7fb5d8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:21.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.526+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1030461875 >> 172.21.15.90:0/1030461875 conn(0x7fb5d80fd5b0 msgr2=0x7fb5d80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:21.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.526+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1030461875 shutdown_connections 2024-09-10T22:02:21.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.526+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1030461875 wait complete. 2024-09-10T22:02:21.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.527+0000 7fb5d7e00640 1 Processor -- start 2024-09-10T22:02:21.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.527+0000 7fb5d7e00640 1 -- start start 2024-09-10T22:02:21.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.527+0000 7fb5d7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5d81016a0 0x7fb5d819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:21.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.527+0000 7fb5d7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5d8101fc0 0x7fb5d819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:21.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.527+0000 7fb5d7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d810f250 0x7fb5d81a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:21.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.527+0000 7fb5d7e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8114cd0 con 0x7fb5d810f250 2024-09-10T22:02:21.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.527+0000 7fb5d7e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8114b50 con 0x7fb5d8101fc0 2024-09-10T22:02:21.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.527+0000 7fb5d7e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d8114e50 con 0x7fb5d81016a0 2024-09-10T22:02:21.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d6a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5d81016a0 0x7fb5d819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:21.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d810f250 0x7fb5d81a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:21.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d6a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5d81016a0 0x7fb5d819b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:38552/0 (socket says 172.21.15.90:38552) 2024-09-10T22:02:21.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d6a00640 1 -- 172.21.15.90:0/1149851059 learned_addr learned my addr 172.21.15.90:0/1149851059 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:21.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5ce000640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5d8101fc0 0x7fb5d819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:21.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d7400640 1 -- 172.21.15.90:0/1149851059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5d81016a0 msgr2=0x7fb5d819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:21.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d7400640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb5d81016a0 0x7fb5d819b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:21.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d7400640 1 -- 172.21.15.90:0/1149851059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5d8101fc0 msgr2=0x7fb5d819b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:21.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d7400640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb5d8101fc0 0x7fb5d819b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:21.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d7400640 1 -- 172.21.15.90:0/1149851059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d81a1170 con 0x7fb5d810f250 2024-09-10T22:02:21.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.528+0000 7fb5d7400640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d810f250 0x7fb5d81a09a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0005e30 tx=0x7fb5d0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:21.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.529+0000 7fb5cfe00640 1 -- 172.21.15.90:0/1149851059 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0034040 con 0x7fb5d810f250 2024-09-10T22:02:21.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.529+0000 7fb5cfe00640 1 -- 172.21.15.90:0/1149851059 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb5d0030850 con 0x7fb5d810f250 2024-09-10T22:02:21.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.529+0000 7fb5cfe00640 1 -- 172.21.15.90:0/1149851059 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00045a0 con 0x7fb5d810f250 2024-09-10T22:02:21.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.529+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d81a1400 con 0x7fb5d810f250 2024-09-10T22:02:21.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.529+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d81a7b20 con 0x7fb5d810f250 2024-09-10T22:02:21.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.530+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb5d8110840 con 0x7fb5d810f250 2024-09-10T22:02:21.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.532+0000 7fb5cfe00640 1 -- 172.21.15.90:0/1149851059 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb5d002f860 con 0x7fb5d810f250 2024-09-10T22:02:21.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.534+0000 7fb5cfe00640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb5bc0b5460 0x7fb5bc0b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:21.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.534+0000 7fb5cfe00640 1 -- 172.21.15.90:0/1149851059 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fb5d0107170 con 0x7fb5d810f250 2024-09-10T22:02:21.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.534+0000 7fb5d6a00640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb5bc0b5460 0x7fb5bc0b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:21.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.534+0000 7fb5cfe00640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] conn(0x7fb5bc0c0720 0x7fb5bc0c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:21.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.534+0000 7fb5cfe00640 1 -- 172.21.15.90:0/1149851059 --> [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5d003ce50 con 0x7fb5bc0c0720 2024-09-10T22:02:21.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.534+0000 7fb5cfe00640 1 -- 172.21.15.90:0/1149851059 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fb5d0107520 con 0x7fb5d810f250 2024-09-10T22:02:21.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.535+0000 7fb5d6a00640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb5bc0b5460 0x7fb5bc0b7920 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800bca0 tx=0x7fb5c800b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:21.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.546+0000 7fb5ce000640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] conn(0x7fb5bc0c0720 0x7fb5bc0c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:21.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.547+0000 7fb5ce000640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] conn(0x7fb5bc0c0720 0x7fb5bc0c2bc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:21.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.548+0000 7fb5cfe00640 1 -- 172.21.15.90:0/1149851059 <== osd.8 v2:172.21.15.171:6816/4137047501 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb5d003ce50 con 0x7fb5bc0c0720 2024-09-10T22:02:21.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.562+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 --> [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb5d806b1b0 con 0x7fb5bc0c0720 2024-09-10T22:02:21.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.563+0000 7fb5cfe00640 1 -- 172.21.15.90:0/1149851059 <== osd.8 v2:172.21.15.171:6816/4137047501 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb5d806b1b0 con 0x7fb5bc0c0720 2024-09-10T22:02:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.563+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 >> [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] conn(0x7fb5bc0c0720 msgr2=0x7fb5bc0c2bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.563+0000 7fb5d7e00640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] conn(0x7fb5bc0c0720 0x7fb5bc0c2bc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.563+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb5bc0b5460 msgr2=0x7fb5bc0b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.563+0000 7fb5d7e00640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb5bc0b5460 0x7fb5bc0b7920 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800bca0 tx=0x7fb5c800b410 comp rx=0 tx=0).stop 2024-09-10T22:02:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.563+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d810f250 msgr2=0x7fb5d81a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.563+0000 7fb5d7e00640 1 --2- 172.21.15.90:0/1149851059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5d810f250 0x7fb5d81a09a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0005e30 tx=0x7fb5d0003ce0 comp rx=0 tx=0).stop 2024-09-10T22:02:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.563+0000 7fb5d7400640 1 -- 172.21.15.90:0/1149851059 reap_dead start 2024-09-10T22:02:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.564+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 shutdown_connections 2024-09-10T22:02:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.564+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 >> 172.21.15.90:0/1149851059 conn(0x7fb5d80fd5b0 msgr2=0x7fb5d81113d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.564+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 shutdown_connections 2024-09-10T22:02:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:21.564+0000 7fb5d7e00640 1 -- 172.21.15.90:0/1149851059 wait complete. 2024-09-10T22:02:22.722 INFO:teuthology.orchestra.run.smithi090.stdout:231928234040 2024-09-10T22:02:22.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T22:02:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:22 smithi171 ceph-mon[34138]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T22:02:23.004 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:22 smithi090 ceph-mon[30350]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T22:02:23.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.002+0000 7f854ae00640 1 -- 172.21.15.90:0/2926580288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f854410b330 msgr2=0x7f8544111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.002+0000 7f854ae00640 1 --2- 172.21.15.90:0/2926580288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f854410b330 0x7f8544111b30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f853c009ca0 tx=0x7f853c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.003+0000 7f854ae00640 1 -- 172.21.15.90:0/2926580288 shutdown_connections 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.003+0000 7f854ae00640 1 --2- 172.21.15.90:0/2926580288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f854410b330 0x7f8544111b30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.003+0000 7f854ae00640 1 --2- 172.21.15.90:0/2926580288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f854410a9b0 0x7f854410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.003+0000 7f854ae00640 1 --2- 172.21.15.90:0/2926580288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8544104fa0 0x7f8544105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.003+0000 7f854ae00640 1 -- 172.21.15.90:0/2926580288 >> 172.21.15.90:0/2926580288 conn(0x7f8544100a50 msgr2=0x7f8544102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.003+0000 7f854ae00640 1 -- 172.21.15.90:0/2926580288 shutdown_connections 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.003+0000 7f854ae00640 1 -- 172.21.15.90:0/2926580288 wait complete. 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f854ae00640 1 Processor -- start 2024-09-10T22:02:23.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f854ae00640 1 -- start start 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f854ae00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8544104fa0 0x7f854419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f854ae00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f854410a9b0 0x7f854419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f854ae00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f854410b330 0x7f85441a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f854ae00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8544114e60 con 0x7f854410b330 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f854ae00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8544114ce0 con 0x7f854410a9b0 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f854ae00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8544114fe0 con 0x7f8544104fa0 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f8543e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8544104fa0 0x7f854419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f8543e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8544104fa0 0x7f854419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:38578/0 (socket says 172.21.15.90:38578) 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.004+0000 7f8543e00640 1 -- 172.21.15.90:0/4150361104 learned_addr learned my addr 172.21.15.90:0/4150361104 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.005+0000 7f853be00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f854410a9b0 0x7f854419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.005+0000 7f8548e00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f854410b330 0x7f85441a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.005+0000 7f8543e00640 1 -- 172.21.15.90:0/4150361104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f854410a9b0 msgr2=0x7f854419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:23.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.005+0000 7f8543e00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f854410a9b0 0x7f854419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.005+0000 7f8543e00640 1 -- 172.21.15.90:0/4150361104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f854410b330 msgr2=0x7f85441a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.005+0000 7f8543e00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f854410b330 0x7f85441a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.005+0000 7f8543e00640 1 -- 172.21.15.90:0/4150361104 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85441a1420 con 0x7f8544104fa0 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.006+0000 7f8543e00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8544104fa0 0x7f854419b370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f853400d950 tx=0x7f853400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.006+0000 7f8541600640 1 -- 172.21.15.90:0/4150361104 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534017070 con 0x7f8544104fa0 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.006+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85441a1710 con 0x7f8544104fa0 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.006+0000 7f8541600640 1 -- 172.21.15.90:0/4150361104 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8534009c30 con 0x7f8544104fa0 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.006+0000 7f8541600640 1 -- 172.21.15.90:0/4150361104 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853400f620 con 0x7f8544104fa0 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.006+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85441a7fa0 con 0x7f8544104fa0 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.007+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85440638e0 con 0x7f8544104fa0 2024-09-10T22:02:23.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.010+0000 7f8541600640 1 -- 172.21.15.90:0/4150361104 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f8534010460 con 0x7f8544104fa0 2024-09-10T22:02:23.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.011+0000 7f8541600640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f85280b53c0 0x7f85280b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:23.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.011+0000 7f8541600640 1 -- 172.21.15.90:0/4150361104 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f85340e61b0 con 0x7f8544104fa0 2024-09-10T22:02:23.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.011+0000 7f853be00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f85280b53c0 0x7f85280b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:23.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.012+0000 7f8541600640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] conn(0x7f85280c0680 0x7f85280c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:23.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.012+0000 7f8541600640 1 -- 172.21.15.90:0/4150361104 --> [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8534010ac0 con 0x7f85280c0680 2024-09-10T22:02:23.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.012+0000 7f8541600640 1 -- 172.21.15.90:0/4150361104 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f85340e6560 con 0x7f8544104fa0 2024-09-10T22:02:23.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.012+0000 7f8548e00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] conn(0x7f85280c0680 0x7f85280c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:23.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.012+0000 7f853be00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f85280b53c0 0x7f85280b7880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8544107380 tx=0x7f852c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:23.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.013+0000 7f8548e00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] conn(0x7f85280c0680 0x7f85280c2b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.014+0000 7f8541600640 1 -- 172.21.15.90:0/4150361104 <== osd.7 v2:172.21.15.154:6816/1617520219 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8534010ac0 con 0x7f85280c0680 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.028+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 --> [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8544063ce0 con 0x7f85280c0680 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.029+0000 7f8541600640 1 -- 172.21.15.90:0/4150361104 <== osd.7 v2:172.21.15.154:6816/1617520219 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8544063ce0 con 0x7f85280c0680 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.029+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 >> [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] conn(0x7f85280c0680 msgr2=0x7f85280c2b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.029+0000 7f854ae00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] conn(0x7f85280c0680 0x7f85280c2b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.030+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f85280b53c0 msgr2=0x7f85280b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.030+0000 7f854ae00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f85280b53c0 0x7f85280b7880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8544107380 tx=0x7f852c005f90 comp rx=0 tx=0).stop 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.030+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8544104fa0 msgr2=0x7f854419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.030+0000 7f854ae00640 1 --2- 172.21.15.90:0/4150361104 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8544104fa0 0x7f854419b370 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f853400d950 tx=0x7f853400de10 comp rx=0 tx=0).stop 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.030+0000 7f8548e00640 1 -- 172.21.15.90:0/4150361104 reap_dead start 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.030+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 shutdown_connections 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.030+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 >> 172.21.15.90:0/4150361104 conn(0x7f8544100a50 msgr2=0x7f8544102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.030+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 shutdown_connections 2024-09-10T22:02:23.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:23.030+0000 7f854ae00640 1 -- 172.21.15.90:0/4150361104 wait complete. 2024-09-10T22:02:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:22 smithi154 ceph-mon[34495]: pgmap v370: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T22:02:23.592 INFO:teuthology.orchestra.run.smithi090.stdout:279172874280 2024-09-10T22:02:23.592 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T22:02:24.010 INFO:teuthology.orchestra.run.smithi090.stdout:60129542242 2024-09-10T22:02:24.010 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T22:02:24.653 INFO:teuthology.orchestra.run.smithi090.stdout:137438953552 2024-09-10T22:02:24.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T22:02:24.920 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:24 smithi090 ceph-mon[30350]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-10T22:02:24.961 INFO:teuthology.orchestra.run.smithi090.stdout:206158430270 2024-09-10T22:02:24.962 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T22:02:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:24 smithi171 ceph-mon[34138]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-10T22:02:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:24 smithi154 ceph-mon[34495]: pgmap v371: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-10T22:02:25.554 INFO:teuthology.orchestra.run.smithi090.stdout:184683593796 2024-09-10T22:02:25.554 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T22:02:25.716 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:25.718 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:25.738 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:25.738 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:25.742 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:25.751 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:25.783 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.432+0000 7f7f80e00640 1 -- 172.21.15.90:0/2713411654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c104fa0 msgr2=0x7f7f7c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.432+0000 7f7f80e00640 1 --2- 172.21.15.90:0/2713411654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c104fa0 0x7f7f7c105380 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009900 tx=0x7f7f6c02a410 comp rx=0 tx=0).stop 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.432+0000 7f7f80e00640 1 -- 172.21.15.90:0/2713411654 shutdown_connections 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.432+0000 7f7f80e00640 1 --2- 172.21.15.90:0/2713411654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f7c10b330 0x7f7f7c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.432+0000 7f7f80e00640 1 --2- 172.21.15.90:0/2713411654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f7c10a9b0 0x7f7f7c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.432+0000 7f7f80e00640 1 --2- 172.21.15.90:0/2713411654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c104fa0 0x7f7f7c105380 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.432+0000 7f7f80e00640 1 -- 172.21.15.90:0/2713411654 >> 172.21.15.90:0/2713411654 conn(0x7f7f7c100a50 msgr2=0x7f7f7c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.432+0000 7f7f80e00640 1 -- 172.21.15.90:0/2713411654 shutdown_connections 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.432+0000 7f7f80e00640 1 -- 172.21.15.90:0/2713411654 wait complete. 2024-09-10T22:02:26.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.433+0000 7f7f80e00640 1 Processor -- start 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.433+0000 7f7f80e00640 1 -- start start 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.433+0000 7f7f80e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f7c104fa0 0x7f7f7c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.433+0000 7f7f80e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c10a9b0 0x7f7f7c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.433+0000 7f7f80e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f7c10b330 0x7f7f7c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f80e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c114e60 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f80e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c114ce0 con 0x7f7f7c104fa0 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f80e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c114fe0 con 0x7f7f7c10b330 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f79400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c10a9b0 0x7f7f7c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f79400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c10a9b0 0x7f7f7c19b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49576/0 (socket says 172.21.15.90:49576) 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f79400640 1 -- 172.21.15.90:0/3772633727 learned_addr learned my addr 172.21.15.90:0/3772633727 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f79e00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f7c104fa0 0x7f7f7c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:26.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f7a800640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f7c10b330 0x7f7f7c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f79400640 1 -- 172.21.15.90:0/3772633727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f7c10b330 msgr2=0x7f7f7c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f79400640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f7c10b330 0x7f7f7c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f79400640 1 -- 172.21.15.90:0/3772633727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f7c104fa0 msgr2=0x7f7f7c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f79400640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f7c104fa0 0x7f7f7c19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.434+0000 7f7f79400640 1 -- 172.21.15.90:0/3772633727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c1a13c0 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.435+0000 7f7f79400640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c10a9b0 0x7f7f7c19b850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800c920 tx=0x7f7f6800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.435+0000 7f7f72a00640 1 -- 172.21.15.90:0/3772633727 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68007c70 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.435+0000 7f7f72a00640 1 -- 172.21.15.90:0/3772633727 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7f68007e10 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.435+0000 7f7f72a00640 1 -- 172.21.15.90:0/3772633727 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68018710 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.435+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c1a16b0 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.435+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c1a7d70 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.436+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7c105380 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.439+0000 7f7f72a00640 1 -- 172.21.15.90:0/3772633727 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f7f6800f450 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.441+0000 7f7f72a00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7f600b5440 0x7f7f600b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.441+0000 7f7f72a00640 1 -- 172.21.15.90:0/3772633727 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f7f680e6550 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.441+0000 7f7f79e00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7f600b5440 0x7f7f600b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.442+0000 7f7f79e00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7f600b5440 0x7f7f600b7900 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c006010 tx=0x7f7f6c005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:26.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.446+0000 7f7f72a00640 1 -- 172.21.15.90:0/3772633727 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7f68014030 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.656+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7f7c02e5c0 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.656+0000 7f7f72a00640 1 -- 172.21.15.90:0/3772633727 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7f680acf40 con 0x7f7f7c10a9b0 2024-09-10T22:02:26.658 INFO:teuthology.orchestra.run.smithi090.stdout:158913790028 2024-09-10T22:02:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7f600b5440 msgr2=0x7f7f600b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7f600b5440 0x7f7f600b7900 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c006010 tx=0x7f7f6c005e80 comp rx=0 tx=0).stop 2024-09-10T22:02:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c10a9b0 msgr2=0x7f7f7c19b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:26.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c10a9b0 0x7f7f7c19b850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800c920 tx=0x7f7f6800cde0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 shutdown_connections 2024-09-10T22:02:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7f600b5440 0x7f7f600b7900 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7f7c10b330 0x7f7f7c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f7c10a9b0 0x7f7f7c19b850 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 --2- 172.21.15.90:0/3772633727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7f7c104fa0 0x7f7f7c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.658+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 >> 172.21.15.90:0/3772633727 conn(0x7f7f7c100a50 msgr2=0x7f7f7c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.659+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 shutdown_connections 2024-09-10T22:02:26.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.659+0000 7f7f80e00640 1 -- 172.21.15.90:0/3772633727 wait complete. 2024-09-10T22:02:26.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.840+0000 7f48a9000640 1 -- 172.21.15.90:0/2254040717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108fa0 msgr2=0x7f48a410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:26.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.840+0000 7f48a9000640 1 --2- 172.21.15.90:0/2254040717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108fa0 0x7f48a410f7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f489c009ca0 tx=0x7f489c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.840+0000 7f48a9000640 1 -- 172.21.15.90:0/2254040717 shutdown_connections 2024-09-10T22:02:26.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.840+0000 7f48a9000640 1 --2- 172.21.15.90:0/2254040717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108fa0 0x7f48a410f7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.840+0000 7f48a9000640 1 --2- 172.21.15.90:0/2254040717 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48a4108620 0x7f48a4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.840+0000 7f48a9000640 1 --2- 172.21.15.90:0/2254040717 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48a4102c10 0x7f48a4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.840+0000 7f48a9000640 1 -- 172.21.15.90:0/2254040717 >> 172.21.15.90:0/2254040717 conn(0x7f48a40fe6c0 msgr2=0x7f48a4100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:26.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.840+0000 7f48a9000640 1 -- 172.21.15.90:0/2254040717 shutdown_connections 2024-09-10T22:02:26.843 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:26 smithi090 ceph-mon[30350]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-10T22:02:26.843 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:26 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3772633727' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:02:26.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.840+0000 7f48a9000640 1 -- 172.21.15.90:0/2254040717 wait complete. 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.841+0000 7f48a9000640 1 Processor -- start 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.841+0000 7f48a9000640 1 -- start start 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f48a9000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48a4102c10 0x7f48a419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f48a9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108620 0x7f48a419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f48a9000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48a4108fa0 0x7f48a41a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f48a9000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4112b60 con 0x7f48a4108620 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f48a9000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41129e0 con 0x7f48a4102c10 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f48a9000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f48a4112ce0 con 0x7f48a4108fa0 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f489be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108620 0x7f48a419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:26.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f489be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108620 0x7f48a419b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49584/0 (socket says 172.21.15.90:49584) 2024-09-10T22:02:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f489be00640 1 -- 172.21.15.90:0/1879969211 learned_addr learned my addr 172.21.15.90:0/1879969211 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.842+0000 7f48a3e00640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48a4108fa0 0x7f48a41a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f489be00640 1 -- 172.21.15.90:0/1879969211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48a4108fa0 msgr2=0x7f48a41a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f489be00640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48a4108fa0 0x7f48a41a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f489be00640 1 -- 172.21.15.90:0/1879969211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48a4102c10 msgr2=0x7f48a419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f489be00640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48a4102c10 0x7f48a419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f489be00640 1 -- 172.21.15.90:0/1879969211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a41a1160 con 0x7f48a4108620 2024-09-10T22:02:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f48a3e00640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48a4108fa0 0x7f48a41a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:26.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f489be00640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108620 0x7f48a419b5f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f488c00d950 tx=0x7f488c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:26.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f48a0c00640 1 -- 172.21.15.90:0/1879969211 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c017070 con 0x7f48a4108620 2024-09-10T22:02:26.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f48a0c00640 1 -- 172.21.15.90:0/1879969211 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f488c009c30 con 0x7f48a4108620 2024-09-10T22:02:26.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a41a1450 con 0x7f48a4108620 2024-09-10T22:02:26.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.843+0000 7f48a0c00640 1 -- 172.21.15.90:0/1879969211 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c00f620 con 0x7f48a4108620 2024-09-10T22:02:26.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.844+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a41a7b10 con 0x7f48a4108620 2024-09-10T22:02:26.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.845+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a4102ff0 con 0x7f48a4108620 2024-09-10T22:02:26.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.847+0000 7f48a0c00640 1 -- 172.21.15.90:0/1879969211 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f488c005070 con 0x7f48a4108620 2024-09-10T22:02:26.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.848+0000 7f48a0c00640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f48880b5460 0x7f48880b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:26.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.849+0000 7f48a0c00640 1 -- 172.21.15.90:0/1879969211 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f488c0e58c0 con 0x7f48a4108620 2024-09-10T22:02:26.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.849+0000 7f48a3400640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f48880b5460 0x7f48880b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:26.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.849+0000 7f48a3400640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f48880b5460 0x7f48880b7920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4894005fd0 tx=0x7f4894005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:26.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:26.852+0000 7f48a0c00640 1 -- 172.21.15.90:0/1879969211 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f488c0ac1a0 con 0x7f48a4108620 2024-09-10T22:02:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:26 smithi171 ceph-mon[34138]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-10T22:02:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:26 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3772633727' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:02:27.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.043+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f48a419c1e0 con 0x7f48a4108620 2024-09-10T22:02:27.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.043+0000 7f48a0c00640 1 -- 172.21.15.90:0/1879969211 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f488c009de0 con 0x7f48a4108620 2024-09-10T22:02:27.045 INFO:teuthology.orchestra.run.smithi090.stdout:279172874283 2024-09-10T22:02:27.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f48880b5460 msgr2=0x7f48880b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f48880b5460 0x7f48880b7920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4894005fd0 tx=0x7f4894005850 comp rx=0 tx=0).stop 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108620 msgr2=0x7f48a419b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108620 0x7f48a419b5f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f488c00d950 tx=0x7f488c00de10 comp rx=0 tx=0).stop 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 shutdown_connections 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f48880b5460 0x7f48880b7920 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f48a4108fa0 0x7f48a41a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48a4108620 0x7f48a419b5f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 --2- 172.21.15.90:0/1879969211 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f48a4102c10 0x7f48a419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 >> 172.21.15.90:0/1879969211 conn(0x7f48a40fe6c0 msgr2=0x7f48a4075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 shutdown_connections 2024-09-10T22:02:27.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.046+0000 7f48a9000640 1 -- 172.21.15.90:0/1879969211 wait complete. 2024-09-10T22:02:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:26 smithi154 ceph-mon[34495]: pgmap v372: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-10T22:02:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:26 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3772633727' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T22:02:27.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.665+0000 7fcf7f800640 1 -- 172.21.15.90:0/3947476184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf78104fc0 msgr2=0x7fcf781053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.665+0000 7fcf7f800640 1 --2- 172.21.15.90:0/3947476184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf78104fc0 0x7fcf781053a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009900 tx=0x7fcf6c02a410 comp rx=0 tx=0).stop 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.665+0000 7fcf7f800640 1 -- 172.21.15.90:0/3947476184 shutdown_connections 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.665+0000 7fcf7f800640 1 --2- 172.21.15.90:0/3947476184 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf7810b350 0x7fcf78111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.665+0000 7fcf7f800640 1 --2- 172.21.15.90:0/3947476184 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf7810a9d0 0x7fcf7810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.665+0000 7fcf7f800640 1 --2- 172.21.15.90:0/3947476184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf78104fc0 0x7fcf781053a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.665+0000 7fcf7f800640 1 -- 172.21.15.90:0/3947476184 >> 172.21.15.90:0/3947476184 conn(0x7fcf781009f0 msgr2=0x7fcf78102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.665+0000 7fcf7f800640 1 -- 172.21.15.90:0/3947476184 shutdown_connections 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.665+0000 7fcf7f800640 1 -- 172.21.15.90:0/3947476184 wait complete. 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.666+0000 7fcf7f800640 1 Processor -- start 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.666+0000 7fcf7f800640 1 -- start start 2024-09-10T22:02:27.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.666+0000 7fcf7f800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf78104fc0 0x7fcf7819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.666+0000 7fcf7f800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf7810a9d0 0x7fcf7819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.666+0000 7fcf7f800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7810b350 0x7fcf781a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.666+0000 7fcf7f800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78114e30 con 0x7fcf7810b350 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.666+0000 7fcf7f800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78114cb0 con 0x7fcf7810a9d0 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.666+0000 7fcf7f800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78114fb0 con 0x7fcf78104fc0 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf7d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7810b350 0x7fcf781a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf7ce00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf78104fc0 0x7fcf7819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf7d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7810b350 0x7fcf781a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49602/0 (socket says 172.21.15.90:49602) 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf7d800640 1 -- 172.21.15.90:0/1720727983 learned_addr learned my addr 172.21.15.90:0/1720727983 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf77e00640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf7810a9d0 0x7fcf7819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf7d800640 1 -- 172.21.15.90:0/1720727983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf78104fc0 msgr2=0x7fcf7819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf7d800640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf78104fc0 0x7fcf7819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf7d800640 1 -- 172.21.15.90:0/1720727983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf7810a9d0 msgr2=0x7fcf7819b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:27.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf7d800640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf7810a9d0 0x7fcf7819b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.667+0000 7fcf7d800640 1 -- 172.21.15.90:0/1720727983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf781a1490 con 0x7fcf7810b350 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.668+0000 7fcf7d800640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7810b350 0x7fcf781a0cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000eaa0 tx=0x7fcf7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.668+0000 7fcf75600640 1 -- 172.21.15.90:0/1720727983 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7000cdb0 con 0x7fcf7810b350 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.668+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf781a1780 con 0x7fcf7810b350 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.668+0000 7fcf75600640 1 -- 172.21.15.90:0/1720727983 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf70005020 con 0x7fcf7810b350 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.668+0000 7fcf75600640 1 -- 172.21.15.90:0/1720727983 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf700116d0 con 0x7fcf7810b350 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.668+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf781a7e40 con 0x7fcf7810b350 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.669+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf781053a0 con 0x7fcf7810b350 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.671+0000 7fcf75600640 1 -- 172.21.15.90:0/1720727983 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fcf7000f040 con 0x7fcf7810b350 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.673+0000 7fcf75600640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf600b53c0 0x7fcf600b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.673+0000 7fcf75600640 1 -- 172.21.15.90:0/1720727983 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7fcf700e6080 con 0x7fcf7810b350 2024-09-10T22:02:27.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.673+0000 7fcf7ce00640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf600b53c0 0x7fcf600b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:27.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.674+0000 7fcf7ce00640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf600b53c0 0x7fcf600b7880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0057c0 tx=0x7fcf6c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:27.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.677+0000 7fcf75600640 1 -- 172.21.15.90:0/1720727983 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcf70016030 con 0x7fcf7810b350 2024-09-10T22:02:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:27 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1879969211' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:02:27.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.879+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcf7810cba0 con 0x7fcf7810b350 2024-09-10T22:02:27.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.879+0000 7fcf75600640 1 -- 172.21.15.90:0/1720727983 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf700ac310 con 0x7fcf7810b350 2024-09-10T22:02:27.881 INFO:teuthology.orchestra.run.smithi090.stdout:60129542245 2024-09-10T22:02:27.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.881+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf600b53c0 msgr2=0x7fcf600b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:27.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.881+0000 7fcf7f800640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf600b53c0 0x7fcf600b7880 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0057c0 tx=0x7fcf6c0058d0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.881+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7810b350 msgr2=0x7fcf781a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:27.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.881+0000 7fcf7f800640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7810b350 0x7fcf781a0cc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000eaa0 tx=0x7fcf7000ef60 comp rx=0 tx=0).stop 2024-09-10T22:02:27.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.882+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 shutdown_connections 2024-09-10T22:02:27.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.882+0000 7fcf7f800640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf600b53c0 0x7fcf600b7880 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.882+0000 7fcf7f800640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf7810b350 0x7fcf781a0cc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.882+0000 7fcf7f800640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf7810a9d0 0x7fcf7819b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.882+0000 7fcf7f800640 1 --2- 172.21.15.90:0/1720727983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf78104fc0 0x7fcf7819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:27.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.882+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 >> 172.21.15.90:0/1720727983 conn(0x7fcf781009f0 msgr2=0x7fcf78102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:27.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.882+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 shutdown_connections 2024-09-10T22:02:27.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:27.882+0000 7fcf7f800640 1 -- 172.21.15.90:0/1720727983 wait complete. 2024-09-10T22:02:27.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:27 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1879969211' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:02:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:27 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1879969211' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T22:02:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:28 smithi090 ceph-mon[30350]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-10T22:02:28.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:28 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1720727983' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:02:28.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:28 smithi171 ceph-mon[34138]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-10T22:02:28.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:28 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1720727983' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:02:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:28 smithi154 ceph-mon[34495]: pgmap v373: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-09-10T22:02:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:28 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1720727983' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T22:02:29.642 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542242 got 60129542245 for osd.1 2024-09-10T22:02:29.642 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:29.929 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:29 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1649832858' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:02:29.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:29 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1649832858' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:02:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.808+0000 7f6a2b200640 1 -- 172.21.15.90:0/2679225667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c104430 msgr2=0x7f6a2c10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.808+0000 7f6a2b200640 1 --2- 172.21.15.90:0/2679225667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c104430 0x7f6a2c10c920 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009900 tx=0x7f6a1402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.808+0000 7f6a2b200640 1 -- 172.21.15.90:0/2679225667 shutdown_connections 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.808+0000 7f6a2b200640 1 --2- 172.21.15.90:0/2679225667 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2c10cf70 0x7f6a2c10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.808+0000 7f6a2b200640 1 --2- 172.21.15.90:0/2679225667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c104430 0x7f6a2c10c920 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.808+0000 7f6a2b200640 1 --2- 172.21.15.90:0/2679225667 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a2c103b10 0x7f6a2c103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.808+0000 7f6a2b200640 1 -- 172.21.15.90:0/2679225667 >> 172.21.15.90:0/2679225667 conn(0x7f6a2c0fd5b0 msgr2=0x7f6a2c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.808+0000 7f6a2b200640 1 -- 172.21.15.90:0/2679225667 shutdown_connections 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.808+0000 7f6a2b200640 1 -- 172.21.15.90:0/2679225667 wait complete. 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.809+0000 7f6a2b200640 1 Processor -- start 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2b200640 1 -- start start 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a2c103b10 0x7f6a2c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2c104430 0x7f6a2c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c10cf70 0x7f6a2c1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c112ae0 con 0x7f6a2c10cf70 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c112960 con 0x7f6a2c104430 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6a2c112c60 con 0x7f6a2c103b10 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c10cf70 0x7f6a2c1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c10cf70 0x7f6a2c1a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41000/0 (socket says 172.21.15.90:41000) 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a29e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a2c103b10 0x7f6a2c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a2a800640 1 -- 172.21.15.90:0/1649832858 learned_addr learned my addr 172.21.15.90:0/1649832858 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.810+0000 7f6a21400640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2c104430 0x7f6a2c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a2a800640 1 -- 172.21.15.90:0/1649832858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a2c103b10 msgr2=0x7f6a2c19b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a2a800640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a2c103b10 0x7f6a2c19b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a2a800640 1 -- 172.21.15.90:0/1649832858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2c104430 msgr2=0x7f6a2c19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a2a800640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2c104430 0x7f6a2c19b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a2a800640 1 -- 172.21.15.90:0/1649832858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2c1a1210 con 0x7f6a2c10cf70 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a21400640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2c104430 0x7f6a2c19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a2a800640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c10cf70 0x7f6a2c1a0a40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c970 tx=0x7f6a2400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a23400640 1 -- 172.21.15.90:0/1649832858 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24007bb0 con 0x7f6a2c10cf70 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a23400640 1 -- 172.21.15.90:0/1649832858 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a24007d50 con 0x7f6a2c10cf70 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2c1a1500 con 0x7f6a2c10cf70 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a23400640 1 -- 172.21.15.90:0/1649832858 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24018660 con 0x7f6a2c10cf70 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.811+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a2c1a7bc0 con 0x7f6a2c10cf70 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.813+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a2c103ef0 con 0x7f6a2c10cf70 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.815+0000 7f6a23400640 1 -- 172.21.15.90:0/1649832858 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f6a24020030 con 0x7f6a2c10cf70 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.817+0000 7f6a23400640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6a100b5460 0x7f6a100b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.817+0000 7f6a29e00640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6a100b5460 0x7f6a100b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:30.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.817+0000 7f6a23400640 1 -- 172.21.15.90:0/1649832858 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f6a240e5f70 con 0x7f6a2c10cf70 2024-09-10T22:02:30.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.818+0000 7f6a29e00640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6a100b5460 0x7f6a100b7920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c005fd0 tx=0x7f6a1c009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:30.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:28.821+0000 7f6a23400640 1 -- 172.21.15.90:0/1649832858 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6a24014030 con 0x7f6a2c10cf70 2024-09-10T22:02:30.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.011+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6a2c1a7f10 con 0x7f6a2c10cf70 2024-09-10T22:02:30.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.012+0000 7f6a23400640 1 -- 172.21.15.90:0/1649832858 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6a240ac850 con 0x7f6a2c10cf70 2024-09-10T22:02:30.048 INFO:teuthology.orchestra.run.smithi090.stdout:137438953554 2024-09-10T22:02:30.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6a100b5460 msgr2=0x7f6a100b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6a100b5460 0x7f6a100b7920 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c005fd0 tx=0x7f6a1c009680 comp rx=0 tx=0).stop 2024-09-10T22:02:30.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c10cf70 msgr2=0x7f6a2c1a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c10cf70 0x7f6a2c1a0a40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400c970 tx=0x7f6a2400ce30 comp rx=0 tx=0).stop 2024-09-10T22:02:30.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 shutdown_connections 2024-09-10T22:02:30.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6a100b5460 0x7f6a100b7920 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a2c10cf70 0x7f6a2c1a0a40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6a2c104430 0x7f6a2c19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 --2- 172.21.15.90:0/1649832858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6a2c103b10 0x7f6a2c19b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 >> 172.21.15.90:0/1649832858 conn(0x7f6a2c0fd5b0 msgr2=0x7f6a2c10f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:30.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 shutdown_connections 2024-09-10T22:02:30.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:29.014+0000 7f6a2b200640 1 -- 172.21.15.90:0/1649832858 wait complete. 2024-09-10T22:02:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:29 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1649832858' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T22:02:30.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.726+0000 7f9566800640 1 -- 172.21.15.90:0/1503517298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560113990 msgr2=0x7f9560115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.726+0000 7f9566800640 1 --2- 172.21.15.90:0/1503517298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560113990 0x7f9560115d80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9558009ca0 tx=0x7f955802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.726+0000 7f9566800640 1 -- 172.21.15.90:0/1503517298 shutdown_connections 2024-09-10T22:02:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.726+0000 7f9566800640 1 --2- 172.21.15.90:0/1503517298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560113990 0x7f9560115d80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.726+0000 7f9566800640 1 --2- 172.21.15.90:0/1503517298 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9560077ff0 0x7f9560113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.726+0000 7f9566800640 1 --2- 172.21.15.90:0/1503517298 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f95600776d0 0x7f9560077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.726+0000 7f9566800640 1 -- 172.21.15.90:0/1503517298 >> 172.21.15.90:0/1503517298 conn(0x7f9560100a50 msgr2=0x7f9560102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.726+0000 7f9566800640 1 -- 172.21.15.90:0/1503517298 shutdown_connections 2024-09-10T22:02:30.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.726+0000 7f9566800640 1 -- 172.21.15.90:0/1503517298 wait complete. 2024-09-10T22:02:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.727+0000 7f9566800640 1 Processor -- start 2024-09-10T22:02:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.727+0000 7f9566800640 1 -- start start 2024-09-10T22:02:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.727+0000 7f9566800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95600776d0 0x7f956019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:30.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.727+0000 7f9566800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560077ff0 0x7f956019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.727+0000 7f9566800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9560113990 0x7f95601a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.727+0000 7f9566800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9560119260 con 0x7f9560077ff0 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.727+0000 7f9566800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f95601190e0 con 0x7f9560113990 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.727+0000 7f9566800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f95601193e0 con 0x7f95600776d0 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560077ff0 0x7f956019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560077ff0 0x7f956019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41012/0 (socket says 172.21.15.90:41012) 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955ea00640 1 -- 172.21.15.90:0/1868064121 learned_addr learned my addr 172.21.15.90:0/1868064121 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955fe00640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9560113990 0x7f95601a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955f400640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95600776d0 0x7f956019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955ea00640 1 -- 172.21.15.90:0/1868064121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95600776d0 msgr2=0x7f956019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955ea00640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95600776d0 0x7f956019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955ea00640 1 -- 172.21.15.90:0/1868064121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9560113990 msgr2=0x7f95601a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955ea00640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9560113990 0x7f95601a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.728+0000 7f955ea00640 1 -- 172.21.15.90:0/1868064121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95601a5840 con 0x7f9560077ff0 2024-09-10T22:02:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.729+0000 7f955ea00640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560077ff0 0x7f956019fcd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f954c00d950 tx=0x7f954c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.729+0000 7f9557e00640 1 -- 172.21.15.90:0/1868064121 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c017070 con 0x7f9560077ff0 2024-09-10T22:02:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.729+0000 7f9557e00640 1 -- 172.21.15.90:0/1868064121 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f954c009c30 con 0x7f9560077ff0 2024-09-10T22:02:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.729+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95601a5b30 con 0x7f9560077ff0 2024-09-10T22:02:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.729+0000 7f9557e00640 1 -- 172.21.15.90:0/1868064121 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c00f620 con 0x7f9560077ff0 2024-09-10T22:02:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.729+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95601ac1f0 con 0x7f9560077ff0 2024-09-10T22:02:30.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.730+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9560077ab0 con 0x7f9560077ff0 2024-09-10T22:02:30.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.733+0000 7f9557e00640 1 -- 172.21.15.90:0/1868064121 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f954c010460 con 0x7f9560077ff0 2024-09-10T22:02:30.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.735+0000 7f9557e00640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f95440b53c0 0x7f95440b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:30.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.735+0000 7f9557e00640 1 -- 172.21.15.90:0/1868064121 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f954c0e5dd0 con 0x7f9560077ff0 2024-09-10T22:02:30.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.735+0000 7f955f400640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f95440b53c0 0x7f95440b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:30.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.736+0000 7f955f400640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f95440b53c0 0x7f95440b7880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9550009bd0 tx=0x7f9550005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:30.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.739+0000 7f9557e00640 1 -- 172.21.15.90:0/1868064121 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f954c009de0 con 0x7f9560077ff0 2024-09-10T22:02:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.930+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f95601ac5c0 con 0x7f9560077ff0 2024-09-10T22:02:30.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.931+0000 7f9557e00640 1 -- 172.21.15.90:0/1868064121 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f954c0ac060 con 0x7f9560077ff0 2024-09-10T22:02:30.932 INFO:teuthology.orchestra.run.smithi090.stdout:111669149784 2024-09-10T22:02:30.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f95440b53c0 msgr2=0x7f95440b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f95440b53c0 0x7f95440b7880 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9550009bd0 tx=0x7f9550005f90 comp rx=0 tx=0).stop 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560077ff0 msgr2=0x7f956019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560077ff0 0x7f956019fcd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f954c00d950 tx=0x7f954c00de10 comp rx=0 tx=0).stop 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 shutdown_connections 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f95440b53c0 0x7f95440b7880 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9560113990 0x7f95601a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9560077ff0 0x7f956019fcd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 --2- 172.21.15.90:0/1868064121 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f95600776d0 0x7f956019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 >> 172.21.15.90:0/1868064121 conn(0x7f9560100a50 msgr2=0x7f9560102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 shutdown_connections 2024-09-10T22:02:30.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:30.933+0000 7f9566800640 1 -- 172.21.15.90:0/1868064121 wait complete. 2024-09-10T22:02:30.949 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:30 smithi090 ceph-mon[30350]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-10T22:02:30.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:30 smithi171 ceph-mon[34138]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-10T22:02:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:30 smithi154 ceph-mon[34495]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-10T22:02:31.262 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790025 got 158913790028 for osd.5 2024-09-10T22:02:31.262 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:31.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:31 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1868064121' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:02:31.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:31 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1868064121' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:02:32.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.924+0000 7f4517600640 1 -- 172.21.15.90:0/2946936386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010a9d0 msgr2=0x7f451010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:32.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.924+0000 7f4517600640 1 --2- 172.21.15.90:0/2946936386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010a9d0 0x7f451010ae10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f44f8009900 tx=0x7f44f802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.924+0000 7f4517600640 1 -- 172.21.15.90:0/2946936386 shutdown_connections 2024-09-10T22:02:32.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.924+0000 7f4517600640 1 --2- 172.21.15.90:0/2946936386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f451010b350 0x7f4510111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.924+0000 7f4517600640 1 --2- 172.21.15.90:0/2946936386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010a9d0 0x7f451010ae10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.924+0000 7f4517600640 1 --2- 172.21.15.90:0/2946936386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4510104fc0 0x7f45101053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.924+0000 7f4517600640 1 -- 172.21.15.90:0/2946936386 >> 172.21.15.90:0/2946936386 conn(0x7f45101009f0 msgr2=0x7f4510102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.924+0000 7f4517600640 1 -- 172.21.15.90:0/2946936386 shutdown_connections 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.924+0000 7f4517600640 1 -- 172.21.15.90:0/2946936386 wait complete. 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.925+0000 7f4517600640 1 Processor -- start 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.926+0000 7f4517600640 1 -- start start 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.926+0000 7f4517600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4510104fc0 0x7f45100776b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.926+0000 7f4517600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f451010a9d0 0x7f4510077bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.926+0000 7f4517600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010b350 0x7f4510078130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.926+0000 7f4517600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4510114e70 con 0x7f451010b350 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.926+0000 7f4517600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4510114cf0 con 0x7f451010a9d0 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.926+0000 7f4517600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4510114ff0 con 0x7f4510104fc0 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4515600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010b350 0x7f4510078130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4514c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4510104fc0 0x7f45100776b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4515600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010b350 0x7f4510078130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41036/0 (socket says 172.21.15.90:41036) 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4515600640 1 -- 172.21.15.90:0/2453627952 learned_addr learned my addr 172.21.15.90:0/2453627952 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:32.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4507e00640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f451010a9d0 0x7f4510077bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4515600640 1 -- 172.21.15.90:0/2453627952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4510104fc0 msgr2=0x7f45100776b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4515600640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4510104fc0 0x7f45100776b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4515600640 1 -- 172.21.15.90:0/2453627952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f451010a9d0 msgr2=0x7f4510077bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4515600640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f451010a9d0 0x7f4510077bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.927+0000 7f4515600640 1 -- 172.21.15.90:0/2453627952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451019b1b0 con 0x7f451010b350 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.928+0000 7f4515600640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010b350 0x7f4510078130 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4508009fd0 tx=0x7f450800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.928+0000 7f450e000640 1 -- 172.21.15.90:0/2453627952 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508017070 con 0x7f451010b350 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.928+0000 7f450e000640 1 -- 172.21.15.90:0/2453627952 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4508003e00 con 0x7f451010b350 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.928+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451019b4a0 con 0x7f451010b350 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.928+0000 7f450e000640 1 -- 172.21.15.90:0/2453627952 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508010460 con 0x7f451010b350 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.928+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451019b960 con 0x7f451010b350 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.929+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45101053a0 con 0x7f451010b350 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.932+0000 7f450e000640 1 -- 172.21.15.90:0/2453627952 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4508003970 con 0x7f451010b350 2024-09-10T22:02:32.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.933+0000 7f450e000640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44f40b53c0 0x7f44f40b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:32.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.934+0000 7f450e000640 1 -- 172.21.15.90:0/2453627952 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f45080e6350 con 0x7f451010b350 2024-09-10T22:02:32.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.934+0000 7f4514c00640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44f40b53c0 0x7f44f40b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:32.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.935+0000 7f4514c00640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44f40b53c0 0x7f44f40b7880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4500009bd0 tx=0x7f4500005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:32.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:31.935+0000 7f450e000640 1 -- 172.21.15.90:0/2453627952 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f45080ac5e0 con 0x7f451010b350 2024-09-10T22:02:32.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.133+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f45100789b0 con 0x7f451010b350 2024-09-10T22:02:32.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.133+0000 7f450e000640 1 -- 172.21.15.90:0/2453627952 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f45100789b0 con 0x7f451010b350 2024-09-10T22:02:32.135 INFO:teuthology.orchestra.run.smithi090.stdout:257698037810 2024-09-10T22:02:32.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44f40b53c0 msgr2=0x7f44f40b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:32.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44f40b53c0 0x7f44f40b7880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4500009bd0 tx=0x7f4500005f90 comp rx=0 tx=0).stop 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010b350 msgr2=0x7f4510078130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010b350 0x7f4510078130 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4508009fd0 tx=0x7f450800b260 comp rx=0 tx=0).stop 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 shutdown_connections 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44f40b53c0 0x7f44f40b7880 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f451010b350 0x7f4510078130 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f451010a9d0 0x7f4510077bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 --2- 172.21.15.90:0/2453627952 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4510104fc0 0x7f45100776b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 >> 172.21.15.90:0/2453627952 conn(0x7f45101009f0 msgr2=0x7f4510102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 shutdown_connections 2024-09-10T22:02:32.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.136+0000 7f4517600640 1 -- 172.21.15.90:0/2453627952 wait complete. 2024-09-10T22:02:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:31 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1868064121' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T22:02:32.610 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953552 got 137438953554 for osd.4 2024-09-10T22:02:32.611 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:32.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:32 smithi090 ceph-mon[30350]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T22:02:32.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:32 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2453627952' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:02:32.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:32 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3207179419' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:02:32.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:32 smithi171 ceph-mon[34138]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T22:02:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:32 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2453627952' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:02:32.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:32 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3207179419' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:02:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:32 smithi154 ceph-mon[34495]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T22:02:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:32 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2453627952' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T22:02:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:32 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3207179419' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T22:02:33.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.281+0000 7f005d200640 1 -- 172.21.15.90:0/3431385314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 msgr2=0x7f00581051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:33.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.281+0000 7f005d200640 1 --2- 172.21.15.90:0/3431385314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 0x7f00581051d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0048009ca0 tx=0x7f004802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.281+0000 7f005d200640 1 -- 172.21.15.90:0/3431385314 shutdown_connections 2024-09-10T22:02:33.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.281+0000 7f005d200640 1 --2- 172.21.15.90:0/3431385314 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f005810b180 0x7f0058111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.281+0000 7f005d200640 1 --2- 172.21.15.90:0/3431385314 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f005810a800 0x7f005810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.281+0000 7f005d200640 1 --2- 172.21.15.90:0/3431385314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 0x7f00581051d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.281+0000 7f005d200640 1 -- 172.21.15.90:0/3431385314 >> 172.21.15.90:0/3431385314 conn(0x7f0058100840 msgr2=0x7f0058102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:33.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.281+0000 7f005d200640 1 -- 172.21.15.90:0/3431385314 shutdown_connections 2024-09-10T22:02:33.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.281+0000 7f005d200640 1 -- 172.21.15.90:0/3431385314 wait complete. 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.282+0000 7f005d200640 1 Processor -- start 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.282+0000 7f005d200640 1 -- start start 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f005d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 0x7f005819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f005d200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f005810a800 0x7f005819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f005d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f005810b180 0x7f00581a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f005d200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0058114d40 con 0x7f0058104df0 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f005d200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0058114bc0 con 0x7f005810a800 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f005d200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0058114ec0 con 0x7f005810b180 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f0057400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 0x7f005819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f0057400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 0x7f005819b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41044/0 (socket says 172.21.15.90:41044) 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f0057400640 1 -- 172.21.15.90:0/3207179419 learned_addr learned my addr 172.21.15.90:0/3207179419 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f004ea00640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f005810a800 0x7f005819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:33.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f0057e00640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f005810b180 0x7f00581a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f0057400640 1 -- 172.21.15.90:0/3207179419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f005810b180 msgr2=0x7f00581a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f0057400640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f005810b180 0x7f00581a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f0057400640 1 -- 172.21.15.90:0/3207179419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f005810a800 msgr2=0x7f005819b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f0057400640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f005810a800 0x7f005819b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.283+0000 7f0057400640 1 -- 172.21.15.90:0/3207179419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00581a1240 con 0x7f0058104df0 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.284+0000 7f0057400640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 0x7f005819b190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0048009dc0 tx=0x7f0048032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.284+0000 7f0054c00640 1 -- 172.21.15.90:0/3207179419 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048004040 con 0x7f0058104df0 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.284+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00581a14d0 con 0x7f0058104df0 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.284+0000 7f0054c00640 1 -- 172.21.15.90:0/3207179419 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0048004670 con 0x7f0058104df0 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.284+0000 7f0054c00640 1 -- 172.21.15.90:0/3207179419 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048030c80 con 0x7f0058104df0 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.284+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00581a7bf0 con 0x7f0058104df0 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.285+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00581051d0 con 0x7f0058104df0 2024-09-10T22:02:33.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.287+0000 7f0054c00640 1 -- 172.21.15.90:0/3207179419 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f00480041e0 con 0x7f0058104df0 2024-09-10T22:02:33.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.289+0000 7f0054c00640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f003c0b54e0 0x7f003c0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:33.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.289+0000 7f0054c00640 1 -- 172.21.15.90:0/3207179419 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f00481075f0 con 0x7f0058104df0 2024-09-10T22:02:33.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.289+0000 7f004ea00640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f003c0b54e0 0x7f003c0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:33.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.290+0000 7f004ea00640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f003c0b54e0 0x7f003c0b79a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f005810a3d0 tx=0x7f004000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:33.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.292+0000 7f0054c00640 1 -- 172.21.15.90:0/3207179419 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f00480cded0 con 0x7f0058104df0 2024-09-10T22:02:33.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.481+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f00581a7eb0 con 0x7f0058104df0 2024-09-10T22:02:33.314 INFO:teuthology.orchestra.run.smithi090.stdout:231928234042 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.482+0000 7f0054c00640 1 -- 172.21.15.90:0/3207179419 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f00480f0090 con 0x7f0058104df0 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f003c0b54e0 msgr2=0x7f003c0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f003c0b54e0 0x7f003c0b79a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f005810a3d0 tx=0x7f004000b390 comp rx=0 tx=0).stop 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 msgr2=0x7f005819b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 0x7f005819b190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0048009dc0 tx=0x7f0048032bc0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 shutdown_connections 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f003c0b54e0 0x7f003c0b79a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f005810b180 0x7f00581a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f005810a800 0x7f005819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 --2- 172.21.15.90:0/3207179419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058104df0 0x7f005819b190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 >> 172.21.15.90:0/3207179419 conn(0x7f0058100840 msgr2=0x7f0058101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 shutdown_connections 2024-09-10T22:02:33.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:32.484+0000 7f005d200640 1 -- 172.21.15.90:0/3207179419 wait complete. 2024-09-10T22:02:34.097 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234040 got 231928234042 for osd.8 2024-09-10T22:02:34.098 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:34.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.533+0000 7f4553400640 1 -- 172.21.15.90:0/2046112058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c104fa0 msgr2=0x7f454c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.533+0000 7f4553400640 1 --2- 172.21.15.90:0/2046112058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c104fa0 0x7f454c105380 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f453c009900 tx=0x7f453c02a410 comp rx=0 tx=0).stop 2024-09-10T22:02:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.534+0000 7f4553400640 1 -- 172.21.15.90:0/2046112058 shutdown_connections 2024-09-10T22:02:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.534+0000 7f4553400640 1 --2- 172.21.15.90:0/2046112058 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c10b330 0x7f454c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.534+0000 7f4553400640 1 --2- 172.21.15.90:0/2046112058 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f454c10a9b0 0x7f454c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.534+0000 7f4553400640 1 --2- 172.21.15.90:0/2046112058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c104fa0 0x7f454c105380 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.534+0000 7f4553400640 1 -- 172.21.15.90:0/2046112058 >> 172.21.15.90:0/2046112058 conn(0x7f454c100a50 msgr2=0x7f454c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.534+0000 7f4553400640 1 -- 172.21.15.90:0/2046112058 shutdown_connections 2024-09-10T22:02:34.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.534+0000 7f4553400640 1 -- 172.21.15.90:0/2046112058 wait complete. 2024-09-10T22:02:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.535+0000 7f4553400640 1 Processor -- start 2024-09-10T22:02:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.535+0000 7f4553400640 1 -- start start 2024-09-10T22:02:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.535+0000 7f4553400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c104fa0 0x7f454c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:34.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.535+0000 7f4553400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c10a9b0 0x7f454c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.535+0000 7f4553400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f454c10b330 0x7f454c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.535+0000 7f4553400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f454c114e70 con 0x7f454c10a9b0 2024-09-10T22:02:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.535+0000 7f4553400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f454c114cf0 con 0x7f454c104fa0 2024-09-10T22:02:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.535+0000 7f4553400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f454c114ff0 con 0x7f454c10b330 2024-09-10T22:02:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4543e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c10a9b0 0x7f454c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4543e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c10a9b0 0x7f454c19b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41064/0 (socket says 172.21.15.90:41064) 2024-09-10T22:02:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4543e00640 1 -- 172.21.15.90:0/3217258386 learned_addr learned my addr 172.21.15.90:0/3217258386 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4550a00640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c104fa0 0x7f454c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:34.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4551400640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f454c10b330 0x7f454c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4543e00640 1 -- 172.21.15.90:0/3217258386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f454c10b330 msgr2=0x7f454c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4543e00640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f454c10b330 0x7f454c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4543e00640 1 -- 172.21.15.90:0/3217258386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c104fa0 msgr2=0x7f454c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4543e00640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c104fa0 0x7f454c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.536+0000 7f4543e00640 1 -- 172.21.15.90:0/3217258386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454c1a14b0 con 0x7f454c10a9b0 2024-09-10T22:02:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.537+0000 7f4543e00640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c10a9b0 0x7f454c19b940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f453400d950 tx=0x7f453400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:34.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.537+0000 7f454a000640 1 -- 172.21.15.90:0/3217258386 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534017070 con 0x7f454c10a9b0 2024-09-10T22:02:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.537+0000 7f454a000640 1 -- 172.21.15.90:0/3217258386 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4534009c30 con 0x7f454c10a9b0 2024-09-10T22:02:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.537+0000 7f454a000640 1 -- 172.21.15.90:0/3217258386 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453400f620 con 0x7f454c10a9b0 2024-09-10T22:02:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.537+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454c1a17a0 con 0x7f454c10a9b0 2024-09-10T22:02:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.537+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f454c1a7ee0 con 0x7f454c10a9b0 2024-09-10T22:02:34.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.538+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f454c105380 con 0x7f454c10a9b0 2024-09-10T22:02:34.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.540+0000 7f454a000640 1 -- 172.21.15.90:0/3217258386 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4534010460 con 0x7f454c10a9b0 2024-09-10T22:02:34.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.542+0000 7f454a000640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45300b5440 0x7f45300b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:34.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.542+0000 7f454a000640 1 -- 172.21.15.90:0/3217258386 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9286+0+0 (secure 0 0 0) 0x7f45340e5d50 con 0x7f454c10a9b0 2024-09-10T22:02:34.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.542+0000 7f4550a00640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45300b5440 0x7f45300b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:34.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.543+0000 7f4550a00640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45300b5440 0x7f45300b7900 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f453c0057c0 tx=0x7f453c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:34.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.546+0000 7f454a000640 1 -- 172.21.15.90:0/3217258386 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4534009de0 con 0x7f454c10a9b0 2024-09-10T22:02:34.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.740+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f454c02e5c0 con 0x7f454c10a9b0 2024-09-10T22:02:34.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.740+0000 7f454a000640 1 -- 172.21.15.90:0/3217258386 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f45340abfe0 con 0x7f454c10a9b0 2024-09-10T22:02:34.742 INFO:teuthology.orchestra.run.smithi090.stdout:38654705771 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45300b5440 msgr2=0x7f45300b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45300b5440 0x7f45300b7900 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f453c0057c0 tx=0x7f453c0058d0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c10a9b0 msgr2=0x7f454c19b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c10a9b0 0x7f454c19b940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f453400d950 tx=0x7f453400de10 comp rx=0 tx=0).stop 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 shutdown_connections 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45300b5440 0x7f45300b7900 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f454c10b330 0x7f454c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f454c10a9b0 0x7f454c19b940 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 --2- 172.21.15.90:0/3217258386 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f454c104fa0 0x7f454c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 >> 172.21.15.90:0/3217258386 conn(0x7f454c100a50 msgr2=0x7f454c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 shutdown_connections 2024-09-10T22:02:34.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:34.743+0000 7f4553400640 1 -- 172.21.15.90:0/3217258386 wait complete. 2024-09-10T22:02:34.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:34 smithi090 ceph-mon[30350]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:02:34.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-10T22:02:34.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]} : dispatch 2024-09-10T22:02:34.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]} : dispatch 2024-09-10T22:02:34.963 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:34 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]} : dispatch 2024-09-10T22:02:34.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:34 smithi171 ceph-mon[34138]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:02:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-10T22:02:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]} : dispatch 2024-09-10T22:02:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]} : dispatch 2024-09-10T22:02:34.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:34 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]} : dispatch 2024-09-10T22:02:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:34 smithi154 ceph-mon[34495]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:02:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-10T22:02:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]} : dispatch 2024-09-10T22:02:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]} : dispatch 2024-09-10T22:02:35.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:34 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]} : dispatch 2024-09-10T22:02:35.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-10T22:02:35.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-09-10T22:02:35.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-10T22:02:35.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:35 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-09-10T22:02:35.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:35 smithi090 ceph-mon[30350]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T22:02:35.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:35 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3217258386' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:02:35.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-10T22:02:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-09-10T22:02:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-10T22:02:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:35 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-09-10T22:02:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:35 smithi171 ceph-mon[34138]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T22:02:35.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:35 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3217258386' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:02:36.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037807 got 257698037810 for osd.9 2024-09-10T22:02:36.030 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-10T22:02:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-09-10T22:02:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-09-10T22:02:36.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:35 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-09-10T22:02:36.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:35 smithi154 ceph-mon[34495]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T22:02:36.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:35 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3217258386' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T22:02:36.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.197+0000 7f1faa000640 1 -- 172.21.15.90:0/171751864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410b180 msgr2=0x7f1fa4111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:36.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.197+0000 7f1faa000640 1 --2- 172.21.15.90:0/171751864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410b180 0x7f1fa4111980 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009ca0 tx=0x7f1f9c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.197+0000 7f1faa000640 1 -- 172.21.15.90:0/171751864 shutdown_connections 2024-09-10T22:02:36.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.197+0000 7f1faa000640 1 --2- 172.21.15.90:0/171751864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410b180 0x7f1fa4111980 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.197+0000 7f1faa000640 1 --2- 172.21.15.90:0/171751864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fa410a800 0x7f1fa410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.197+0000 7f1faa000640 1 --2- 172.21.15.90:0/171751864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa4104df0 0x7f1fa41051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.197+0000 7f1faa000640 1 -- 172.21.15.90:0/171751864 >> 172.21.15.90:0/171751864 conn(0x7f1fa4100840 msgr2=0x7f1fa4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:36.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.197+0000 7f1faa000640 1 -- 172.21.15.90:0/171751864 shutdown_connections 2024-09-10T22:02:36.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.197+0000 7f1faa000640 1 -- 172.21.15.90:0/171751864 wait complete. 2024-09-10T22:02:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.198+0000 7f1faa000640 1 Processor -- start 2024-09-10T22:02:36.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1faa000640 1 -- start start 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1faa000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fa4104df0 0x7f1fa419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1faa000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410a800 0x7f1fa419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1faa000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa410b180 0x7f1fa41a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1faa000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4114d40 con 0x7f1fa410a800 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1faa000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4114bc0 con 0x7f1fa410b180 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1faa000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4114ec0 con 0x7f1fa4104df0 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1fa3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410a800 0x7f1fa419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1fa3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410a800 0x7f1fa419b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41074/0 (socket says 172.21.15.90:41074) 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1fa3e00640 1 -- 172.21.15.90:0/2102954098 learned_addr learned my addr 172.21.15.90:0/2102954098 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1fa9600640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa410b180 0x7f1fa41a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.199+0000 7f1fa8c00640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fa4104df0 0x7f1fa419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:36.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1fa3e00640 1 -- 172.21.15.90:0/2102954098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fa4104df0 msgr2=0x7f1fa419b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1fa3e00640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fa4104df0 0x7f1fa419b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1fa3e00640 1 -- 172.21.15.90:0/2102954098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa410b180 msgr2=0x7f1fa41a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1fa3e00640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa410b180 0x7f1fa41a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1fa3e00640 1 -- 172.21.15.90:0/2102954098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa41a11a0 con 0x7f1fa410a800 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1fa3e00640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410a800 0x7f1fa419b630 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000d770 tx=0x7f1f9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1fa1600640 1 -- 172.21.15.90:0/2102954098 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90017070 con 0x7f1fa410a800 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1fa1600640 1 -- 172.21.15.90:0/2102954098 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1f900054b0 con 0x7f1fa410a800 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1fa1600640 1 -- 172.21.15.90:0/2102954098 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90004c60 con 0x7f1fa410a800 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa41a1490 con 0x7f1fa410a800 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.200+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa41a7b50 con 0x7f1fa410a800 2024-09-10T22:02:36.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.201+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fa41051d0 con 0x7f1fa410a800 2024-09-10T22:02:36.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.204+0000 7f1fa1600640 1 -- 172.21.15.90:0/2102954098 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f1f90004e00 con 0x7f1fa410a800 2024-09-10T22:02:36.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.206+0000 7f1fa1600640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1f8c0b54e0 0x7f1f8c0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:36.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.206+0000 7f1fa1600640 1 -- 172.21.15.90:0/2102954098 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f1f900e5cc0 con 0x7f1fa410a800 2024-09-10T22:02:36.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.206+0000 7f1fa8c00640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1f8c0b54e0 0x7f1f8c0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:36.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.207+0000 7f1fa8c00640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1f8c0b54e0 0x7f1f8c0b79a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009b40 tx=0x7f1f98005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:36.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.210+0000 7f1fa1600640 1 -- 172.21.15.90:0/2102954098 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1f900ec080 con 0x7f1fa410a800 2024-09-10T22:02:36.281 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874280 got 279172874283 for osd.10 2024-09-10T22:02:36.281 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:36.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.409+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1fa41a7f70 con 0x7f1fa410a800 2024-09-10T22:02:36.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.410+0000 7f1fa1600640 1 -- 172.21.15.90:0/2102954098 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1f900ac530 con 0x7f1fa410a800 2024-09-10T22:02:36.412 INFO:teuthology.orchestra.run.smithi090.stdout:81604378720 2024-09-10T22:02:36.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1f8c0b54e0 msgr2=0x7f1f8c0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1f8c0b54e0 0x7f1f8c0b79a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009b40 tx=0x7f1f98005f90 comp rx=0 tx=0).stop 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410a800 msgr2=0x7f1fa419b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410a800 0x7f1fa419b630 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000d770 tx=0x7f1f9000dc30 comp rx=0 tx=0).stop 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 shutdown_connections 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1f8c0b54e0 0x7f1f8c0b79a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1fa410b180 0x7f1fa41a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa410a800 0x7f1fa419b630 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 --2- 172.21.15.90:0/2102954098 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1fa4104df0 0x7f1fa419b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 >> 172.21.15.90:0/2102954098 conn(0x7f1fa4100840 msgr2=0x7f1fa4103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:36.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 shutdown_connections 2024-09-10T22:02:36.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:36.412+0000 7f1faa000640 1 -- 172.21.15.90:0/2102954098 wait complete. 2024-09-10T22:02:36.596 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:36 smithi090 ceph-mon[30350]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T22:02:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:36 smithi090 ceph-mon[30350]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T22:02:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:36 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2102954098' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:02:36.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:36 smithi171 ceph-mon[34138]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T22:02:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:36 smithi171 ceph-mon[34138]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T22:02:36.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:36 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2102954098' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:02:37.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.040+0000 7fb0cda00640 1 -- 172.21.15.90:0/2773282276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 msgr2=0x7fb0c8105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:37.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.040+0000 7fb0cda00640 1 --2- 172.21.15.90:0/2773282276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 0x7fb0c8105380 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009900 tx=0x7fb0b802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.040+0000 7fb0cda00640 1 -- 172.21.15.90:0/2773282276 shutdown_connections 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.040+0000 7fb0cda00640 1 --2- 172.21.15.90:0/2773282276 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0c810b330 0x7fb0c8111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.040+0000 7fb0cda00640 1 --2- 172.21.15.90:0/2773282276 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0c810a9b0 0x7fb0c810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.040+0000 7fb0cda00640 1 --2- 172.21.15.90:0/2773282276 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 0x7fb0c8105380 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.040+0000 7fb0cda00640 1 -- 172.21.15.90:0/2773282276 >> 172.21.15.90:0/2773282276 conn(0x7fb0c8100a50 msgr2=0x7fb0c8102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.040+0000 7fb0cda00640 1 -- 172.21.15.90:0/2773282276 shutdown_connections 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.040+0000 7fb0cda00640 1 -- 172.21.15.90:0/2773282276 wait complete. 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.041+0000 7fb0cda00640 1 Processor -- start 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.041+0000 7fb0cda00640 1 -- start start 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.041+0000 7fb0cda00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 0x7fb0c819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.041+0000 7fb0cda00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0c810a9b0 0x7fb0c819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.041+0000 7fb0cda00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0c810b330 0x7fb0c81a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:37.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.041+0000 7fb0cda00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8114e60 con 0x7fb0c8104fa0 2024-09-10T22:02:37.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.041+0000 7fb0cda00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8114ce0 con 0x7fb0c810a9b0 2024-09-10T22:02:37.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.041+0000 7fb0cda00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c8114fe0 con 0x7fb0c810b330 2024-09-10T22:02:37.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.041+0000 7fb0c6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 0x7fb0c819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:37.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 0x7fb0c819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41092/0 (socket says 172.21.15.90:41092) 2024-09-10T22:02:37.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c6a00640 1 -- 172.21.15.90:0/870842865 learned_addr learned my addr 172.21.15.90:0/870842865 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:37.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c6000640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0c810a9b0 0x7fb0c819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:37.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c7400640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0c810b330 0x7fb0c81a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c6a00640 1 -- 172.21.15.90:0/870842865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0c810b330 msgr2=0x7fb0c81a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c6a00640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0c810b330 0x7fb0c81a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c6a00640 1 -- 172.21.15.90:0/870842865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0c810a9b0 msgr2=0x7fb0c819b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c6a00640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0c810a9b0 0x7fb0c819b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c6a00640 1 -- 172.21.15.90:0/870842865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c81a1460 con 0x7fb0c8104fa0 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c7400640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0c810b330 0x7fb0c81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.042+0000 7fb0c6a00640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 0x7fb0c819b3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb0b802a910 tx=0x7fb0b8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.043+0000 7fb0bf400640 1 -- 172.21.15.90:0/870842865 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b803f070 con 0x7fb0c8104fa0 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.043+0000 7fb0bf400640 1 -- 172.21.15.90:0/870842865 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb0b803baf0 con 0x7fb0c8104fa0 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.043+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c81a16f0 con 0x7fb0c8104fa0 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.043+0000 7fb0bf400640 1 -- 172.21.15.90:0/870842865 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8035030 con 0x7fb0c8104fa0 2024-09-10T22:02:37.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.043+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c81a7e10 con 0x7fb0c8104fa0 2024-09-10T22:02:37.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.044+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c8105380 con 0x7fb0c8104fa0 2024-09-10T22:02:37.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.046+0000 7fb0bf400640 1 -- 172.21.15.90:0/870842865 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb0b803b660 con 0x7fb0c8104fa0 2024-09-10T22:02:37.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.047+0000 7fb0bf400640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb0ac0b5440 0x7fb0ac0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:37.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.048+0000 7fb0bf400640 1 -- 172.21.15.90:0/870842865 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb0b8106ff0 con 0x7fb0c8104fa0 2024-09-10T22:02:37.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.048+0000 7fb0c6000640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb0ac0b5440 0x7fb0ac0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:37.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.049+0000 7fb0c6000640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb0ac0b5440 0x7fb0ac0b7900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0c810a160 tx=0x7fb0b4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:37.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.050+0000 7fb0bf400640 1 -- 172.21.15.90:0/870842865 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb0b80cd8a0 con 0x7fb0c8104fa0 2024-09-10T22:02:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:36 smithi154 ceph-mon[34495]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T22:02:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:36 smithi154 ceph-mon[34495]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T22:02:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:36 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2102954098' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T22:02:37.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.242+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb0c819c4e0 con 0x7fb0c8104fa0 2024-09-10T22:02:37.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.242+0000 7fb0bf400640 1 -- 172.21.15.90:0/870842865 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb0b80cd1a0 con 0x7fb0c8104fa0 2024-09-10T22:02:37.244 INFO:teuthology.orchestra.run.smithi090.stdout:304942678056 2024-09-10T22:02:37.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.244+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb0ac0b5440 msgr2=0x7fb0ac0b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:37.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.244+0000 7fb0cda00640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb0ac0b5440 0x7fb0ac0b7900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0c810a160 tx=0x7fb0b4005f90 comp rx=0 tx=0).stop 2024-09-10T22:02:37.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.244+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 msgr2=0x7fb0c819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.244+0000 7fb0cda00640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 0x7fb0c819b3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb0b802a910 tx=0x7fb0b8003b90 comp rx=0 tx=0).stop 2024-09-10T22:02:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.245+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 shutdown_connections 2024-09-10T22:02:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.245+0000 7fb0cda00640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb0ac0b5440 0x7fb0ac0b7900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.245+0000 7fb0cda00640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb0c810b330 0x7fb0c81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.245+0000 7fb0cda00640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb0c810a9b0 0x7fb0c819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.245+0000 7fb0cda00640 1 --2- 172.21.15.90:0/870842865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0c8104fa0 0x7fb0c819b3b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.245+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 >> 172.21.15.90:0/870842865 conn(0x7fb0c8100a50 msgr2=0x7fb0c81013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.245+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 shutdown_connections 2024-09-10T22:02:37.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:37.245+0000 7fb0cda00640 1 -- 172.21.15.90:0/870842865 wait complete. 2024-09-10T22:02:37.990 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:37 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/870842865' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:02:37.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:37 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/870842865' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:02:38.029 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705767 got 38654705771 for osd.0 2024-09-10T22:02:38.029 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:38.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:37 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/870842865' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T22:02:38.706 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149781 got 111669149784 for osd.3 2024-09-10T22:02:38.707 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:38.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:38 smithi171 ceph-mon[34138]: pgmap v380: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-09-10T22:02:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:38 smithi171 ceph-mon[34138]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T22:02:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:02:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:38 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:39.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:38 smithi090 ceph-mon[30350]: pgmap v380: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-09-10T22:02:39.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:38 smithi090 ceph-mon[30350]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T22:02:39.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:02:39.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:39.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:39.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:39.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:38 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:39.121 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378716 got 81604378720 for osd.2 2024-09-10T22:02:39.121 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:38 smithi154 ceph-mon[34495]: pgmap v380: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-09-10T22:02:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:38 smithi154 ceph-mon[34495]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T22:02:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:02:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:38 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:39.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.764+0000 7f9518600640 1 -- 172.21.15.90:0/3449075822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510105570 msgr2=0x7f95101059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.764+0000 7f9518600640 1 --2- 172.21.15.90:0/3449075822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510105570 0x7f95101059b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009900 tx=0x7f94fc02a410 comp rx=0 tx=0).stop 2024-09-10T22:02:39.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.764+0000 7f9518600640 1 -- 172.21.15.90:0/3449075822 shutdown_connections 2024-09-10T22:02:39.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.764+0000 7f9518600640 1 --2- 172.21.15.90:0/3449075822 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510105ef0 0x7f951010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.764+0000 7f9518600640 1 --2- 172.21.15.90:0/3449075822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510105570 0x7f95101059b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.764+0000 7f9518600640 1 --2- 172.21.15.90:0/3449075822 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9510101220 0x7f9510101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.764+0000 7f9518600640 1 -- 172.21.15.90:0/3449075822 >> 172.21.15.90:0/3449075822 conn(0x7f9510078100 msgr2=0x7f95100ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:39.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.765+0000 7f9518600640 1 -- 172.21.15.90:0/3449075822 shutdown_connections 2024-09-10T22:02:39.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.765+0000 7f9518600640 1 -- 172.21.15.90:0/3449075822 wait complete. 2024-09-10T22:02:39.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.765+0000 7f9518600640 1 Processor -- start 2024-09-10T22:02:39.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9518600640 1 -- start start 2024-09-10T22:02:39.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9518600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510101220 0x7f951019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9518600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9510105570 0x7f951019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9518600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510105ef0 0x7f95101a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9518600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9510112cb0 con 0x7f9510101220 2024-09-10T22:02:39.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9518600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9510112b30 con 0x7f9510105ef0 2024-09-10T22:02:39.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9518600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9510112e30 con 0x7f9510105570 2024-09-10T22:02:39.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9515c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510101220 0x7f951019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9515c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510101220 0x7f951019b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52698/0 (socket says 172.21.15.90:52698) 2024-09-10T22:02:39.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9515c00640 1 -- 172.21.15.90:0/1053540682 learned_addr learned my addr 172.21.15.90:0/1053540682 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:39.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.766+0000 7f9516600640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510105ef0 0x7f95101a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9515200640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9510105570 0x7f951019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9515c00640 1 -- 172.21.15.90:0/1053540682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9510105570 msgr2=0x7f951019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9515c00640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9510105570 0x7f951019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9515c00640 1 -- 172.21.15.90:0/1053540682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510105ef0 msgr2=0x7f95101a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9515c00640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510105ef0 0x7f95101a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9515c00640 1 -- 172.21.15.90:0/1053540682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95101a1510 con 0x7f9510101220 2024-09-10T22:02:39.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9515200640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9510105570 0x7f951019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:39.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9516600640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510105ef0 0x7f95101a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:39.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9515c00640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510101220 0x7f951019b460 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f950400ea80 tx=0x7f950400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:39.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f950e000640 1 -- 172.21.15.90:0/1053540682 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950400ce40 con 0x7f9510101220 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f950e000640 1 -- 172.21.15.90:0/1053540682 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9504005020 con 0x7f9510101220 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95101a1800 con 0x7f9510101220 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f950e000640 1 -- 172.21.15.90:0/1053540682 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95040116d0 con 0x7f9510101220 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.767+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95101a7ec0 con 0x7f9510101220 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.768+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9510101600 con 0x7f9510101220 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.770+0000 7f950e000640 1 -- 172.21.15.90:0/1053540682 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f950400f040 con 0x7f9510101220 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.772+0000 7f950e000640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f94f80b53c0 0x7f94f80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.772+0000 7f950e000640 1 -- 172.21.15.90:0/1053540682 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f95040e65d0 con 0x7f9510101220 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.772+0000 7f9515200640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f94f80b53c0 0x7f94f80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.773+0000 7f9515200640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f94f80b53c0 0x7f94f80b7880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f94fc02a930 tx=0x7f94fc005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:39.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.775+0000 7f950e000640 1 -- 172.21.15.90:0/1053540682 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9504016030 con 0x7f9510101220 2024-09-10T22:02:39.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.964+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f95101a8300 con 0x7f9510101220 2024-09-10T22:02:39.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.964+0000 7f950e000640 1 -- 172.21.15.90:0/1053540682 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f95040ace40 con 0x7f9510101220 2024-09-10T22:02:39.966 INFO:teuthology.orchestra.run.smithi090.stdout:206158430274 2024-09-10T22:02:39.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f94f80b53c0 msgr2=0x7f94f80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f94f80b53c0 0x7f94f80b7880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f94fc02a930 tx=0x7f94fc005fd0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510101220 msgr2=0x7f951019b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510101220 0x7f951019b460 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f950400ea80 tx=0x7f950400ef40 comp rx=0 tx=0).stop 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 shutdown_connections 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f94f80b53c0 0x7f94f80b7880 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9510105ef0 0x7f95101a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9510105570 0x7f951019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 --2- 172.21.15.90:0/1053540682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9510101220 0x7f951019b460 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 >> 172.21.15.90:0/1053540682 conn(0x7f9510078100 msgr2=0x7f9510108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 shutdown_connections 2024-09-10T22:02:39.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:39.967+0000 7f9518600640 1 -- 172.21.15.90:0/1053540682 wait complete. 2024-09-10T22:02:40.374 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678051 got 304942678056 for osd.11 2024-09-10T22:02:40.375 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:40.648 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:40 smithi171 ceph-mon[34138]: pgmap v381: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-09-10T22:02:40.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:40 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1053540682' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:02:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:40 smithi090 ceph-mon[30350]: pgmap v381: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-09-10T22:02:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:40 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1053540682' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:02:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:40 smithi154 ceph-mon[34495]: pgmap v381: 129 pgs: 1 peering, 128 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-09-10T22:02:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:40 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1053540682' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T22:02:41.202 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430270 got 206158430274 for osd.7 2024-09-10T22:02:41.203 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:41.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.790+0000 7f6f3a600640 1 -- 172.21.15.90:0/1724045764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34102c00 msgr2=0x7f6f34102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.790+0000 7f6f3a600640 1 --2- 172.21.15.90:0/1724045764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34102c00 0x7f6f34102fe0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009900 tx=0x7f6f2802a410 comp rx=0 tx=0).stop 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.790+0000 7f6f3a600640 1 -- 172.21.15.90:0/1724045764 shutdown_connections 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.790+0000 7f6f3a600640 1 --2- 172.21.15.90:0/1724045764 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f34108f90 0x7f6f3410f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.790+0000 7f6f3a600640 1 --2- 172.21.15.90:0/1724045764 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f34108610 0x7f6f34108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.790+0000 7f6f3a600640 1 --2- 172.21.15.90:0/1724045764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34102c00 0x7f6f34102fe0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.790+0000 7f6f3a600640 1 -- 172.21.15.90:0/1724045764 >> 172.21.15.90:0/1724045764 conn(0x7f6f340fe6d0 msgr2=0x7f6f34100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.790+0000 7f6f3a600640 1 -- 172.21.15.90:0/1724045764 shutdown_connections 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.790+0000 7f6f3a600640 1 -- 172.21.15.90:0/1724045764 wait complete. 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.791+0000 7f6f3a600640 1 Processor -- start 2024-09-10T22:02:41.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.791+0000 7f6f3a600640 1 -- start start 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f3a600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f34102c00 0x7f6f3419b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f3a600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f34108610 0x7f6f3419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f3a600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34108f90 0x7f6f341a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f3a600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34112b50 con 0x7f6f34108f90 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f3a600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6f341129d0 con 0x7f6f34108610 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f3a600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34112cd0 con 0x7f6f34102c00 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f39c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34108f90 0x7f6f341a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f39c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34108f90 0x7f6f341a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52720/0 (socket says 172.21.15.90:52720) 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f39c00640 1 -- 172.21.15.90:0/971887297 learned_addr learned my addr 172.21.15.90:0/971887297 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f33e00640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f34108610 0x7f6f3419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f39200640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f34102c00 0x7f6f3419b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f39c00640 1 -- 172.21.15.90:0/971887297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f34102c00 msgr2=0x7f6f3419b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:41.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f39c00640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f34102c00 0x7f6f3419b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f39c00640 1 -- 172.21.15.90:0/971887297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f34108610 msgr2=0x7f6f3419b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f39c00640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f34108610 0x7f6f3419b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.792+0000 7f6f39c00640 1 -- 172.21.15.90:0/971887297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f341a11f0 con 0x7f6f34108f90 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.793+0000 7f6f39c00640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34108f90 0x7f6f341a0a20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00eb10 tx=0x7f6f2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.793+0000 7f6f31600640 1 -- 172.21.15.90:0/971887297 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c004b20 con 0x7f6f34108f90 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.793+0000 7f6f31600640 1 -- 172.21.15.90:0/971887297 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f2c004cc0 con 0x7f6f34108f90 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.793+0000 7f6f31600640 1 -- 172.21.15.90:0/971887297 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c010690 con 0x7f6f34108f90 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.793+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f341a14e0 con 0x7f6f34108f90 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.793+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341a7ba0 con 0x7f6f34108f90 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.794+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f34102fe0 con 0x7f6f34108f90 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.798+0000 7f6f31600640 1 -- 172.21.15.90:0/971887297 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f6f2c005000 con 0x7f6f34108f90 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.799+0000 7f6f31600640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f1c0b5460 0x7f6f1c0b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:41.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.800+0000 7f6f31600640 1 -- 172.21.15.90:0/971887297 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f6f2c0e69f0 con 0x7f6f34108f90 2024-09-10T22:02:41.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.800+0000 7f6f39200640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f1c0b5460 0x7f6f1c0b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:41.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.800+0000 7f6f39200640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f1c0b5460 0x7f6f1c0b7920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f28006010 tx=0x7f6f28005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:41.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.801+0000 7f6f31600640 1 -- 172.21.15.90:0/971887297 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6f2c056220 con 0x7f6f34108f90 2024-09-10T22:02:41.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.993+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6f341a7f30 con 0x7f6f34108f90 2024-09-10T22:02:41.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.994+0000 7f6f31600640 1 -- 172.21.15.90:0/971887297 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6f2c0ad260 con 0x7f6f34108f90 2024-09-10T22:02:41.996 INFO:teuthology.orchestra.run.smithi090.stdout:184683593801 2024-09-10T22:02:41.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f1c0b5460 msgr2=0x7f6f1c0b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:41.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f1c0b5460 0x7f6f1c0b7920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f28006010 tx=0x7f6f28005e80 comp rx=0 tx=0).stop 2024-09-10T22:02:41.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34108f90 msgr2=0x7f6f341a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:41.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34108f90 0x7f6f341a0a20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00eb10 tx=0x7f6f2c00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 shutdown_connections 2024-09-10T22:02:41.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f1c0b5460 0x7f6f1c0b7920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f34108f90 0x7f6f341a0a20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f34108610 0x7f6f3419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 --2- 172.21.15.90:0/971887297 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f34102c00 0x7f6f3419b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:41.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 >> 172.21.15.90:0/971887297 conn(0x7f6f340fe6d0 msgr2=0x7f6f340ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:41.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 shutdown_connections 2024-09-10T22:02:41.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:41.996+0000 7f6f3a600640 1 -- 172.21.15.90:0/971887297 wait complete. 2024-09-10T22:02:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:41 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:41 smithi090 ceph-mon[30350]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T22:02:42.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:41 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:41 smithi154 ceph-mon[34495]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T22:02:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:41 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:41 smithi171 ceph-mon[34138]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-10T22:02:43.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:42 smithi090 ceph-mon[30350]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T22:02:43.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:42 smithi090 ceph-mon[30350]: Cluster is now healthy 2024-09-10T22:02:43.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:42 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/971887297' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:02:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:42 smithi154 ceph-mon[34495]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T22:02:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:42 smithi154 ceph-mon[34495]: Cluster is now healthy 2024-09-10T22:02:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:42 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/971887297' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:02:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:42 smithi171 ceph-mon[34138]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T22:02:43.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:42 smithi171 ceph-mon[34138]: Cluster is now healthy 2024-09-10T22:02:43.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:42 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/971887297' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T22:02:43.552 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593796 got 184683593801 for osd.6 2024-09-10T22:02:43.552 DEBUG:teuthology.parallel:result is None 2024-09-10T22:02:43.552 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:02:43.552 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:02:43.955 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:44.069 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:43 smithi154 ceph-mon[34495]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-10T22:02:44.095 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:43 smithi090 ceph-mon[30350]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-10T22:02:44.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:43 smithi171 ceph-mon[34138]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.498+0000 7f6131600640 1 -- 172.21.15.90:0/1534004260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c1087b0 msgr2=0x7f612c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.498+0000 7f6131600640 1 --2- 172.21.15.90:0/1534004260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c1087b0 0x7f612c108bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6114009900 tx=0x7f611402a410 comp rx=0 tx=0).stop 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.498+0000 7f6131600640 1 -- 172.21.15.90:0/1534004260 shutdown_connections 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.498+0000 7f6131600640 1 --2- 172.21.15.90:0/1534004260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f612c109130 0x7f612c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.498+0000 7f6131600640 1 --2- 172.21.15.90:0/1534004260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c1087b0 0x7f612c108bf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.498+0000 7f6131600640 1 --2- 172.21.15.90:0/1534004260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f612c102da0 0x7f612c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.498+0000 7f6131600640 1 -- 172.21.15.90:0/1534004260 >> 172.21.15.90:0/1534004260 conn(0x7f612c0fe810 msgr2=0x7f612c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.498+0000 7f6131600640 1 -- 172.21.15.90:0/1534004260 shutdown_connections 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.498+0000 7f6131600640 1 -- 172.21.15.90:0/1534004260 wait complete. 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.499+0000 7f6131600640 1 Processor -- start 2024-09-10T22:02:44.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.499+0000 7f6131600640 1 -- start start 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.499+0000 7f6131600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f612c102da0 0x7f612c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.499+0000 7f6131600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f612c1087b0 0x7f612c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.499+0000 7f6131600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c109130 0x7f612c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.499+0000 7f6131600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f612c112ca0 con 0x7f612c109130 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.499+0000 7f6131600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f612c112b20 con 0x7f612c102da0 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.499+0000 7f6131600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f612c112e20 con 0x7f612c1087b0 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c109130 0x7f612c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c109130 0x7f612c075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52738/0 (socket says 172.21.15.90:52738) 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612b400640 1 -- 172.21.15.90:0/2253415372 learned_addr learned my addr 172.21.15.90:0/2253415372 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612aa00640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f612c102da0 0x7f612c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f6122000640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f612c1087b0 0x7f612c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612b400640 1 -- 172.21.15.90:0/2253415372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f612c1087b0 msgr2=0x7f612c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612b400640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f612c1087b0 0x7f612c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612b400640 1 -- 172.21.15.90:0/2253415372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f612c102da0 msgr2=0x7f612c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612b400640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f612c102da0 0x7f612c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612b400640 1 -- 172.21.15.90:0/2253415372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612c1a75d0 con 0x7f612c109130 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612aa00640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f612c102da0 0x7f612c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f6122000640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f612c1087b0 0x7f612c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f612b400640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c109130 0x7f612c075f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f612400d750 tx=0x7f612400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.500+0000 7f6123e00640 1 -- 172.21.15.90:0/2253415372 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124017070 con 0x7f612c109130 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.501+0000 7f6123e00640 1 -- 172.21.15.90:0/2253415372 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f61240054b0 con 0x7f612c109130 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.501+0000 7f6123e00640 1 -- 172.21.15.90:0/2253415372 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124004b90 con 0x7f612c109130 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.501+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612c1a78c0 con 0x7f612c109130 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.501+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612c1a7db0 con 0x7f612c109130 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.502+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f612c103180 con 0x7f612c109130 2024-09-10T22:02:44.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.504+0000 7f6123e00640 1 -- 172.21.15.90:0/2253415372 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f6124004d30 con 0x7f612c109130 2024-09-10T22:02:44.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.505+0000 7f6123e00640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f61100b5440 0x7f61100b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:44.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.506+0000 7f6123e00640 1 -- 172.21.15.90:0/2253415372 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f61240e6140 con 0x7f612c109130 2024-09-10T22:02:44.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.506+0000 7f612aa00640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f61100b5440 0x7f61100b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:44.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.506+0000 7f612aa00640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f61100b5440 0x7f61100b7900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f611c005c40 tx=0x7f611c00a430 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:44.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.507+0000 7f6123e00640 1 -- 172.21.15.90:0/2253415372 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f61240aca30 con 0x7f612c109130 2024-09-10T22:02:44.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.708+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f612c063590 con 0x7f61100b5440 2024-09-10T22:02:44.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.714+0000 7f6123e00640 1 -- 172.21.15.90:0/2253415372 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340687 (secure 0 0 0) 0x7f612c063590 con 0x7f61100b5440 2024-09-10T22:02:44.722 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-09-10T22:02:44.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f61100b5440 msgr2=0x7f61100b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f61100b5440 0x7f61100b7900 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f611c005c40 tx=0x7f611c00a430 comp rx=0 tx=0).stop 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c109130 msgr2=0x7f612c075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c109130 0x7f612c075f20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f612400d750 tx=0x7f612400dc10 comp rx=0 tx=0).stop 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 shutdown_connections 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f61100b5440 0x7f61100b7900 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f612c109130 0x7f612c075f20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f612c1087b0 0x7f612c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 --2- 172.21.15.90:0/2253415372 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f612c102da0 0x7f612c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 >> 172.21.15.90:0/2253415372 conn(0x7f612c0fe810 msgr2=0x7f612c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 shutdown_connections 2024-09-10T22:02:44.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:44.720+0000 7f6131600640 1 -- 172.21.15.90:0/2253415372 wait complete. 2024-09-10T22:02:45.445 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-10T22:02:43.614910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661245,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":797,"num_read_kb":502,"num_write":374,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349160,"kb_used_data":12308,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592664,"statfs":{"total":1151940427776,"available":1151582887936,"internally_reserved":0,"allocated":12603392,"data_stored":8381381,"data_compressed":498330,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.901029"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":130,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:41.871904+0000","last_change":"2024-09-10T22:00:30.824265+0000","last_active":"2024-09-10T22:02:41.871904+0000","last_peered":"2024-09-10T22:02:41.871904+0000","last_clean":"2024-09-10T22:02:41.871904+0000","last_became_active":"2024-09-10T22:00:30.823905+0000","last_became_peered":"2024-09-10T22:00:30.823905+0000","last_unstale":"2024-09-10T22:02:41.871904+0000","last_undegraded":"2024-09-10T22:02:41.871904+0000","last_fullsized":"2024-09-10T22:02:41.871904+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:16.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723654+0000","last_change":"2024-09-10T22:00:34.497719+0000","last_active":"2024-09-10T22:02:35.723654+0000","last_peered":"2024-09-10T22:02:35.723654+0000","last_clean":"2024-09-10T22:02:35.723654+0000","last_became_active":"2024-09-10T22:00:34.497380+0000","last_became_peered":"2024-09-10T22:00:34.497380+0000","last_unstale":"2024-09-10T22:02:35.723654+0000","last_undegraded":"2024-09-10T22:02:35.723654+0000","last_fullsized":"2024-09-10T22:02:35.723654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:26.336299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723219+0000","last_change":"2024-09-10T22:00:43.710835+0000","last_active":"2024-09-10T22:02:35.723219+0000","last_peered":"2024-09-10T22:02:35.723219+0000","last_clean":"2024-09-10T22:02:35.723219+0000","last_became_active":"2024-09-10T22:00:43.710363+0000","last_became_peered":"2024-09-10T22:00:43.710363+0000","last_unstale":"2024-09-10T22:02:35.723219+0000","last_undegraded":"2024-09-10T22:02:35.723219+0000","last_fullsized":"2024-09-10T22:02:35.723219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:28.173482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723977+0000","last_change":"2024-09-10T22:00:31.083228+0000","last_active":"2024-09-10T22:02:35.723977+0000","last_peered":"2024-09-10T22:02:35.723977+0000","last_clean":"2024-09-10T22:02:35.723977+0000","last_became_active":"2024-09-10T22:00:31.082911+0000","last_became_peered":"2024-09-10T22:00:31.082911+0000","last_unstale":"2024-09-10T22:02:35.723977+0000","last_undegraded":"2024-09-10T22:02:35.723977+0000","last_fullsized":"2024-09-10T22:02:35.723977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:28.598628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948166+0000","last_change":"2024-09-10T22:00:34.509939+0000","last_active":"2024-09-10T22:02:35.948166+0000","last_peered":"2024-09-10T22:02:35.948166+0000","last_clean":"2024-09-10T22:02:35.948166+0000","last_became_active":"2024-09-10T22:00:34.509536+0000","last_became_peered":"2024-09-10T22:00:34.509536+0000","last_unstale":"2024-09-10T22:02:35.948166+0000","last_undegraded":"2024-09-10T22:02:35.948166+0000","last_fullsized":"2024-09-10T22:02:35.948166+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:27.602378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897046+0000","last_change":"2024-09-10T22:00:43.714662+0000","last_active":"2024-09-10T22:02:35.897046+0000","last_peered":"2024-09-10T22:02:35.897046+0000","last_clean":"2024-09-10T22:02:35.897046+0000","last_became_active":"2024-09-10T22:00:43.714074+0000","last_became_peered":"2024-09-10T22:00:43.714074+0000","last_unstale":"2024-09-10T22:02:35.897046+0000","last_undegraded":"2024-09-10T22:02:35.897046+0000","last_fullsized":"2024-09-10T22:02:35.897046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:41.204519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948479+0000","last_change":"2024-09-10T22:01:35.209979+0000","last_active":"2024-09-10T22:02:35.948479+0000","last_peered":"2024-09-10T22:02:35.948479+0000","last_clean":"2024-09-10T22:02:35.948479+0000","last_became_active":"2024-09-10T22:01:35.197192+0000","last_became_peered":"2024-09-10T22:01:35.197192+0000","last_unstale":"2024-09-10T22:02:35.948479+0000","last_undegraded":"2024-09-10T22:02:35.948479+0000","last_fullsized":"2024-09-10T22:02:35.948479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:47.707679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948620+0000","last_change":"2024-09-10T22:00:34.502795+0000","last_active":"2024-09-10T22:02:35.948620+0000","last_peered":"2024-09-10T22:02:35.948620+0000","last_clean":"2024-09-10T22:02:35.948620+0000","last_became_active":"2024-09-10T22:00:34.501112+0000","last_became_peered":"2024-09-10T22:00:34.501112+0000","last_unstale":"2024-09-10T22:02:35.948620+0000","last_undegraded":"2024-09-10T22:02:35.948620+0000","last_fullsized":"2024-09-10T22:02:35.948620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:45.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897276+0000","last_change":"2024-09-10T22:01:35.619765+0000","last_active":"2024-09-10T22:02:35.897276+0000","last_peered":"2024-09-10T22:02:35.897276+0000","last_clean":"2024-09-10T22:02:35.897276+0000","last_became_active":"2024-09-10T22:01:35.619407+0000","last_became_peered":"2024-09-10T22:01:35.619407+0000","last_unstale":"2024-09-10T22:02:35.897276+0000","last_undegraded":"2024-09-10T22:02:35.897276+0000","last_fullsized":"2024-09-10T22:02:35.897276+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:17.064585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726605+0000","last_change":"2024-09-10T22:00:30.818224+0000","last_active":"2024-09-10T22:02:35.726605+0000","last_peered":"2024-09-10T22:02:35.726605+0000","last_clean":"2024-09-10T22:02:35.726605+0000","last_became_active":"2024-09-10T22:00:30.817781+0000","last_became_peered":"2024-09-10T22:00:30.817781+0000","last_unstale":"2024-09-10T22:02:35.726605+0000","last_undegraded":"2024-09-10T22:02:35.726605+0000","last_fullsized":"2024-09-10T22:02:35.726605+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:47.097642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.950484+0000","last_change":"2024-09-10T22:02:35.949974+0000","last_active":"2024-09-10T22:02:35.950484+0000","last_peered":"2024-09-10T22:02:35.950484+0000","last_clean":"2024-09-10T22:02:35.950484+0000","last_became_active":"2024-09-10T22:02:35.949618+0000","last_became_peered":"2024-09-10T22:02:35.949618+0000","last_unstale":"2024-09-10T22:02:35.950484+0000","last_undegraded":"2024-09-10T22:02:35.950484+0000","last_fullsized":"2024-09-10T22:02:35.950484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:02.877650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897232+0000","last_change":"2024-09-10T22:00:43.712066+0000","last_active":"2024-09-10T22:02:35.897232+0000","last_peered":"2024-09-10T22:02:35.897232+0000","last_clean":"2024-09-10T22:02:35.897232+0000","last_became_active":"2024-09-10T22:00:43.711614+0000","last_became_peered":"2024-09-10T22:00:43.711614+0000","last_unstale":"2024-09-10T22:02:35.897232+0000","last_undegraded":"2024-09-10T22:02:35.897232+0000","last_fullsized":"2024-09-10T22:02:35.897232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:42.403367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863184+0000","last_change":"2024-09-10T22:00:34.501943+0000","last_active":"2024-09-10T22:02:35.863184+0000","last_peered":"2024-09-10T22:02:35.863184+0000","last_clean":"2024-09-10T22:02:35.863184+0000","last_became_active":"2024-09-10T22:00:34.501602+0000","last_became_peered":"2024-09-10T22:00:34.501602+0000","last_unstale":"2024-09-10T22:02:35.863184+0000","last_undegraded":"2024-09-10T22:02:35.863184+0000","last_fullsized":"2024-09-10T22:02:35.863184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:53.200974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726534+0000","last_change":"2024-09-10T22:01:35.200791+0000","last_active":"2024-09-10T22:02:35.726534+0000","last_peered":"2024-09-10T22:02:35.726534+0000","last_clean":"2024-09-10T22:02:35.726534+0000","last_became_active":"2024-09-10T22:01:35.196059+0000","last_became_peered":"2024-09-10T22:01:35.196059+0000","last_unstale":"2024-09-10T22:02:35.726534+0000","last_undegraded":"2024-09-10T22:02:35.726534+0000","last_fullsized":"2024-09-10T22:02:35.726534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:51.349216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.896947+0000","last_change":"2024-09-10T22:00:43.760567+0000","last_active":"2024-09-10T22:02:35.896947+0000","last_peered":"2024-09-10T22:02:35.896947+0000","last_clean":"2024-09-10T22:02:35.896947+0000","last_became_active":"2024-09-10T22:00:43.760064+0000","last_became_peered":"2024-09-10T22:00:43.760064+0000","last_unstale":"2024-09-10T22:02:35.896947+0000","last_undegraded":"2024-09-10T22:02:35.896947+0000","last_fullsized":"2024-09-10T22:02:35.896947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:55.469525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722689+0000","last_change":"2024-09-10T22:00:34.486225+0000","last_active":"2024-09-10T22:02:35.722689+0000","last_peered":"2024-09-10T22:02:35.722689+0000","last_clean":"2024-09-10T22:02:35.722689+0000","last_became_active":"2024-09-10T22:00:34.485896+0000","last_became_peered":"2024-09-10T22:00:34.485896+0000","last_unstale":"2024-09-10T22:02:35.722689+0000","last_undegraded":"2024-09-10T22:02:35.722689+0000","last_fullsized":"2024-09-10T22:02:35.722689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:00.151927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723589+0000","last_change":"2024-09-10T22:00:30.817958+0000","last_active":"2024-09-10T22:02:35.723589+0000","last_peered":"2024-09-10T22:02:35.723589+0000","last_clean":"2024-09-10T22:02:35.723589+0000","last_became_active":"2024-09-10T22:00:30.817646+0000","last_became_peered":"2024-09-10T22:00:30.817646+0000","last_unstale":"2024-09-10T22:02:35.723589+0000","last_undegraded":"2024-09-10T22:02:35.723589+0000","last_fullsized":"2024-09-10T22:02:35.723589+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:27.722184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897603+0000","last_change":"2024-09-10T22:00:43.716465+0000","last_active":"2024-09-10T22:02:35.897603+0000","last_peered":"2024-09-10T22:02:35.897603+0000","last_clean":"2024-09-10T22:02:35.897603+0000","last_became_active":"2024-09-10T22:00:43.716165+0000","last_became_peered":"2024-09-10T22:00:43.716165+0000","last_unstale":"2024-09-10T22:02:35.897603+0000","last_undegraded":"2024-09-10T22:02:35.897603+0000","last_fullsized":"2024-09-10T22:02:35.897603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:18.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948499+0000","last_change":"2024-09-10T22:00:34.493992+0000","last_active":"2024-09-10T22:02:35.948499+0000","last_peered":"2024-09-10T22:02:35.948499+0000","last_clean":"2024-09-10T22:02:35.948499+0000","last_became_active":"2024-09-10T22:00:34.493670+0000","last_became_peered":"2024-09-10T22:00:34.493670+0000","last_unstale":"2024-09-10T22:02:35.948499+0000","last_undegraded":"2024-09-10T22:02:35.948499+0000","last_fullsized":"2024-09-10T22:02:35.948499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:04.991819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722602+0000","last_change":"2024-09-10T22:00:30.819830+0000","last_active":"2024-09-10T22:02:35.722602+0000","last_peered":"2024-09-10T22:02:35.722602+0000","last_clean":"2024-09-10T22:02:35.722602+0000","last_became_active":"2024-09-10T22:00:30.819401+0000","last_became_peered":"2024-09-10T22:00:30.819401+0000","last_unstale":"2024-09-10T22:02:35.722602+0000","last_undegraded":"2024-09-10T22:02:35.722602+0000","last_fullsized":"2024-09-10T22:02:35.722602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:37.823107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724219+0000","last_change":"2024-09-10T22:00:43.718500+0000","last_active":"2024-09-10T22:02:35.724219+0000","last_peered":"2024-09-10T22:02:35.724219+0000","last_clean":"2024-09-10T22:02:35.724219+0000","last_became_active":"2024-09-10T22:00:43.717667+0000","last_became_peered":"2024-09-10T22:00:43.717667+0000","last_unstale":"2024-09-10T22:02:35.724219+0000","last_undegraded":"2024-09-10T22:02:35.724219+0000","last_fullsized":"2024-09-10T22:02:35.724219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:23.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.725822+0000","last_change":"2024-09-10T22:00:34.494162+0000","last_active":"2024-09-10T22:02:35.725822+0000","last_peered":"2024-09-10T22:02:35.725822+0000","last_clean":"2024-09-10T22:02:35.725822+0000","last_became_active":"2024-09-10T22:00:34.493819+0000","last_became_peered":"2024-09-10T22:00:34.493819+0000","last_unstale":"2024-09-10T22:02:35.725822+0000","last_undegraded":"2024-09-10T22:02:35.725822+0000","last_fullsized":"2024-09-10T22:02:35.725822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:43.264363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948452+0000","last_change":"2024-09-10T22:00:31.081559+0000","last_active":"2024-09-10T22:02:35.948452+0000","last_peered":"2024-09-10T22:02:35.948452+0000","last_clean":"2024-09-10T22:02:35.948452+0000","last_became_active":"2024-09-10T22:00:31.081257+0000","last_became_peered":"2024-09-10T22:00:31.081257+0000","last_unstale":"2024-09-10T22:02:35.948452+0000","last_undegraded":"2024-09-10T22:02:35.948452+0000","last_fullsized":"2024-09-10T22:02:35.948452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:53.235765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897631+0000","last_change":"2024-09-10T22:01:35.445171+0000","last_active":"2024-09-10T22:02:35.897631+0000","last_peered":"2024-09-10T22:02:35.897631+0000","last_clean":"2024-09-10T22:02:35.897631+0000","last_became_active":"2024-09-10T22:01:35.444651+0000","last_became_peered":"2024-09-10T22:01:35.444651+0000","last_unstale":"2024-09-10T22:02:35.897631+0000","last_undegraded":"2024-09-10T22:02:35.897631+0000","last_fullsized":"2024-09-10T22:02:35.897631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:36.457151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948017+0000","last_change":"2024-09-10T22:01:35.750982+0000","last_active":"2024-09-10T22:02:35.948017+0000","last_peered":"2024-09-10T22:02:35.948017+0000","last_clean":"2024-09-10T22:02:35.948017+0000","last_became_active":"2024-09-10T22:01:35.750572+0000","last_became_peered":"2024-09-10T22:01:35.750572+0000","last_unstale":"2024-09-10T22:02:35.948017+0000","last_undegraded":"2024-09-10T22:02:35.948017+0000","last_fullsized":"2024-09-10T22:02:35.948017+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:36.596313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897938+0000","last_change":"2024-09-10T22:00:31.311864+0000","last_active":"2024-09-10T22:02:35.897938+0000","last_peered":"2024-09-10T22:02:35.897938+0000","last_clean":"2024-09-10T22:02:35.897938+0000","last_became_active":"2024-09-10T22:00:31.311611+0000","last_became_peered":"2024-09-10T22:00:31.311611+0000","last_unstale":"2024-09-10T22:02:35.897938+0000","last_undegraded":"2024-09-10T22:02:35.897938+0000","last_fullsized":"2024-09-10T22:02:35.897938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:27.969999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723426+0000","last_change":"2024-09-10T22:00:43.708023+0000","last_active":"2024-09-10T22:02:35.723426+0000","last_peered":"2024-09-10T22:02:35.723426+0000","last_clean":"2024-09-10T22:02:35.723426+0000","last_became_active":"2024-09-10T22:00:43.707086+0000","last_became_peered":"2024-09-10T22:00:43.707086+0000","last_unstale":"2024-09-10T22:02:35.723426+0000","last_undegraded":"2024-09-10T22:02:35.723426+0000","last_fullsized":"2024-09-10T22:02:35.723426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:15.585104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724003+0000","last_change":"2024-09-10T22:00:34.494104+0000","last_active":"2024-09-10T22:02:35.724003+0000","last_peered":"2024-09-10T22:02:35.724003+0000","last_clean":"2024-09-10T22:02:35.724003+0000","last_became_active":"2024-09-10T22:00:34.493404+0000","last_became_peered":"2024-09-10T22:00:34.493404+0000","last_unstale":"2024-09-10T22:02:35.724003+0000","last_undegraded":"2024-09-10T22:02:35.724003+0000","last_fullsized":"2024-09-10T22:02:35.724003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:47.444100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'14","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723513+0000","last_change":"2024-09-10T22:00:30.819427+0000","last_active":"2024-09-10T22:02:35.723513+0000","last_peered":"2024-09-10T22:02:35.723513+0000","last_clean":"2024-09-10T22:02:35.723513+0000","last_became_active":"2024-09-10T22:00:30.819098+0000","last_became_peered":"2024-09-10T22:00:30.819098+0000","last_unstale":"2024-09-10T22:02:35.723513+0000","last_undegraded":"2024-09-10T22:02:35.723513+0000","last_fullsized":"2024-09-10T22:02:35.723513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:23.068155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722254+0000","last_change":"2024-09-10T22:00:43.760602+0000","last_active":"2024-09-10T22:02:35.722254+0000","last_peered":"2024-09-10T22:02:35.722254+0000","last_clean":"2024-09-10T22:02:35.722254+0000","last_became_active":"2024-09-10T22:00:43.760172+0000","last_became_peered":"2024-09-10T22:00:43.760172+0000","last_unstale":"2024-09-10T22:02:35.722254+0000","last_undegraded":"2024-09-10T22:02:35.722254+0000","last_fullsized":"2024-09-10T22:02:35.722254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:40.332101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894374+0000","last_change":"2024-09-10T22:00:34.501632+0000","last_active":"2024-09-10T22:02:35.894374+0000","last_peered":"2024-09-10T22:02:35.894374+0000","last_clean":"2024-09-10T22:02:35.894374+0000","last_became_active":"2024-09-10T22:00:34.500924+0000","last_became_peered":"2024-09-10T22:00:34.500924+0000","last_unstale":"2024-09-10T22:02:35.894374+0000","last_undegraded":"2024-09-10T22:02:35.894374+0000","last_fullsized":"2024-09-10T22:02:35.894374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:28.052570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726206+0000","last_change":"2024-09-10T22:00:31.081391+0000","last_active":"2024-09-10T22:02:35.726206+0000","last_peered":"2024-09-10T22:02:35.726206+0000","last_clean":"2024-09-10T22:02:35.726206+0000","last_became_active":"2024-09-10T22:00:31.081018+0000","last_became_peered":"2024-09-10T22:00:31.081018+0000","last_unstale":"2024-09-10T22:02:35.726206+0000","last_undegraded":"2024-09-10T22:02:35.726206+0000","last_fullsized":"2024-09-10T22:02:35.726206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:09.450785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948407+0000","last_change":"2024-09-10T22:00:43.711592+0000","last_active":"2024-09-10T22:02:35.948407+0000","last_peered":"2024-09-10T22:02:35.948407+0000","last_clean":"2024-09-10T22:02:35.948407+0000","last_became_active":"2024-09-10T22:00:43.711144+0000","last_became_peered":"2024-09-10T22:00:43.711144+0000","last_unstale":"2024-09-10T22:02:35.948407+0000","last_undegraded":"2024-09-10T22:02:35.948407+0000","last_fullsized":"2024-09-10T22:02:35.948407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:01.818907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871767+0000","last_change":"2024-09-10T22:00:34.494070+0000","last_active":"2024-09-10T22:02:35.871767+0000","last_peered":"2024-09-10T22:02:35.871767+0000","last_clean":"2024-09-10T22:02:35.871767+0000","last_became_active":"2024-09-10T22:00:34.493324+0000","last_became_peered":"2024-09-10T22:00:34.493324+0000","last_unstale":"2024-09-10T22:02:35.871767+0000","last_undegraded":"2024-09-10T22:02:35.871767+0000","last_fullsized":"2024-09-10T22:02:35.871767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:29.969743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723888+0000","last_change":"2024-09-10T22:00:31.082439+0000","last_active":"2024-09-10T22:02:35.723888+0000","last_peered":"2024-09-10T22:02:35.723888+0000","last_clean":"2024-09-10T22:02:35.723888+0000","last_became_active":"2024-09-10T22:00:31.082025+0000","last_became_peered":"2024-09-10T22:00:31.082025+0000","last_unstale":"2024-09-10T22:02:35.723888+0000","last_undegraded":"2024-09-10T22:02:35.723888+0000","last_fullsized":"2024-09-10T22:02:35.723888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:21.933419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726032+0000","last_change":"2024-09-10T22:00:43.760997+0000","last_active":"2024-09-10T22:02:35.726032+0000","last_peered":"2024-09-10T22:02:35.726032+0000","last_clean":"2024-09-10T22:02:35.726032+0000","last_became_active":"2024-09-10T22:00:43.760124+0000","last_became_peered":"2024-09-10T22:00:43.760124+0000","last_unstale":"2024-09-10T22:02:35.726032+0000","last_undegraded":"2024-09-10T22:02:35.726032+0000","last_fullsized":"2024-09-10T22:02:35.726032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:29.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871775+0000","last_change":"2024-09-10T22:00:34.511730+0000","last_active":"2024-09-10T22:02:35.871775+0000","last_peered":"2024-09-10T22:02:35.871775+0000","last_clean":"2024-09-10T22:02:35.871775+0000","last_became_active":"2024-09-10T22:00:34.511440+0000","last_became_peered":"2024-09-10T22:00:34.511440+0000","last_unstale":"2024-09-10T22:02:35.871775+0000","last_undegraded":"2024-09-10T22:02:35.871775+0000","last_fullsized":"2024-09-10T22:02:35.871775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:09.053835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:38.868379+0000","last_change":"2024-09-10T22:00:30.825999+0000","last_active":"2024-09-10T22:02:38.868379+0000","last_peered":"2024-09-10T22:02:38.868379+0000","last_clean":"2024-09-10T22:02:38.868379+0000","last_became_active":"2024-09-10T22:00:30.825334+0000","last_became_peered":"2024-09-10T22:00:30.825334+0000","last_unstale":"2024-09-10T22:02:38.868379+0000","last_undegraded":"2024-09-10T22:02:38.868379+0000","last_fullsized":"2024-09-10T22:02:38.868379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:14.684222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894160+0000","last_change":"2024-09-10T22:00:43.710517+0000","last_active":"2024-09-10T22:02:35.894160+0000","last_peered":"2024-09-10T22:02:35.894160+0000","last_clean":"2024-09-10T22:02:35.894160+0000","last_became_active":"2024-09-10T22:00:43.709252+0000","last_became_peered":"2024-09-10T22:00:43.709252+0000","last_unstale":"2024-09-10T22:02:35.894160+0000","last_undegraded":"2024-09-10T22:02:35.894160+0000","last_fullsized":"2024-09-10T22:02:35.894160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:21.355152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.899302+0000","last_change":"2024-09-10T22:02:35.898878+0000","last_active":"2024-09-10T22:02:35.899302+0000","last_peered":"2024-09-10T22:02:35.899302+0000","last_clean":"2024-09-10T22:02:35.899302+0000","last_became_active":"2024-09-10T22:02:35.898531+0000","last_became_peered":"2024-09-10T22:02:35.898531+0000","last_unstale":"2024-09-10T22:02:35.899302+0000","last_undegraded":"2024-09-10T22:02:35.899302+0000","last_fullsized":"2024-09-10T22:02:35.899302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:02.735208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":116,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:38.902964+0000","last_change":"2024-09-10T22:00:31.081590+0000","last_active":"2024-09-10T22:02:38.902964+0000","last_peered":"2024-09-10T22:02:38.902964+0000","last_clean":"2024-09-10T22:02:38.902964+0000","last_became_active":"2024-09-10T22:00:31.081250+0000","last_became_peered":"2024-09-10T22:00:31.081250+0000","last_unstale":"2024-09-10T22:02:38.902964+0000","last_undegraded":"2024-09-10T22:02:38.902964+0000","last_fullsized":"2024-09-10T22:02:38.902964+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:12.872832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893728+0000","last_change":"2024-09-10T22:00:43.761351+0000","last_active":"2024-09-10T22:02:35.893728+0000","last_peered":"2024-09-10T22:02:35.893728+0000","last_clean":"2024-09-10T22:02:35.893728+0000","last_became_active":"2024-09-10T22:00:43.760903+0000","last_became_peered":"2024-09-10T22:00:43.760903+0000","last_unstale":"2024-09-10T22:02:35.893728+0000","last_undegraded":"2024-09-10T22:02:35.893728+0000","last_fullsized":"2024-09-10T22:02:35.893728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:14.113974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893929+0000","last_change":"2024-09-10T22:00:34.504671+0000","last_active":"2024-09-10T22:02:35.893929+0000","last_peered":"2024-09-10T22:02:35.893929+0000","last_clean":"2024-09-10T22:02:35.893929+0000","last_became_active":"2024-09-10T22:00:34.504368+0000","last_became_peered":"2024-09-10T22:00:34.504368+0000","last_unstale":"2024-09-10T22:02:35.893929+0000","last_undegraded":"2024-09-10T22:02:35.893929+0000","last_fullsized":"2024-09-10T22:02:35.893929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:59.771589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863056+0000","last_change":"2024-09-10T22:00:30.820401+0000","last_active":"2024-09-10T22:02:35.863056+0000","last_peered":"2024-09-10T22:02:35.863056+0000","last_clean":"2024-09-10T22:02:35.863056+0000","last_became_active":"2024-09-10T22:00:30.820194+0000","last_became_peered":"2024-09-10T22:00:30.820194+0000","last_unstale":"2024-09-10T22:02:35.863056+0000","last_undegraded":"2024-09-10T22:02:35.863056+0000","last_fullsized":"2024-09-10T22:02:35.863056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:47.767102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727182+0000","last_change":"2024-09-10T22:00:43.762001+0000","last_active":"2024-09-10T22:02:35.727182+0000","last_peered":"2024-09-10T22:02:35.727182+0000","last_clean":"2024-09-10T22:02:35.727182+0000","last_became_active":"2024-09-10T22:00:43.761323+0000","last_became_peered":"2024-09-10T22:00:43.761323+0000","last_unstale":"2024-09-10T22:02:35.727182+0000","last_undegraded":"2024-09-10T22:02:35.727182+0000","last_fullsized":"2024-09-10T22:02:35.727182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:50.113545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722782+0000","last_change":"2024-09-10T22:00:34.498591+0000","last_active":"2024-09-10T22:02:35.722782+0000","last_peered":"2024-09-10T22:02:35.722782+0000","last_clean":"2024-09-10T22:02:35.722782+0000","last_became_active":"2024-09-10T22:00:34.496449+0000","last_became_peered":"2024-09-10T22:00:34.496449+0000","last_unstale":"2024-09-10T22:02:35.722782+0000","last_undegraded":"2024-09-10T22:02:35.722782+0000","last_fullsized":"2024-09-10T22:02:35.722782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:54.815268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897905+0000","last_change":"2024-09-10T22:00:30.817201+0000","last_active":"2024-09-10T22:02:35.897905+0000","last_peered":"2024-09-10T22:02:35.897905+0000","last_clean":"2024-09-10T22:02:35.897905+0000","last_became_active":"2024-09-10T22:00:30.816792+0000","last_became_peered":"2024-09-10T22:00:30.816792+0000","last_unstale":"2024-09-10T22:02:35.897905+0000","last_undegraded":"2024-09-10T22:02:35.897905+0000","last_fullsized":"2024-09-10T22:02:35.897905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:48.239811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863107+0000","last_change":"2024-09-10T21:59:01.830452+0000","last_active":"2024-09-10T22:02:35.863107+0000","last_peered":"2024-09-10T22:02:35.863107+0000","last_clean":"2024-09-10T22:02:35.863107+0000","last_became_active":"2024-09-10T21:59:01.823744+0000","last_became_peered":"2024-09-10T21:59:01.823744+0000","last_unstale":"2024-09-10T22:02:35.863107+0000","last_undegraded":"2024-09-10T22:02:35.863107+0000","last_fullsized":"2024-09-10T22:02:35.863107+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_clean_scrub_stamp":"2024-09-10T21:54:48.621299+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:39.573257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863503+0000","last_change":"2024-09-10T22:00:43.713888+0000","last_active":"2024-09-10T22:02:35.863503+0000","last_peered":"2024-09-10T22:02:35.863503+0000","last_clean":"2024-09-10T22:02:35.863503+0000","last_became_active":"2024-09-10T22:00:43.713329+0000","last_became_peered":"2024-09-10T22:00:43.713329+0000","last_unstale":"2024-09-10T22:02:35.863503+0000","last_undegraded":"2024-09-10T22:02:35.863503+0000","last_fullsized":"2024-09-10T22:02:35.863503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:01.062089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863656+0000","last_change":"2024-09-10T22:00:34.502589+0000","last_active":"2024-09-10T22:02:35.863656+0000","last_peered":"2024-09-10T22:02:35.863656+0000","last_clean":"2024-09-10T22:02:35.863656+0000","last_became_active":"2024-09-10T22:00:34.502216+0000","last_became_peered":"2024-09-10T22:00:34.502216+0000","last_unstale":"2024-09-10T22:02:35.863656+0000","last_undegraded":"2024-09-10T22:02:35.863656+0000","last_fullsized":"2024-09-10T22:02:35.863656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:44.959375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":118,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:36.874645+0000","last_change":"2024-09-10T22:00:31.311807+0000","last_active":"2024-09-10T22:02:36.874645+0000","last_peered":"2024-09-10T22:02:36.874645+0000","last_clean":"2024-09-10T22:02:36.874645+0000","last_became_active":"2024-09-10T22:00:31.311424+0000","last_became_peered":"2024-09-10T22:00:31.311424+0000","last_unstale":"2024-09-10T22:02:36.874645+0000","last_undegraded":"2024-09-10T22:02:36.874645+0000","last_fullsized":"2024-09-10T22:02:36.874645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:09.754777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948128+0000","last_change":"2024-09-10T22:00:43.711882+0000","last_active":"2024-09-10T22:02:35.948128+0000","last_peered":"2024-09-10T22:02:35.948128+0000","last_clean":"2024-09-10T22:02:35.948128+0000","last_became_active":"2024-09-10T22:00:43.711408+0000","last_became_peered":"2024-09-10T22:00:43.711408+0000","last_unstale":"2024-09-10T22:02:35.948128+0000","last_undegraded":"2024-09-10T22:02:35.948128+0000","last_fullsized":"2024-09-10T22:02:35.948128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:31.155723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863336+0000","last_change":"2024-09-10T22:00:34.505653+0000","last_active":"2024-09-10T22:02:35.863336+0000","last_peered":"2024-09-10T22:02:35.863336+0000","last_clean":"2024-09-10T22:02:35.863336+0000","last_became_active":"2024-09-10T22:00:34.505317+0000","last_became_peered":"2024-09-10T22:00:34.505317+0000","last_unstale":"2024-09-10T22:02:35.863336+0000","last_undegraded":"2024-09-10T22:02:35.863336+0000","last_fullsized":"2024-09-10T22:02:35.863336+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:35.050892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727910+0000","last_change":"2024-09-10T22:00:30.817431+0000","last_active":"2024-09-10T22:02:35.727910+0000","last_peered":"2024-09-10T22:02:35.727910+0000","last_clean":"2024-09-10T22:02:35.727910+0000","last_became_active":"2024-09-10T22:00:30.816952+0000","last_became_peered":"2024-09-10T22:00:30.816952+0000","last_unstale":"2024-09-10T22:02:35.727910+0000","last_undegraded":"2024-09-10T22:02:35.727910+0000","last_fullsized":"2024-09-10T22:02:35.727910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:37.881641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947445+0000","last_change":"2024-09-10T22:00:43.720627+0000","last_active":"2024-09-10T22:02:35.947445+0000","last_peered":"2024-09-10T22:02:35.947445+0000","last_clean":"2024-09-10T22:02:35.947445+0000","last_became_active":"2024-09-10T22:00:43.720235+0000","last_became_peered":"2024-09-10T22:00:43.720235+0000","last_unstale":"2024-09-10T22:02:35.947445+0000","last_undegraded":"2024-09-10T22:02:35.947445+0000","last_fullsized":"2024-09-10T22:02:35.947445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:50.549536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863837+0000","last_change":"2024-09-10T22:00:34.501649+0000","last_active":"2024-09-10T22:02:35.863837+0000","last_peered":"2024-09-10T22:02:35.863837+0000","last_clean":"2024-09-10T22:02:35.863837+0000","last_became_active":"2024-09-10T22:00:34.498882+0000","last_became_peered":"2024-09-10T22:00:34.498882+0000","last_unstale":"2024-09-10T22:02:35.863837+0000","last_undegraded":"2024-09-10T22:02:35.863837+0000","last_fullsized":"2024-09-10T22:02:35.863837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894461+0000","last_change":"2024-09-10T22:00:30.821943+0000","last_active":"2024-09-10T22:02:35.894461+0000","last_peered":"2024-09-10T22:02:35.894461+0000","last_clean":"2024-09-10T22:02:35.894461+0000","last_became_active":"2024-09-10T22:00:30.821372+0000","last_became_peered":"2024-09-10T22:00:30.821372+0000","last_unstale":"2024-09-10T22:02:35.894461+0000","last_undegraded":"2024-09-10T22:02:35.894461+0000","last_fullsized":"2024-09-10T22:02:35.894461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:04.360383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726948+0000","last_change":"2024-09-10T22:00:43.710045+0000","last_active":"2024-09-10T22:02:35.726948+0000","last_peered":"2024-09-10T22:02:35.726948+0000","last_clean":"2024-09-10T22:02:35.726948+0000","last_became_active":"2024-09-10T22:00:43.709416+0000","last_became_peered":"2024-09-10T22:00:43.709416+0000","last_unstale":"2024-09-10T22:02:35.726948+0000","last_undegraded":"2024-09-10T22:02:35.726948+0000","last_fullsized":"2024-09-10T22:02:35.726948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:56.005549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948742+0000","last_change":"2024-09-10T22:00:34.496163+0000","last_active":"2024-09-10T22:02:35.948742+0000","last_peered":"2024-09-10T22:02:35.948742+0000","last_clean":"2024-09-10T22:02:35.948742+0000","last_became_active":"2024-09-10T22:00:34.495814+0000","last_became_peered":"2024-09-10T22:00:34.495814+0000","last_unstale":"2024-09-10T22:02:35.948742+0000","last_undegraded":"2024-09-10T22:02:35.948742+0000","last_fullsized":"2024-09-10T22:02:35.948742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:05.398154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726371+0000","last_change":"2024-09-10T22:00:30.812702+0000","last_active":"2024-09-10T22:02:35.726371+0000","last_peered":"2024-09-10T22:02:35.726371+0000","last_clean":"2024-09-10T22:02:35.726371+0000","last_became_active":"2024-09-10T22:00:30.812369+0000","last_became_peered":"2024-09-10T22:00:30.812369+0000","last_unstale":"2024-09-10T22:02:35.726371+0000","last_undegraded":"2024-09-10T22:02:35.726371+0000","last_fullsized":"2024-09-10T22:02:35.726371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:52.332876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727160+0000","last_change":"2024-09-10T22:00:43.715573+0000","last_active":"2024-09-10T22:02:35.727160+0000","last_peered":"2024-09-10T22:02:35.727160+0000","last_clean":"2024-09-10T22:02:35.727160+0000","last_became_active":"2024-09-10T22:00:43.714475+0000","last_became_peered":"2024-09-10T22:00:43.714475+0000","last_unstale":"2024-09-10T22:02:35.727160+0000","last_undegraded":"2024-09-10T22:02:35.727160+0000","last_fullsized":"2024-09-10T22:02:35.727160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:55.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.949841+0000","last_change":"2024-09-10T22:02:35.949769+0000","last_active":"2024-09-10T22:02:35.949841+0000","last_peered":"2024-09-10T22:02:35.949841+0000","last_clean":"2024-09-10T22:02:35.949841+0000","last_became_active":"2024-09-10T22:02:35.949440+0000","last_became_peered":"2024-09-10T22:02:35.949440+0000","last_unstale":"2024-09-10T22:02:35.949841+0000","last_undegraded":"2024-09-10T22:02:35.949841+0000","last_fullsized":"2024-09-10T22:02:35.949841+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:03.639412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948309+0000","last_change":"2024-09-10T22:01:35.903381+0000","last_active":"2024-09-10T22:02:35.948309+0000","last_peered":"2024-09-10T22:02:35.948309+0000","last_clean":"2024-09-10T22:02:35.948309+0000","last_became_active":"2024-09-10T22:01:35.878262+0000","last_became_peered":"2024-09-10T22:01:35.878262+0000","last_unstale":"2024-09-10T22:02:35.948309+0000","last_undegraded":"2024-09-10T22:02:35.948309+0000","last_fullsized":"2024-09-10T22:02:35.948309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:45.419449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893719+0000","last_change":"2024-09-10T22:00:43.720728+0000","last_active":"2024-09-10T22:02:35.893719+0000","last_peered":"2024-09-10T22:02:35.893719+0000","last_clean":"2024-09-10T22:02:35.893719+0000","last_became_active":"2024-09-10T22:00:43.720305+0000","last_became_peered":"2024-09-10T22:00:43.720305+0000","last_unstale":"2024-09-10T22:02:35.893719+0000","last_undegraded":"2024-09-10T22:02:35.893719+0000","last_fullsized":"2024-09-10T22:02:35.893719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:54.986822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.862895+0000","last_change":"2024-09-10T22:00:34.501571+0000","last_active":"2024-09-10T22:02:35.862895+0000","last_peered":"2024-09-10T22:02:35.862895+0000","last_clean":"2024-09-10T22:02:35.862895+0000","last_became_active":"2024-09-10T22:00:34.501211+0000","last_became_peered":"2024-09-10T22:00:34.501211+0000","last_unstale":"2024-09-10T22:02:35.862895+0000","last_undegraded":"2024-09-10T22:02:35.862895+0000","last_fullsized":"2024-09-10T22:02:35.862895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:05.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":124,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:40.906640+0000","last_change":"2024-09-10T22:00:30.824296+0000","last_active":"2024-09-10T22:02:40.906640+0000","last_peered":"2024-09-10T22:02:40.906640+0000","last_clean":"2024-09-10T22:02:40.906640+0000","last_became_active":"2024-09-10T22:00:30.823886+0000","last_became_peered":"2024-09-10T22:00:30.823886+0000","last_unstale":"2024-09-10T22:02:40.906640+0000","last_undegraded":"2024-09-10T22:02:40.906640+0000","last_fullsized":"2024-09-10T22:02:40.906640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:26.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724313+0000","last_change":"2024-09-10T22:00:43.717580+0000","last_active":"2024-09-10T22:02:35.724313+0000","last_peered":"2024-09-10T22:02:35.724313+0000","last_clean":"2024-09-10T22:02:35.724313+0000","last_became_active":"2024-09-10T22:00:43.708377+0000","last_became_peered":"2024-09-10T22:00:43.708377+0000","last_unstale":"2024-09-10T22:02:35.724313+0000","last_undegraded":"2024-09-10T22:02:35.724313+0000","last_fullsized":"2024-09-10T22:02:35.724313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:54.682292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863307+0000","last_change":"2024-09-10T22:00:34.514348+0000","last_active":"2024-09-10T22:02:35.863307+0000","last_peered":"2024-09-10T22:02:35.863307+0000","last_clean":"2024-09-10T22:02:35.863307+0000","last_became_active":"2024-09-10T22:00:34.513797+0000","last_became_peered":"2024-09-10T22:00:34.513797+0000","last_unstale":"2024-09-10T22:02:35.863307+0000","last_undegraded":"2024-09-10T22:02:35.863307+0000","last_fullsized":"2024-09-10T22:02:35.863307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:12.797643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728017+0000","last_change":"2024-09-10T22:00:30.826486+0000","last_active":"2024-09-10T22:02:35.728017+0000","last_peered":"2024-09-10T22:02:35.728017+0000","last_clean":"2024-09-10T22:02:35.728017+0000","last_became_active":"2024-09-10T22:00:30.826111+0000","last_became_peered":"2024-09-10T22:00:30.826111+0000","last_unstale":"2024-09-10T22:02:35.728017+0000","last_undegraded":"2024-09-10T22:02:35.728017+0000","last_fullsized":"2024-09-10T22:02:35.728017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:45.609925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948404+0000","last_change":"2024-09-10T22:00:43.709432+0000","last_active":"2024-09-10T22:02:35.948404+0000","last_peered":"2024-09-10T22:02:35.948404+0000","last_clean":"2024-09-10T22:02:35.948404+0000","last_became_active":"2024-09-10T22:00:43.708627+0000","last_became_peered":"2024-09-10T22:00:43.708627+0000","last_unstale":"2024-09-10T22:02:35.948404+0000","last_undegraded":"2024-09-10T22:02:35.948404+0000","last_fullsized":"2024-09-10T22:02:35.948404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:12.799976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894017+0000","last_change":"2024-09-10T22:00:34.499203+0000","last_active":"2024-09-10T22:02:35.894017+0000","last_peered":"2024-09-10T22:02:35.894017+0000","last_clean":"2024-09-10T22:02:35.894017+0000","last_became_active":"2024-09-10T22:00:34.497877+0000","last_became_peered":"2024-09-10T22:00:34.497877+0000","last_unstale":"2024-09-10T22:02:35.894017+0000","last_undegraded":"2024-09-10T22:02:35.894017+0000","last_fullsized":"2024-09-10T22:02:35.894017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:41.897603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723917+0000","last_change":"2024-09-10T22:00:30.820852+0000","last_active":"2024-09-10T22:02:35.723917+0000","last_peered":"2024-09-10T22:02:35.723917+0000","last_clean":"2024-09-10T22:02:35.723917+0000","last_became_active":"2024-09-10T22:00:30.820023+0000","last_became_peered":"2024-09-10T22:00:30.820023+0000","last_unstale":"2024-09-10T22:02:35.723917+0000","last_undegraded":"2024-09-10T22:02:35.723917+0000","last_fullsized":"2024-09-10T22:02:35.723917+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:54.245142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727756+0000","last_change":"2024-09-10T22:00:43.716323+0000","last_active":"2024-09-10T22:02:35.727756+0000","last_peered":"2024-09-10T22:02:35.727756+0000","last_clean":"2024-09-10T22:02:35.727756+0000","last_became_active":"2024-09-10T22:00:43.715827+0000","last_became_peered":"2024-09-10T22:00:43.715827+0000","last_unstale":"2024-09-10T22:02:35.727756+0000","last_undegraded":"2024-09-10T22:02:35.727756+0000","last_fullsized":"2024-09-10T22:02:35.727756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:10.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726443+0000","last_change":"2024-09-10T22:00:34.500114+0000","last_active":"2024-09-10T22:02:35.726443+0000","last_peered":"2024-09-10T22:02:35.726443+0000","last_clean":"2024-09-10T22:02:35.726443+0000","last_became_active":"2024-09-10T22:00:34.499334+0000","last_became_peered":"2024-09-10T22:00:34.499334+0000","last_unstale":"2024-09-10T22:02:35.726443+0000","last_undegraded":"2024-09-10T22:02:35.726443+0000","last_fullsized":"2024-09-10T22:02:35.726443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:11.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":219,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:38.878447+0000","last_change":"2024-09-10T22:00:30.822561+0000","last_active":"2024-09-10T22:02:38.878447+0000","last_peered":"2024-09-10T22:02:38.878447+0000","last_clean":"2024-09-10T22:02:38.878447+0000","last_became_active":"2024-09-10T22:00:30.822201+0000","last_became_peered":"2024-09-10T22:00:30.822201+0000","last_unstale":"2024-09-10T22:02:38.878447+0000","last_undegraded":"2024-09-10T22:02:38.878447+0000","last_fullsized":"2024-09-10T22:02:38.878447+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:41.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62657,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":142,"num_read_kb":132,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871420+0000","last_change":"2024-09-10T22:01:35.444988+0000","last_active":"2024-09-10T22:02:35.871420+0000","last_peered":"2024-09-10T22:02:35.871420+0000","last_clean":"2024-09-10T22:02:35.871420+0000","last_became_active":"2024-09-10T22:01:35.444498+0000","last_became_peered":"2024-09-10T22:01:35.444498+0000","last_unstale":"2024-09-10T22:02:35.871420+0000","last_undegraded":"2024-09-10T22:02:35.871420+0000","last_fullsized":"2024-09-10T22:02:35.871420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:35.017660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948621+0000","last_change":"2024-09-10T22:00:34.503234+0000","last_active":"2024-09-10T22:02:35.948621+0000","last_peered":"2024-09-10T22:02:35.948621+0000","last_clean":"2024-09-10T22:02:35.948621+0000","last_became_active":"2024-09-10T22:00:34.501501+0000","last_became_peered":"2024-09-10T22:00:34.501501+0000","last_unstale":"2024-09-10T22:02:35.948621+0000","last_undegraded":"2024-09-10T22:02:35.948621+0000","last_fullsized":"2024-09-10T22:02:35.948621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:23.772491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728049+0000","last_change":"2024-09-10T22:00:31.082131+0000","last_active":"2024-09-10T22:02:35.728049+0000","last_peered":"2024-09-10T22:02:35.728049+0000","last_clean":"2024-09-10T22:02:35.728049+0000","last_became_active":"2024-09-10T22:00:31.081405+0000","last_became_peered":"2024-09-10T22:00:31.081405+0000","last_unstale":"2024-09-10T22:02:35.728049+0000","last_undegraded":"2024-09-10T22:02:35.728049+0000","last_fullsized":"2024-09-10T22:02:35.728049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:02.400004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897364+0000","last_change":"2024-09-10T22:00:43.705732+0000","last_active":"2024-09-10T22:02:35.897364+0000","last_peered":"2024-09-10T22:02:35.897364+0000","last_clean":"2024-09-10T22:02:35.897364+0000","last_became_active":"2024-09-10T22:00:43.705274+0000","last_became_peered":"2024-09-10T22:00:43.705274+0000","last_unstale":"2024-09-10T22:02:35.897364+0000","last_undegraded":"2024-09-10T22:02:35.897364+0000","last_fullsized":"2024-09-10T22:02:35.897364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:54.707636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948169+0000","last_change":"2024-09-10T22:00:34.499585+0000","last_active":"2024-09-10T22:02:35.948169+0000","last_peered":"2024-09-10T22:02:35.948169+0000","last_clean":"2024-09-10T22:02:35.948169+0000","last_became_active":"2024-09-10T22:00:34.499261+0000","last_became_peered":"2024-09-10T22:00:34.499261+0000","last_unstale":"2024-09-10T22:02:35.948169+0000","last_undegraded":"2024-09-10T22:02:35.948169+0000","last_fullsized":"2024-09-10T22:02:35.948169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:41.156093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863634+0000","last_change":"2024-09-10T22:00:30.823322+0000","last_active":"2024-09-10T22:02:35.863634+0000","last_peered":"2024-09-10T22:02:35.863634+0000","last_clean":"2024-09-10T22:02:35.863634+0000","last_became_active":"2024-09-10T22:00:30.822987+0000","last_became_peered":"2024-09-10T22:00:30.822987+0000","last_unstale":"2024-09-10T22:02:35.863634+0000","last_undegraded":"2024-09-10T22:02:35.863634+0000","last_fullsized":"2024-09-10T22:02:35.863634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:33.605627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723207+0000","last_change":"2024-09-10T22:00:43.708746+0000","last_active":"2024-09-10T22:02:35.723207+0000","last_peered":"2024-09-10T22:02:35.723207+0000","last_clean":"2024-09-10T22:02:35.723207+0000","last_became_active":"2024-09-10T22:00:43.708283+0000","last_became_peered":"2024-09-10T22:00:43.708283+0000","last_unstale":"2024-09-10T22:02:35.723207+0000","last_undegraded":"2024-09-10T22:02:35.723207+0000","last_fullsized":"2024-09-10T22:02:35.723207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:30.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948916+0000","last_change":"2024-09-10T22:00:34.501472+0000","last_active":"2024-09-10T22:02:35.948916+0000","last_peered":"2024-09-10T22:02:35.948916+0000","last_clean":"2024-09-10T22:02:35.948916+0000","last_became_active":"2024-09-10T22:00:34.501131+0000","last_became_peered":"2024-09-10T22:00:34.501131+0000","last_unstale":"2024-09-10T22:02:35.948916+0000","last_undegraded":"2024-09-10T22:02:35.948916+0000","last_fullsized":"2024-09-10T22:02:35.948916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:11.281413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":181,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:38.877264+0000","last_change":"2024-09-10T22:00:30.828335+0000","last_active":"2024-09-10T22:02:38.877264+0000","last_peered":"2024-09-10T22:02:38.877264+0000","last_clean":"2024-09-10T22:02:38.877264+0000","last_became_active":"2024-09-10T22:00:30.828028+0000","last_became_peered":"2024-09-10T22:00:30.828028+0000","last_unstale":"2024-09-10T22:02:38.877264+0000","last_undegraded":"2024-09-10T22:02:38.877264+0000","last_fullsized":"2024-09-10T22:02:38.877264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:56.399002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722425+0000","last_change":"2024-09-10T22:00:43.710086+0000","last_active":"2024-09-10T22:02:35.722425+0000","last_peered":"2024-09-10T22:02:35.722425+0000","last_clean":"2024-09-10T22:02:35.722425+0000","last_became_active":"2024-09-10T22:00:43.709518+0000","last_became_peered":"2024-09-10T22:00:43.709518+0000","last_unstale":"2024-09-10T22:02:35.722425+0000","last_undegraded":"2024-09-10T22:02:35.722425+0000","last_fullsized":"2024-09-10T22:02:35.722425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:07.636887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947921+0000","last_change":"2024-09-10T22:00:43.711620+0000","last_active":"2024-09-10T22:02:35.947921+0000","last_peered":"2024-09-10T22:02:35.947921+0000","last_clean":"2024-09-10T22:02:35.947921+0000","last_became_active":"2024-09-10T22:00:43.711224+0000","last_became_peered":"2024-09-10T22:00:43.711224+0000","last_unstale":"2024-09-10T22:02:35.947921+0000","last_undegraded":"2024-09-10T22:02:35.947921+0000","last_fullsized":"2024-09-10T22:02:35.947921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:43.860286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722924+0000","last_change":"2024-09-10T22:00:34.494408+0000","last_active":"2024-09-10T22:02:35.722924+0000","last_peered":"2024-09-10T22:02:35.722924+0000","last_clean":"2024-09-10T22:02:35.722924+0000","last_became_active":"2024-09-10T22:00:34.494066+0000","last_became_peered":"2024-09-10T22:00:34.494066+0000","last_unstale":"2024-09-10T22:02:35.722924+0000","last_undegraded":"2024-09-10T22:02:35.722924+0000","last_fullsized":"2024-09-10T22:02:35.722924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:49.565755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727659+0000","last_change":"2024-09-10T22:00:30.821842+0000","last_active":"2024-09-10T22:02:35.727659+0000","last_peered":"2024-09-10T22:02:35.727659+0000","last_clean":"2024-09-10T22:02:35.727659+0000","last_became_active":"2024-09-10T22:00:30.821499+0000","last_became_peered":"2024-09-10T22:00:30.821499+0000","last_unstale":"2024-09-10T22:02:35.727659+0000","last_undegraded":"2024-09-10T22:02:35.727659+0000","last_fullsized":"2024-09-10T22:02:35.727659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:19.125052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948671+0000","last_change":"2024-09-10T22:00:43.706689+0000","last_active":"2024-09-10T22:02:35.948671+0000","last_peered":"2024-09-10T22:02:35.948671+0000","last_clean":"2024-09-10T22:02:35.948671+0000","last_became_active":"2024-09-10T22:00:43.705954+0000","last_became_peered":"2024-09-10T22:00:43.705954+0000","last_unstale":"2024-09-10T22:02:35.948671+0000","last_undegraded":"2024-09-10T22:02:35.948671+0000","last_fullsized":"2024-09-10T22:02:35.948671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:30.800710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722387+0000","last_change":"2024-09-10T22:00:34.490853+0000","last_active":"2024-09-10T22:02:35.722387+0000","last_peered":"2024-09-10T22:02:35.722387+0000","last_clean":"2024-09-10T22:02:35.722387+0000","last_became_active":"2024-09-10T22:00:34.489908+0000","last_became_peered":"2024-09-10T22:00:34.489908+0000","last_unstale":"2024-09-10T22:02:35.722387+0000","last_undegraded":"2024-09-10T22:02:35.722387+0000","last_fullsized":"2024-09-10T22:02:35.722387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:57.105004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893871+0000","last_change":"2024-09-10T22:00:30.816497+0000","last_active":"2024-09-10T22:02:35.893871+0000","last_peered":"2024-09-10T22:02:35.893871+0000","last_clean":"2024-09-10T22:02:35.893871+0000","last_became_active":"2024-09-10T22:00:30.816181+0000","last_became_peered":"2024-09-10T22:00:30.816181+0000","last_unstale":"2024-09-10T22:02:35.893871+0000","last_undegraded":"2024-09-10T22:02:35.893871+0000","last_fullsized":"2024-09-10T22:02:35.893871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:12.779113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871729+0000","last_change":"2024-09-10T22:00:30.821402+0000","last_active":"2024-09-10T22:02:35.871729+0000","last_peered":"2024-09-10T22:02:35.871729+0000","last_clean":"2024-09-10T22:02:35.871729+0000","last_became_active":"2024-09-10T22:00:30.821105+0000","last_became_peered":"2024-09-10T22:00:30.821105+0000","last_unstale":"2024-09-10T22:02:35.871729+0000","last_undegraded":"2024-09-10T22:02:35.871729+0000","last_fullsized":"2024-09-10T22:02:35.871729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:36.090579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727376+0000","last_change":"2024-09-10T22:00:34.511911+0000","last_active":"2024-09-10T22:02:35.727376+0000","last_peered":"2024-09-10T22:02:35.727376+0000","last_clean":"2024-09-10T22:02:35.727376+0000","last_became_active":"2024-09-10T22:00:34.511545+0000","last_became_peered":"2024-09-10T22:00:34.511545+0000","last_unstale":"2024-09-10T22:02:35.727376+0000","last_undegraded":"2024-09-10T22:02:35.727376+0000","last_fullsized":"2024-09-10T22:02:35.727376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:12.484733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723478+0000","last_change":"2024-09-10T22:01:35.750458+0000","last_active":"2024-09-10T22:02:35.723478+0000","last_peered":"2024-09-10T22:02:35.723478+0000","last_clean":"2024-09-10T22:02:35.723478+0000","last_became_active":"2024-09-10T22:01:35.749946+0000","last_became_peered":"2024-09-10T22:01:35.749946+0000","last_unstale":"2024-09-10T22:02:35.723478+0000","last_undegraded":"2024-09-10T22:02:35.723478+0000","last_fullsized":"2024-09-10T22:02:35.723478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.362221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:40.881186+0000","last_change":"2024-09-10T22:00:30.826427+0000","last_active":"2024-09-10T22:02:40.881186+0000","last_peered":"2024-09-10T22:02:40.881186+0000","last_clean":"2024-09-10T22:02:40.881186+0000","last_became_active":"2024-09-10T22:00:30.826038+0000","last_became_peered":"2024-09-10T22:00:30.826038+0000","last_unstale":"2024-09-10T22:02:40.881186+0000","last_undegraded":"2024-09-10T22:02:40.881186+0000","last_fullsized":"2024-09-10T22:02:40.881186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:45.075208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726178+0000","last_change":"2024-09-10T22:00:34.494479+0000","last_active":"2024-09-10T22:02:35.726178+0000","last_peered":"2024-09-10T22:02:35.726178+0000","last_clean":"2024-09-10T22:02:35.726178+0000","last_became_active":"2024-09-10T22:00:34.494087+0000","last_became_peered":"2024-09-10T22:00:34.494087+0000","last_unstale":"2024-09-10T22:02:35.726178+0000","last_undegraded":"2024-09-10T22:02:35.726178+0000","last_fullsized":"2024-09-10T22:02:35.726178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:28.548999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723543+0000","last_change":"2024-09-10T22:00:43.714568+0000","last_active":"2024-09-10T22:02:35.723543+0000","last_peered":"2024-09-10T22:02:35.723543+0000","last_clean":"2024-09-10T22:02:35.723543+0000","last_became_active":"2024-09-10T22:00:43.712923+0000","last_became_peered":"2024-09-10T22:00:43.712923+0000","last_unstale":"2024-09-10T22:02:35.723543+0000","last_undegraded":"2024-09-10T22:02:35.723543+0000","last_fullsized":"2024-09-10T22:02:35.723543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:27.945799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948939+0000","last_change":"2024-09-10T22:00:34.496942+0000","last_active":"2024-09-10T22:02:35.948939+0000","last_peered":"2024-09-10T22:02:35.948939+0000","last_clean":"2024-09-10T22:02:35.948939+0000","last_became_active":"2024-09-10T22:00:34.496475+0000","last_became_peered":"2024-09-10T22:00:34.496475+0000","last_unstale":"2024-09-10T22:02:35.948939+0000","last_undegraded":"2024-09-10T22:02:35.948939+0000","last_fullsized":"2024-09-10T22:02:35.948939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:33.277346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723562+0000","last_change":"2024-09-10T22:00:34.506571+0000","last_active":"2024-09-10T22:02:35.723562+0000","last_peered":"2024-09-10T22:02:35.723562+0000","last_clean":"2024-09-10T22:02:35.723562+0000","last_became_active":"2024-09-10T22:00:34.505718+0000","last_became_peered":"2024-09-10T22:00:34.505718+0000","last_unstale":"2024-09-10T22:02:35.723562+0000","last_undegraded":"2024-09-10T22:02:35.723562+0000","last_fullsized":"2024-09-10T22:02:35.723562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:10.111415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723877+0000","last_change":"2024-09-10T22:00:34.502030+0000","last_active":"2024-09-10T22:02:35.723877+0000","last_peered":"2024-09-10T22:02:35.723877+0000","last_clean":"2024-09-10T22:02:35.723877+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:02:35.723877+0000","last_undegraded":"2024-09-10T22:02:35.723877+0000","last_fullsized":"2024-09-10T22:02:35.723877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:09.532523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947883+0000","last_change":"2024-09-10T22:00:34.513630+0000","last_active":"2024-09-10T22:02:35.947883+0000","last_peered":"2024-09-10T22:02:35.947883+0000","last_clean":"2024-09-10T22:02:35.947883+0000","last_became_active":"2024-09-10T22:00:34.513348+0000","last_became_peered":"2024-09-10T22:00:34.513348+0000","last_unstale":"2024-09-10T22:02:35.947883+0000","last_undegraded":"2024-09-10T22:02:35.947883+0000","last_fullsized":"2024-09-10T22:02:35.947883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:53.942004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894303+0000","last_change":"2024-09-10T22:00:34.508697+0000","last_active":"2024-09-10T22:02:35.894303+0000","last_peered":"2024-09-10T22:02:35.894303+0000","last_clean":"2024-09-10T22:02:35.894303+0000","last_became_active":"2024-09-10T22:00:34.508371+0000","last_became_peered":"2024-09-10T22:00:34.508371+0000","last_unstale":"2024-09-10T22:02:35.894303+0000","last_undegraded":"2024-09-10T22:02:35.894303+0000","last_fullsized":"2024-09-10T22:02:35.894303+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:32.843517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948743+0000","last_change":"2024-09-10T22:00:34.496021+0000","last_active":"2024-09-10T22:02:35.948743+0000","last_peered":"2024-09-10T22:02:35.948743+0000","last_clean":"2024-09-10T22:02:35.948743+0000","last_became_active":"2024-09-10T22:00:34.495663+0000","last_became_peered":"2024-09-10T22:00:34.495663+0000","last_unstale":"2024-09-10T22:02:35.948743+0000","last_undegraded":"2024-09-10T22:02:35.948743+0000","last_fullsized":"2024-09-10T22:02:35.948743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:16.512182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724019+0000","last_change":"2024-09-10T22:00:34.506589+0000","last_active":"2024-09-10T22:02:35.724019+0000","last_peered":"2024-09-10T22:02:35.724019+0000","last_clean":"2024-09-10T22:02:35.724019+0000","last_became_active":"2024-09-10T22:00:34.506214+0000","last_became_peered":"2024-09-10T22:00:34.506214+0000","last_unstale":"2024-09-10T22:02:35.724019+0000","last_undegraded":"2024-09-10T22:02:35.724019+0000","last_fullsized":"2024-09-10T22:02:35.724019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:47.072458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723921+0000","last_change":"2024-09-10T22:00:34.503471+0000","last_active":"2024-09-10T22:02:35.723921+0000","last_peered":"2024-09-10T22:02:35.723921+0000","last_clean":"2024-09-10T22:02:35.723921+0000","last_became_active":"2024-09-10T22:00:34.501145+0000","last_became_peered":"2024-09-10T22:00:34.501145+0000","last_unstale":"2024-09-10T22:02:35.723921+0000","last_undegraded":"2024-09-10T22:02:35.723921+0000","last_fullsized":"2024-09-10T22:02:35.723921+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:24.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727219+0000","last_change":"2024-09-10T22:00:34.490177+0000","last_active":"2024-09-10T22:02:35.727219+0000","last_peered":"2024-09-10T22:02:35.727219+0000","last_clean":"2024-09-10T22:02:35.727219+0000","last_became_active":"2024-09-10T22:00:34.488428+0000","last_became_peered":"2024-09-10T22:00:34.488428+0000","last_unstale":"2024-09-10T22:02:35.727219+0000","last_undegraded":"2024-09-10T22:02:35.727219+0000","last_fullsized":"2024-09-10T22:02:35.727219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:21.111826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722506+0000","last_change":"2024-09-10T22:00:34.499174+0000","last_active":"2024-09-10T22:02:35.722506+0000","last_peered":"2024-09-10T22:02:35.722506+0000","last_clean":"2024-09-10T22:02:35.722506+0000","last_became_active":"2024-09-10T22:00:34.498824+0000","last_became_peered":"2024-09-10T22:00:34.498824+0000","last_unstale":"2024-09-10T22:02:35.722506+0000","last_undegraded":"2024-09-10T22:02:35.722506+0000","last_fullsized":"2024-09-10T22:02:35.722506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:15.946270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948370+0000","last_change":"2024-09-10T22:00:34.505748+0000","last_active":"2024-09-10T22:02:35.948370+0000","last_peered":"2024-09-10T22:02:35.948370+0000","last_clean":"2024-09-10T22:02:35.948370+0000","last_became_active":"2024-09-10T22:00:34.500441+0000","last_became_peered":"2024-09-10T22:00:34.500441+0000","last_unstale":"2024-09-10T22:02:35.948370+0000","last_undegraded":"2024-09-10T22:02:35.948370+0000","last_fullsized":"2024-09-10T22:02:35.948370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:22.177566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894288+0000","last_change":"2024-09-10T22:00:34.502319+0000","last_active":"2024-09-10T22:02:35.894288+0000","last_peered":"2024-09-10T22:02:35.894288+0000","last_clean":"2024-09-10T22:02:35.894288+0000","last_became_active":"2024-09-10T22:00:34.501427+0000","last_became_peered":"2024-09-10T22:00:34.501427+0000","last_unstale":"2024-09-10T22:02:35.894288+0000","last_undegraded":"2024-09-10T22:02:35.894288+0000","last_fullsized":"2024-09-10T22:02:35.894288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:18.463319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948529+0000","last_change":"2024-09-10T22:00:34.497571+0000","last_active":"2024-09-10T22:02:35.948529+0000","last_peered":"2024-09-10T22:02:35.948529+0000","last_clean":"2024-09-10T22:02:35.948529+0000","last_became_active":"2024-09-10T22:00:34.497205+0000","last_became_peered":"2024-09-10T22:00:34.497205+0000","last_unstale":"2024-09-10T22:02:35.948529+0000","last_undegraded":"2024-09-10T22:02:35.948529+0000","last_fullsized":"2024-09-10T22:02:35.948529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:49.924518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723880+0000","last_change":"2024-09-10T22:00:34.504641+0000","last_active":"2024-09-10T22:02:35.723880+0000","last_peered":"2024-09-10T22:02:35.723880+0000","last_clean":"2024-09-10T22:02:35.723880+0000","last_became_active":"2024-09-10T22:00:34.504183+0000","last_became_peered":"2024-09-10T22:00:34.504183+0000","last_unstale":"2024-09-10T22:02:35.723880+0000","last_undegraded":"2024-09-10T22:02:35.723880+0000","last_fullsized":"2024-09-10T22:02:35.723880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:06.704972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723361+0000","last_change":"2024-09-10T22:00:34.503884+0000","last_active":"2024-09-10T22:02:35.723361+0000","last_peered":"2024-09-10T22:02:35.723361+0000","last_clean":"2024-09-10T22:02:35.723361+0000","last_became_active":"2024-09-10T22:00:34.502950+0000","last_became_peered":"2024-09-10T22:00:34.502950+0000","last_unstale":"2024-09-10T22:02:35.723361+0000","last_undegraded":"2024-09-10T22:02:35.723361+0000","last_fullsized":"2024-09-10T22:02:35.723361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:45.560958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.949045+0000","last_change":"2024-09-10T22:00:34.504981+0000","last_active":"2024-09-10T22:02:35.949045+0000","last_peered":"2024-09-10T22:02:35.949045+0000","last_clean":"2024-09-10T22:02:35.949045+0000","last_became_active":"2024-09-10T22:00:34.500474+0000","last_became_peered":"2024-09-10T22:00:34.500474+0000","last_unstale":"2024-09-10T22:02:35.949045+0000","last_undegraded":"2024-09-10T22:02:35.949045+0000","last_fullsized":"2024-09-10T22:02:35.949045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:20.892605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948231+0000","last_change":"2024-09-10T22:00:34.506650+0000","last_active":"2024-09-10T22:02:35.948231+0000","last_peered":"2024-09-10T22:02:35.948231+0000","last_clean":"2024-09-10T22:02:35.948231+0000","last_became_active":"2024-09-10T22:00:34.505698+0000","last_became_peered":"2024-09-10T22:00:34.505698+0000","last_unstale":"2024-09-10T22:02:35.948231+0000","last_undegraded":"2024-09-10T22:02:35.948231+0000","last_fullsized":"2024-09-10T22:02:35.948231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:29.221693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947606+0000","last_change":"2024-09-10T22:00:34.502238+0000","last_active":"2024-09-10T22:02:35.947606+0000","last_peered":"2024-09-10T22:02:35.947606+0000","last_clean":"2024-09-10T22:02:35.947606+0000","last_became_active":"2024-09-10T22:00:34.500049+0000","last_became_peered":"2024-09-10T22:00:34.500049+0000","last_unstale":"2024-09-10T22:02:35.947606+0000","last_undegraded":"2024-09-10T22:02:35.947606+0000","last_fullsized":"2024-09-10T22:02:35.947606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:05.669784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947630+0000","last_change":"2024-09-10T22:00:34.499340+0000","last_active":"2024-09-10T22:02:35.947630+0000","last_peered":"2024-09-10T22:02:35.947630+0000","last_clean":"2024-09-10T22:02:35.947630+0000","last_became_active":"2024-09-10T22:00:34.497655+0000","last_became_peered":"2024-09-10T22:00:34.497655+0000","last_unstale":"2024-09-10T22:02:35.947630+0000","last_undegraded":"2024-09-10T22:02:35.947630+0000","last_fullsized":"2024-09-10T22:02:35.947630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:34.115829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894275+0000","last_change":"2024-09-10T22:00:34.500705+0000","last_active":"2024-09-10T22:02:35.894275+0000","last_peered":"2024-09-10T22:02:35.894275+0000","last_clean":"2024-09-10T22:02:35.894275+0000","last_became_active":"2024-09-10T22:00:34.500071+0000","last_became_peered":"2024-09-10T22:00:34.500071+0000","last_unstale":"2024-09-10T22:02:35.894275+0000","last_undegraded":"2024-09-10T22:02:35.894275+0000","last_fullsized":"2024-09-10T22:02:35.894275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:40.519976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897606+0000","last_change":"2024-09-10T22:00:34.510570+0000","last_active":"2024-09-10T22:02:35.897606+0000","last_peered":"2024-09-10T22:02:35.897606+0000","last_clean":"2024-09-10T22:02:35.897606+0000","last_became_active":"2024-09-10T22:00:34.506922+0000","last_became_peered":"2024-09-10T22:00:34.506922+0000","last_unstale":"2024-09-10T22:02:35.897606+0000","last_undegraded":"2024-09-10T22:02:35.897606+0000","last_fullsized":"2024-09-10T22:02:35.897606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.489031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863526+0000","last_change":"2024-09-10T22:00:34.509985+0000","last_active":"2024-09-10T22:02:35.863526+0000","last_peered":"2024-09-10T22:02:35.863526+0000","last_clean":"2024-09-10T22:02:35.863526+0000","last_became_active":"2024-09-10T22:00:34.509527+0000","last_became_peered":"2024-09-10T22:00:34.509527+0000","last_unstale":"2024-09-10T22:02:35.863526+0000","last_undegraded":"2024-09-10T22:02:35.863526+0000","last_fullsized":"2024-09-10T22:02:35.863526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:49.463334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.872121+0000","last_change":"2024-09-10T22:00:34.497986+0000","last_active":"2024-09-10T22:02:35.872121+0000","last_peered":"2024-09-10T22:02:35.872121+0000","last_clean":"2024-09-10T22:02:35.872121+0000","last_became_active":"2024-09-10T22:00:34.497612+0000","last_became_peered":"2024-09-10T22:00:34.497612+0000","last_unstale":"2024-09-10T22:02:35.872121+0000","last_undegraded":"2024-09-10T22:02:35.872121+0000","last_fullsized":"2024-09-10T22:02:35.872121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:15.696988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.949002+0000","last_change":"2024-09-10T22:00:34.498482+0000","last_active":"2024-09-10T22:02:35.949002+0000","last_peered":"2024-09-10T22:02:35.949002+0000","last_clean":"2024-09-10T22:02:35.949002+0000","last_became_active":"2024-09-10T22:00:34.498139+0000","last_became_peered":"2024-09-10T22:00:34.498139+0000","last_unstale":"2024-09-10T22:02:35.949002+0000","last_undegraded":"2024-09-10T22:02:35.949002+0000","last_fullsized":"2024-09-10T22:02:35.949002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:23.427253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722383+0000","last_change":"2024-09-10T22:00:34.491325+0000","last_active":"2024-09-10T22:02:35.722383+0000","last_peered":"2024-09-10T22:02:35.722383+0000","last_clean":"2024-09-10T22:02:35.722383+0000","last_became_active":"2024-09-10T22:00:34.491026+0000","last_became_peered":"2024-09-10T22:00:34.491026+0000","last_unstale":"2024-09-10T22:02:35.722383+0000","last_undegraded":"2024-09-10T22:02:35.722383+0000","last_fullsized":"2024-09-10T22:02:35.722383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:07.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.862981+0000","last_change":"2024-09-10T22:00:34.503975+0000","last_active":"2024-09-10T22:02:35.862981+0000","last_peered":"2024-09-10T22:02:35.862981+0000","last_clean":"2024-09-10T22:02:35.862981+0000","last_became_active":"2024-09-10T22:00:34.503624+0000","last_became_peered":"2024-09-10T22:00:34.503624+0000","last_unstale":"2024-09-10T22:02:35.862981+0000","last_undegraded":"2024-09-10T22:02:35.862981+0000","last_fullsized":"2024-09-10T22:02:35.862981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:17.678811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871637+0000","last_change":"2024-09-10T22:01:35.596824+0000","last_active":"2024-09-10T22:02:35.871637+0000","last_peered":"2024-09-10T22:02:35.871637+0000","last_clean":"2024-09-10T22:02:35.871637+0000","last_became_active":"2024-09-10T22:01:35.596550+0000","last_became_peered":"2024-09-10T22:01:35.596550+0000","last_unstale":"2024-09-10T22:02:35.871637+0000","last_undegraded":"2024-09-10T22:02:35.871637+0000","last_fullsized":"2024-09-10T22:02:35.871637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:19.617649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.895388+0000","last_change":"2024-09-10T22:02:35.895282+0000","last_active":"2024-09-10T22:02:35.895388+0000","last_peered":"2024-09-10T22:02:35.895388+0000","last_clean":"2024-09-10T22:02:35.895388+0000","last_became_active":"2024-09-10T22:02:35.894850+0000","last_became_peered":"2024-09-10T22:02:35.894850+0000","last_unstale":"2024-09-10T22:02:35.895388+0000","last_undegraded":"2024-09-10T22:02:35.895388+0000","last_fullsized":"2024-09-10T22:02:35.895388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:50.109012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897539+0000","last_change":"2024-09-10T22:00:34.513134+0000","last_active":"2024-09-10T22:02:35.897539+0000","last_peered":"2024-09-10T22:02:35.897539+0000","last_clean":"2024-09-10T22:02:35.897539+0000","last_became_active":"2024-09-10T22:00:34.507578+0000","last_became_peered":"2024-09-10T22:00:34.507578+0000","last_unstale":"2024-09-10T22:02:35.897539+0000","last_undegraded":"2024-09-10T22:02:35.897539+0000","last_fullsized":"2024-09-10T22:02:35.897539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:33.387337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947814+0000","last_change":"2024-09-10T22:00:34.507645+0000","last_active":"2024-09-10T22:02:35.947814+0000","last_peered":"2024-09-10T22:02:35.947814+0000","last_clean":"2024-09-10T22:02:35.947814+0000","last_became_active":"2024-09-10T22:00:34.503614+0000","last_became_peered":"2024-09-10T22:00:34.503614+0000","last_unstale":"2024-09-10T22:02:35.947814+0000","last_undegraded":"2024-09-10T22:02:35.947814+0000","last_fullsized":"2024-09-10T22:02:35.947814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:53.033523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947997+0000","last_change":"2024-09-10T22:00:34.501891+0000","last_active":"2024-09-10T22:02:35.947997+0000","last_peered":"2024-09-10T22:02:35.947997+0000","last_clean":"2024-09-10T22:02:35.947997+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:02:35.947997+0000","last_undegraded":"2024-09-10T22:02:35.947997+0000","last_fullsized":"2024-09-10T22:02:35.947997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:30.068461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.949048+0000","last_change":"2024-09-10T22:00:34.501475+0000","last_active":"2024-09-10T22:02:35.949048+0000","last_peered":"2024-09-10T22:02:35.949048+0000","last_clean":"2024-09-10T22:02:35.949048+0000","last_became_active":"2024-09-10T22:00:34.500909+0000","last_became_peered":"2024-09-10T22:00:34.500909+0000","last_unstale":"2024-09-10T22:02:35.949048+0000","last_undegraded":"2024-09-10T22:02:35.949048+0000","last_fullsized":"2024-09-10T22:02:35.949048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:39.616773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":716,"num_read_kb":433,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246272,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678057,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":536256,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874286,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1155072,"data_stored":1185617,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037813,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":539779,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234044,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":535949,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430274,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":1133340,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593801,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1003520,"data_stored":531578,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790032,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1085440,"data_stored":1128140,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953558,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":535652,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":530895,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378721,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1085440,"data_stored":595764,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542248,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1081344,"data_stored":595377,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":533034,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:02:45.452 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:02:45.825 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:45.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:45 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:45.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:45 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:45.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:45 smithi090 ceph-mon[30350]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:02:45.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:45 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:02:45.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:45 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:02:45.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:45 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:45.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:45 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:45.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:45 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:45.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:45 smithi154 ceph-mon[34495]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:02:45.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:45 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:02:45.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:45 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:02:45.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:45 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:45.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:45 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:45.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:45 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:45.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:45 smithi171 ceph-mon[34138]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:02:45.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:45 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:02:45.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:45 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:02:45.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:45 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:02:46.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.448+0000 7f65fa200640 1 -- 172.21.15.90:0/3076906249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 msgr2=0x7f65f4108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.448+0000 7f65fa200640 1 --2- 172.21.15.90:0/3076906249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 0x7f65f4108a60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009900 tx=0x7f65e002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.448+0000 7f65fa200640 1 -- 172.21.15.90:0/3076906249 shutdown_connections 2024-09-10T22:02:46.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.448+0000 7f65fa200640 1 --2- 172.21.15.90:0/3076906249 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4108fa0 0x7f65f410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.448+0000 7f65fa200640 1 --2- 172.21.15.90:0/3076906249 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 0x7f65f4108a60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.448+0000 7f65fa200640 1 --2- 172.21.15.90:0/3076906249 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f4102c10 0x7f65f4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.448+0000 7f65fa200640 1 -- 172.21.15.90:0/3076906249 >> 172.21.15.90:0/3076906249 conn(0x7f65f40fe6c0 msgr2=0x7f65f4100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.448+0000 7f65fa200640 1 -- 172.21.15.90:0/3076906249 shutdown_connections 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.448+0000 7f65fa200640 1 -- 172.21.15.90:0/3076906249 wait complete. 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.449+0000 7f65fa200640 1 Processor -- start 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.449+0000 7f65fa200640 1 -- start start 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.449+0000 7f65fa200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4102c10 0x7f65f40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.449+0000 7f65fa200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 0x7f65f40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.449+0000 7f65fa200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f4108fa0 0x7f65f4075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65fa200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4112b10 con 0x7f65f4108620 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65fa200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4112990 con 0x7f65f4108fa0 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65fa200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4112c90 con 0x7f65f4102c10 2024-09-10T22:02:46.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 0x7f65f40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 0x7f65f40759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52758/0 (socket says 172.21.15.90:52758) 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f3e00640 1 -- 172.21.15.90:0/4274427885 learned_addr learned my addr 172.21.15.90:0/4274427885 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f8e00640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4102c10 0x7f65f40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f9800640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f4108fa0 0x7f65f4075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f3e00640 1 -- 172.21.15.90:0/4274427885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4102c10 msgr2=0x7f65f40754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f3e00640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4102c10 0x7f65f40754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f3e00640 1 -- 172.21.15.90:0/4274427885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f4108fa0 msgr2=0x7f65f4075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f3e00640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f4108fa0 0x7f65f4075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.450+0000 7f65f3e00640 1 -- 172.21.15.90:0/4274427885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f41a73c0 con 0x7f65f4108620 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.451+0000 7f65f3e00640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 0x7f65f40759e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009a20 tx=0x7f65e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.451+0000 7f65f1600640 1 -- 172.21.15.90:0/4274427885 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e003f030 con 0x7f65f4108620 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.451+0000 7f65f1600640 1 -- 172.21.15.90:0/4274427885 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65e00035f0 con 0x7f65f4108620 2024-09-10T22:02:46.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.451+0000 7f65f1600640 1 -- 172.21.15.90:0/4274427885 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0035070 con 0x7f65f4108620 2024-09-10T22:02:46.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.451+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f41a7650 con 0x7f65f4108620 2024-09-10T22:02:46.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.451+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41a7b60 con 0x7f65f4108620 2024-09-10T22:02:46.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.452+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f4102ff0 con 0x7f65f4108620 2024-09-10T22:02:46.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.455+0000 7f65f1600640 1 -- 172.21.15.90:0/4274427885 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f65e0031c80 con 0x7f65f4108620 2024-09-10T22:02:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.456+0000 7f65f1600640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b54e0 0x7f65d80b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:46.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.456+0000 7f65f1600640 1 -- 172.21.15.90:0/4274427885 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f65e0107170 con 0x7f65f4108620 2024-09-10T22:02:46.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.456+0000 7f65f8e00640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b54e0 0x7f65d80b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:46.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.457+0000 7f65f8e00640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b54e0 0x7f65d80b79a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65e4005c40 tx=0x7f65e400a590 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:46.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.458+0000 7f65f1600640 1 -- 172.21.15.90:0/4274427885 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f65e00cd9e0 con 0x7f65f4108620 2024-09-10T22:02:46.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:46 smithi090 ceph-mon[30350]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T22:02:46.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.640+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65f4063590 con 0x7f65d80b54e0 2024-09-10T22:02:46.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.646+0000 7f65f1600640 1 -- 172.21.15.90:0/4274427885 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340665 (secure 0 0 0) 0x7f65f4063590 con 0x7f65d80b54e0 2024-09-10T22:02:46.651 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:46.652 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-09-10T22:02:46.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b54e0 msgr2=0x7f65d80b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b54e0 0x7f65d80b79a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f65e4005c40 tx=0x7f65e400a590 comp rx=0 tx=0).stop 2024-09-10T22:02:46.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 msgr2=0x7f65f40759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:46.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 0x7f65f40759e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009a20 tx=0x7f65e0004270 comp rx=0 tx=0).stop 2024-09-10T22:02:46.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 shutdown_connections 2024-09-10T22:02:46.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b54e0 0x7f65d80b79a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f4108fa0 0x7f65f4075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f4108620 0x7f65f40759e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 --2- 172.21.15.90:0/4274427885 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4102c10 0x7f65f40754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:46.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 >> 172.21.15.90:0/4274427885 conn(0x7f65f40fe6c0 msgr2=0x7f65f4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:46.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 shutdown_connections 2024-09-10T22:02:46.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:46.650+0000 7f65fa200640 1 -- 172.21.15.90:0/4274427885 wait complete. 2024-09-10T22:02:46.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:46 smithi154 ceph-mon[34495]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T22:02:46.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:46 smithi171 ceph-mon[34138]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-10T22:02:47.459 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-10T22:02:45.615348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661245,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":832,"num_read_kb":522,"num_write":374,"num_write_kb":2296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":308,"ondisk_log_size":308,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349160,"kb_used_data":12308,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592664,"statfs":{"total":1151940427776,"available":1151582887936,"internally_reserved":0,"allocated":12603392,"data_stored":8381381,"data_compressed":498330,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002551"},"pg_stats":[{"pgid":"2.1f","version":"86'13","reported_seq":130,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:41.871904+0000","last_change":"2024-09-10T22:00:30.824265+0000","last_active":"2024-09-10T22:02:41.871904+0000","last_peered":"2024-09-10T22:02:41.871904+0000","last_clean":"2024-09-10T22:02:41.871904+0000","last_became_active":"2024-09-10T22:00:30.823905+0000","last_became_peered":"2024-09-10T22:00:30.823905+0000","last_unstale":"2024-09-10T22:02:41.871904+0000","last_undegraded":"2024-09-10T22:02:41.871904+0000","last_fullsized":"2024-09-10T22:02:41.871904+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:16.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723654+0000","last_change":"2024-09-10T22:00:34.497719+0000","last_active":"2024-09-10T22:02:35.723654+0000","last_peered":"2024-09-10T22:02:35.723654+0000","last_clean":"2024-09-10T22:02:35.723654+0000","last_became_active":"2024-09-10T22:00:34.497380+0000","last_became_peered":"2024-09-10T22:00:34.497380+0000","last_unstale":"2024-09-10T22:02:35.723654+0000","last_undegraded":"2024-09-10T22:02:35.723654+0000","last_fullsized":"2024-09-10T22:02:35.723654+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:26.336299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723219+0000","last_change":"2024-09-10T22:00:43.710835+0000","last_active":"2024-09-10T22:02:35.723219+0000","last_peered":"2024-09-10T22:02:35.723219+0000","last_clean":"2024-09-10T22:02:35.723219+0000","last_became_active":"2024-09-10T22:00:43.710363+0000","last_became_peered":"2024-09-10T22:00:43.710363+0000","last_unstale":"2024-09-10T22:02:35.723219+0000","last_undegraded":"2024-09-10T22:02:35.723219+0000","last_fullsized":"2024-09-10T22:02:35.723219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:28.173482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723977+0000","last_change":"2024-09-10T22:00:31.083228+0000","last_active":"2024-09-10T22:02:35.723977+0000","last_peered":"2024-09-10T22:02:35.723977+0000","last_clean":"2024-09-10T22:02:35.723977+0000","last_became_active":"2024-09-10T22:00:31.082911+0000","last_became_peered":"2024-09-10T22:00:31.082911+0000","last_unstale":"2024-09-10T22:02:35.723977+0000","last_undegraded":"2024-09-10T22:02:35.723977+0000","last_fullsized":"2024-09-10T22:02:35.723977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:28.598628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948166+0000","last_change":"2024-09-10T22:00:34.509939+0000","last_active":"2024-09-10T22:02:35.948166+0000","last_peered":"2024-09-10T22:02:35.948166+0000","last_clean":"2024-09-10T22:02:35.948166+0000","last_became_active":"2024-09-10T22:00:34.509536+0000","last_became_peered":"2024-09-10T22:00:34.509536+0000","last_unstale":"2024-09-10T22:02:35.948166+0000","last_undegraded":"2024-09-10T22:02:35.948166+0000","last_fullsized":"2024-09-10T22:02:35.948166+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:27.602378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897046+0000","last_change":"2024-09-10T22:00:43.714662+0000","last_active":"2024-09-10T22:02:35.897046+0000","last_peered":"2024-09-10T22:02:35.897046+0000","last_clean":"2024-09-10T22:02:35.897046+0000","last_became_active":"2024-09-10T22:00:43.714074+0000","last_became_peered":"2024-09-10T22:00:43.714074+0000","last_unstale":"2024-09-10T22:02:35.897046+0000","last_undegraded":"2024-09-10T22:02:35.897046+0000","last_fullsized":"2024-09-10T22:02:35.897046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:41.204519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948479+0000","last_change":"2024-09-10T22:01:35.209979+0000","last_active":"2024-09-10T22:02:35.948479+0000","last_peered":"2024-09-10T22:02:35.948479+0000","last_clean":"2024-09-10T22:02:35.948479+0000","last_became_active":"2024-09-10T22:01:35.197192+0000","last_became_peered":"2024-09-10T22:01:35.197192+0000","last_unstale":"2024-09-10T22:02:35.948479+0000","last_undegraded":"2024-09-10T22:02:35.948479+0000","last_fullsized":"2024-09-10T22:02:35.948479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:47.707679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948620+0000","last_change":"2024-09-10T22:00:34.502795+0000","last_active":"2024-09-10T22:02:35.948620+0000","last_peered":"2024-09-10T22:02:35.948620+0000","last_clean":"2024-09-10T22:02:35.948620+0000","last_became_active":"2024-09-10T22:00:34.501112+0000","last_became_peered":"2024-09-10T22:00:34.501112+0000","last_unstale":"2024-09-10T22:02:35.948620+0000","last_undegraded":"2024-09-10T22:02:35.948620+0000","last_fullsized":"2024-09-10T22:02:35.948620+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:45.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897276+0000","last_change":"2024-09-10T22:01:35.619765+0000","last_active":"2024-09-10T22:02:35.897276+0000","last_peered":"2024-09-10T22:02:35.897276+0000","last_clean":"2024-09-10T22:02:35.897276+0000","last_became_active":"2024-09-10T22:01:35.619407+0000","last_became_peered":"2024-09-10T22:01:35.619407+0000","last_unstale":"2024-09-10T22:02:35.897276+0000","last_undegraded":"2024-09-10T22:02:35.897276+0000","last_fullsized":"2024-09-10T22:02:35.897276+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:17.064585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726605+0000","last_change":"2024-09-10T22:00:30.818224+0000","last_active":"2024-09-10T22:02:35.726605+0000","last_peered":"2024-09-10T22:02:35.726605+0000","last_clean":"2024-09-10T22:02:35.726605+0000","last_became_active":"2024-09-10T22:00:30.817781+0000","last_became_peered":"2024-09-10T22:00:30.817781+0000","last_unstale":"2024-09-10T22:02:35.726605+0000","last_undegraded":"2024-09-10T22:02:35.726605+0000","last_fullsized":"2024-09-10T22:02:35.726605+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:47.097642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.950484+0000","last_change":"2024-09-10T22:02:35.949974+0000","last_active":"2024-09-10T22:02:35.950484+0000","last_peered":"2024-09-10T22:02:35.950484+0000","last_clean":"2024-09-10T22:02:35.950484+0000","last_became_active":"2024-09-10T22:02:35.949618+0000","last_became_peered":"2024-09-10T22:02:35.949618+0000","last_unstale":"2024-09-10T22:02:35.950484+0000","last_undegraded":"2024-09-10T22:02:35.950484+0000","last_fullsized":"2024-09-10T22:02:35.950484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:02.877650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897232+0000","last_change":"2024-09-10T22:00:43.712066+0000","last_active":"2024-09-10T22:02:35.897232+0000","last_peered":"2024-09-10T22:02:35.897232+0000","last_clean":"2024-09-10T22:02:35.897232+0000","last_became_active":"2024-09-10T22:00:43.711614+0000","last_became_peered":"2024-09-10T22:00:43.711614+0000","last_unstale":"2024-09-10T22:02:35.897232+0000","last_undegraded":"2024-09-10T22:02:35.897232+0000","last_fullsized":"2024-09-10T22:02:35.897232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:42.403367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863411+0000","last_change":"2024-09-10T22:00:34.501943+0000","last_active":"2024-09-10T22:02:35.863411+0000","last_peered":"2024-09-10T22:02:35.863411+0000","last_clean":"2024-09-10T22:02:35.863411+0000","last_became_active":"2024-09-10T22:00:34.501602+0000","last_became_peered":"2024-09-10T22:00:34.501602+0000","last_unstale":"2024-09-10T22:02:35.863411+0000","last_undegraded":"2024-09-10T22:02:35.863411+0000","last_fullsized":"2024-09-10T22:02:35.863411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:53.200974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726534+0000","last_change":"2024-09-10T22:01:35.200791+0000","last_active":"2024-09-10T22:02:35.726534+0000","last_peered":"2024-09-10T22:02:35.726534+0000","last_clean":"2024-09-10T22:02:35.726534+0000","last_became_active":"2024-09-10T22:01:35.196059+0000","last_became_peered":"2024-09-10T22:01:35.196059+0000","last_unstale":"2024-09-10T22:02:35.726534+0000","last_undegraded":"2024-09-10T22:02:35.726534+0000","last_fullsized":"2024-09-10T22:02:35.726534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:51.349216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.896947+0000","last_change":"2024-09-10T22:00:43.760567+0000","last_active":"2024-09-10T22:02:35.896947+0000","last_peered":"2024-09-10T22:02:35.896947+0000","last_clean":"2024-09-10T22:02:35.896947+0000","last_became_active":"2024-09-10T22:00:43.760064+0000","last_became_peered":"2024-09-10T22:00:43.760064+0000","last_unstale":"2024-09-10T22:02:35.896947+0000","last_undegraded":"2024-09-10T22:02:35.896947+0000","last_fullsized":"2024-09-10T22:02:35.896947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:55.469525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722689+0000","last_change":"2024-09-10T22:00:34.486225+0000","last_active":"2024-09-10T22:02:35.722689+0000","last_peered":"2024-09-10T22:02:35.722689+0000","last_clean":"2024-09-10T22:02:35.722689+0000","last_became_active":"2024-09-10T22:00:34.485896+0000","last_became_peered":"2024-09-10T22:00:34.485896+0000","last_unstale":"2024-09-10T22:02:35.722689+0000","last_undegraded":"2024-09-10T22:02:35.722689+0000","last_fullsized":"2024-09-10T22:02:35.722689+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:00.151927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723676+0000","last_change":"2024-09-10T22:00:30.817958+0000","last_active":"2024-09-10T22:02:35.723676+0000","last_peered":"2024-09-10T22:02:35.723676+0000","last_clean":"2024-09-10T22:02:35.723676+0000","last_became_active":"2024-09-10T22:00:30.817646+0000","last_became_peered":"2024-09-10T22:00:30.817646+0000","last_unstale":"2024-09-10T22:02:35.723676+0000","last_undegraded":"2024-09-10T22:02:35.723676+0000","last_fullsized":"2024-09-10T22:02:35.723676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:27.722184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897603+0000","last_change":"2024-09-10T22:00:43.716465+0000","last_active":"2024-09-10T22:02:35.897603+0000","last_peered":"2024-09-10T22:02:35.897603+0000","last_clean":"2024-09-10T22:02:35.897603+0000","last_became_active":"2024-09-10T22:00:43.716165+0000","last_became_peered":"2024-09-10T22:00:43.716165+0000","last_unstale":"2024-09-10T22:02:35.897603+0000","last_undegraded":"2024-09-10T22:02:35.897603+0000","last_fullsized":"2024-09-10T22:02:35.897603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:18.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948550+0000","last_change":"2024-09-10T22:00:34.493992+0000","last_active":"2024-09-10T22:02:35.948550+0000","last_peered":"2024-09-10T22:02:35.948550+0000","last_clean":"2024-09-10T22:02:35.948550+0000","last_became_active":"2024-09-10T22:00:34.493670+0000","last_became_peered":"2024-09-10T22:00:34.493670+0000","last_unstale":"2024-09-10T22:02:35.948550+0000","last_undegraded":"2024-09-10T22:02:35.948550+0000","last_fullsized":"2024-09-10T22:02:35.948550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:04.991819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722649+0000","last_change":"2024-09-10T22:00:30.819830+0000","last_active":"2024-09-10T22:02:35.722649+0000","last_peered":"2024-09-10T22:02:35.722649+0000","last_clean":"2024-09-10T22:02:35.722649+0000","last_became_active":"2024-09-10T22:00:30.819401+0000","last_became_peered":"2024-09-10T22:00:30.819401+0000","last_unstale":"2024-09-10T22:02:35.722649+0000","last_undegraded":"2024-09-10T22:02:35.722649+0000","last_fullsized":"2024-09-10T22:02:35.722649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:37.823107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724219+0000","last_change":"2024-09-10T22:00:43.718500+0000","last_active":"2024-09-10T22:02:35.724219+0000","last_peered":"2024-09-10T22:02:35.724219+0000","last_clean":"2024-09-10T22:02:35.724219+0000","last_became_active":"2024-09-10T22:00:43.717667+0000","last_became_peered":"2024-09-10T22:00:43.717667+0000","last_unstale":"2024-09-10T22:02:35.724219+0000","last_undegraded":"2024-09-10T22:02:35.724219+0000","last_fullsized":"2024-09-10T22:02:35.724219+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:23.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.725822+0000","last_change":"2024-09-10T22:00:34.494162+0000","last_active":"2024-09-10T22:02:35.725822+0000","last_peered":"2024-09-10T22:02:35.725822+0000","last_clean":"2024-09-10T22:02:35.725822+0000","last_became_active":"2024-09-10T22:00:34.493819+0000","last_became_peered":"2024-09-10T22:00:34.493819+0000","last_unstale":"2024-09-10T22:02:35.725822+0000","last_undegraded":"2024-09-10T22:02:35.725822+0000","last_fullsized":"2024-09-10T22:02:35.725822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:43.264363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948452+0000","last_change":"2024-09-10T22:00:31.081559+0000","last_active":"2024-09-10T22:02:35.948452+0000","last_peered":"2024-09-10T22:02:35.948452+0000","last_clean":"2024-09-10T22:02:35.948452+0000","last_became_active":"2024-09-10T22:00:31.081257+0000","last_became_peered":"2024-09-10T22:00:31.081257+0000","last_unstale":"2024-09-10T22:02:35.948452+0000","last_undegraded":"2024-09-10T22:02:35.948452+0000","last_fullsized":"2024-09-10T22:02:35.948452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:53.235765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897631+0000","last_change":"2024-09-10T22:01:35.445171+0000","last_active":"2024-09-10T22:02:35.897631+0000","last_peered":"2024-09-10T22:02:35.897631+0000","last_clean":"2024-09-10T22:02:35.897631+0000","last_became_active":"2024-09-10T22:01:35.444651+0000","last_became_peered":"2024-09-10T22:01:35.444651+0000","last_unstale":"2024-09-10T22:02:35.897631+0000","last_undegraded":"2024-09-10T22:02:35.897631+0000","last_fullsized":"2024-09-10T22:02:35.897631+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:36.457151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948055+0000","last_change":"2024-09-10T22:01:35.750982+0000","last_active":"2024-09-10T22:02:35.948055+0000","last_peered":"2024-09-10T22:02:35.948055+0000","last_clean":"2024-09-10T22:02:35.948055+0000","last_became_active":"2024-09-10T22:01:35.750572+0000","last_became_peered":"2024-09-10T22:01:35.750572+0000","last_unstale":"2024-09-10T22:02:35.948055+0000","last_undegraded":"2024-09-10T22:02:35.948055+0000","last_fullsized":"2024-09-10T22:02:35.948055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:36.596313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897938+0000","last_change":"2024-09-10T22:00:31.311864+0000","last_active":"2024-09-10T22:02:35.897938+0000","last_peered":"2024-09-10T22:02:35.897938+0000","last_clean":"2024-09-10T22:02:35.897938+0000","last_became_active":"2024-09-10T22:00:31.311611+0000","last_became_peered":"2024-09-10T22:00:31.311611+0000","last_unstale":"2024-09-10T22:02:35.897938+0000","last_undegraded":"2024-09-10T22:02:35.897938+0000","last_fullsized":"2024-09-10T22:02:35.897938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:27.969999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723426+0000","last_change":"2024-09-10T22:00:43.708023+0000","last_active":"2024-09-10T22:02:35.723426+0000","last_peered":"2024-09-10T22:02:35.723426+0000","last_clean":"2024-09-10T22:02:35.723426+0000","last_became_active":"2024-09-10T22:00:43.707086+0000","last_became_peered":"2024-09-10T22:00:43.707086+0000","last_unstale":"2024-09-10T22:02:35.723426+0000","last_undegraded":"2024-09-10T22:02:35.723426+0000","last_fullsized":"2024-09-10T22:02:35.723426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:15.585104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724039+0000","last_change":"2024-09-10T22:00:34.494104+0000","last_active":"2024-09-10T22:02:35.724039+0000","last_peered":"2024-09-10T22:02:35.724039+0000","last_clean":"2024-09-10T22:02:35.724039+0000","last_became_active":"2024-09-10T22:00:34.493404+0000","last_became_peered":"2024-09-10T22:00:34.493404+0000","last_unstale":"2024-09-10T22:02:35.724039+0000","last_undegraded":"2024-09-10T22:02:35.724039+0000","last_fullsized":"2024-09-10T22:02:35.724039+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:47.444100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'14","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723513+0000","last_change":"2024-09-10T22:00:30.819427+0000","last_active":"2024-09-10T22:02:35.723513+0000","last_peered":"2024-09-10T22:02:35.723513+0000","last_clean":"2024-09-10T22:02:35.723513+0000","last_became_active":"2024-09-10T22:00:30.819098+0000","last_became_peered":"2024-09-10T22:00:30.819098+0000","last_unstale":"2024-09-10T22:02:35.723513+0000","last_undegraded":"2024-09-10T22:02:35.723513+0000","last_fullsized":"2024-09-10T22:02:35.723513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:23.068155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722303+0000","last_change":"2024-09-10T22:00:43.760602+0000","last_active":"2024-09-10T22:02:35.722303+0000","last_peered":"2024-09-10T22:02:35.722303+0000","last_clean":"2024-09-10T22:02:35.722303+0000","last_became_active":"2024-09-10T22:00:43.760172+0000","last_became_peered":"2024-09-10T22:00:43.760172+0000","last_unstale":"2024-09-10T22:02:35.722303+0000","last_undegraded":"2024-09-10T22:02:35.722303+0000","last_fullsized":"2024-09-10T22:02:35.722303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:40.332101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894374+0000","last_change":"2024-09-10T22:00:34.501632+0000","last_active":"2024-09-10T22:02:35.894374+0000","last_peered":"2024-09-10T22:02:35.894374+0000","last_clean":"2024-09-10T22:02:35.894374+0000","last_became_active":"2024-09-10T22:00:34.500924+0000","last_became_peered":"2024-09-10T22:00:34.500924+0000","last_unstale":"2024-09-10T22:02:35.894374+0000","last_undegraded":"2024-09-10T22:02:35.894374+0000","last_fullsized":"2024-09-10T22:02:35.894374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:28.052570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726206+0000","last_change":"2024-09-10T22:00:31.081391+0000","last_active":"2024-09-10T22:02:35.726206+0000","last_peered":"2024-09-10T22:02:35.726206+0000","last_clean":"2024-09-10T22:02:35.726206+0000","last_became_active":"2024-09-10T22:00:31.081018+0000","last_became_peered":"2024-09-10T22:00:31.081018+0000","last_unstale":"2024-09-10T22:02:35.726206+0000","last_undegraded":"2024-09-10T22:02:35.726206+0000","last_fullsized":"2024-09-10T22:02:35.726206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:09.450785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948407+0000","last_change":"2024-09-10T22:00:43.711592+0000","last_active":"2024-09-10T22:02:35.948407+0000","last_peered":"2024-09-10T22:02:35.948407+0000","last_clean":"2024-09-10T22:02:35.948407+0000","last_became_active":"2024-09-10T22:00:43.711144+0000","last_became_peered":"2024-09-10T22:00:43.711144+0000","last_unstale":"2024-09-10T22:02:35.948407+0000","last_undegraded":"2024-09-10T22:02:35.948407+0000","last_fullsized":"2024-09-10T22:02:35.948407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:01.818907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871767+0000","last_change":"2024-09-10T22:00:34.494070+0000","last_active":"2024-09-10T22:02:35.871767+0000","last_peered":"2024-09-10T22:02:35.871767+0000","last_clean":"2024-09-10T22:02:35.871767+0000","last_became_active":"2024-09-10T22:00:34.493324+0000","last_became_peered":"2024-09-10T22:00:34.493324+0000","last_unstale":"2024-09-10T22:02:35.871767+0000","last_undegraded":"2024-09-10T22:02:35.871767+0000","last_fullsized":"2024-09-10T22:02:35.871767+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:29.969743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723888+0000","last_change":"2024-09-10T22:00:31.082439+0000","last_active":"2024-09-10T22:02:35.723888+0000","last_peered":"2024-09-10T22:02:35.723888+0000","last_clean":"2024-09-10T22:02:35.723888+0000","last_became_active":"2024-09-10T22:00:31.082025+0000","last_became_peered":"2024-09-10T22:00:31.082025+0000","last_unstale":"2024-09-10T22:02:35.723888+0000","last_undegraded":"2024-09-10T22:02:35.723888+0000","last_fullsized":"2024-09-10T22:02:35.723888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:21.933419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726032+0000","last_change":"2024-09-10T22:00:43.760997+0000","last_active":"2024-09-10T22:02:35.726032+0000","last_peered":"2024-09-10T22:02:35.726032+0000","last_clean":"2024-09-10T22:02:35.726032+0000","last_became_active":"2024-09-10T22:00:43.760124+0000","last_became_peered":"2024-09-10T22:00:43.760124+0000","last_unstale":"2024-09-10T22:02:35.726032+0000","last_undegraded":"2024-09-10T22:02:35.726032+0000","last_fullsized":"2024-09-10T22:02:35.726032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:29.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871775+0000","last_change":"2024-09-10T22:00:34.511730+0000","last_active":"2024-09-10T22:02:35.871775+0000","last_peered":"2024-09-10T22:02:35.871775+0000","last_clean":"2024-09-10T22:02:35.871775+0000","last_became_active":"2024-09-10T22:00:34.511440+0000","last_became_peered":"2024-09-10T22:00:34.511440+0000","last_unstale":"2024-09-10T22:02:35.871775+0000","last_undegraded":"2024-09-10T22:02:35.871775+0000","last_fullsized":"2024-09-10T22:02:35.871775+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:09.053835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":125,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:42.874342+0000","last_change":"2024-09-10T22:00:30.825999+0000","last_active":"2024-09-10T22:02:42.874342+0000","last_peered":"2024-09-10T22:02:42.874342+0000","last_clean":"2024-09-10T22:02:42.874342+0000","last_became_active":"2024-09-10T22:00:30.825334+0000","last_became_peered":"2024-09-10T22:00:30.825334+0000","last_unstale":"2024-09-10T22:02:42.874342+0000","last_undegraded":"2024-09-10T22:02:42.874342+0000","last_fullsized":"2024-09-10T22:02:42.874342+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:14.684222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894160+0000","last_change":"2024-09-10T22:00:43.710517+0000","last_active":"2024-09-10T22:02:35.894160+0000","last_peered":"2024-09-10T22:02:35.894160+0000","last_clean":"2024-09-10T22:02:35.894160+0000","last_became_active":"2024-09-10T22:00:43.709252+0000","last_became_peered":"2024-09-10T22:00:43.709252+0000","last_unstale":"2024-09-10T22:02:35.894160+0000","last_undegraded":"2024-09-10T22:02:35.894160+0000","last_fullsized":"2024-09-10T22:02:35.894160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:21.355152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.899302+0000","last_change":"2024-09-10T22:02:35.898878+0000","last_active":"2024-09-10T22:02:35.899302+0000","last_peered":"2024-09-10T22:02:35.899302+0000","last_clean":"2024-09-10T22:02:35.899302+0000","last_became_active":"2024-09-10T22:02:35.898531+0000","last_became_peered":"2024-09-10T22:02:35.898531+0000","last_unstale":"2024-09-10T22:02:35.899302+0000","last_undegraded":"2024-09-10T22:02:35.899302+0000","last_fullsized":"2024-09-10T22:02:35.899302+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:02.735208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:42.909175+0000","last_change":"2024-09-10T22:00:31.081590+0000","last_active":"2024-09-10T22:02:42.909175+0000","last_peered":"2024-09-10T22:02:42.909175+0000","last_clean":"2024-09-10T22:02:42.909175+0000","last_became_active":"2024-09-10T22:00:31.081250+0000","last_became_peered":"2024-09-10T22:00:31.081250+0000","last_unstale":"2024-09-10T22:02:42.909175+0000","last_undegraded":"2024-09-10T22:02:42.909175+0000","last_fullsized":"2024-09-10T22:02:42.909175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:12.872832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893728+0000","last_change":"2024-09-10T22:00:43.761351+0000","last_active":"2024-09-10T22:02:35.893728+0000","last_peered":"2024-09-10T22:02:35.893728+0000","last_clean":"2024-09-10T22:02:35.893728+0000","last_became_active":"2024-09-10T22:00:43.760903+0000","last_became_peered":"2024-09-10T22:00:43.760903+0000","last_unstale":"2024-09-10T22:02:35.893728+0000","last_undegraded":"2024-09-10T22:02:35.893728+0000","last_fullsized":"2024-09-10T22:02:35.893728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:14.113974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893929+0000","last_change":"2024-09-10T22:00:34.504671+0000","last_active":"2024-09-10T22:02:35.893929+0000","last_peered":"2024-09-10T22:02:35.893929+0000","last_clean":"2024-09-10T22:02:35.893929+0000","last_became_active":"2024-09-10T22:00:34.504368+0000","last_became_peered":"2024-09-10T22:00:34.504368+0000","last_unstale":"2024-09-10T22:02:35.893929+0000","last_undegraded":"2024-09-10T22:02:35.893929+0000","last_fullsized":"2024-09-10T22:02:35.893929+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:59.771589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863056+0000","last_change":"2024-09-10T22:00:30.820401+0000","last_active":"2024-09-10T22:02:35.863056+0000","last_peered":"2024-09-10T22:02:35.863056+0000","last_clean":"2024-09-10T22:02:35.863056+0000","last_became_active":"2024-09-10T22:00:30.820194+0000","last_became_peered":"2024-09-10T22:00:30.820194+0000","last_unstale":"2024-09-10T22:02:35.863056+0000","last_undegraded":"2024-09-10T22:02:35.863056+0000","last_fullsized":"2024-09-10T22:02:35.863056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:47.767102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727182+0000","last_change":"2024-09-10T22:00:43.762001+0000","last_active":"2024-09-10T22:02:35.727182+0000","last_peered":"2024-09-10T22:02:35.727182+0000","last_clean":"2024-09-10T22:02:35.727182+0000","last_became_active":"2024-09-10T22:00:43.761323+0000","last_became_peered":"2024-09-10T22:00:43.761323+0000","last_unstale":"2024-09-10T22:02:35.727182+0000","last_undegraded":"2024-09-10T22:02:35.727182+0000","last_fullsized":"2024-09-10T22:02:35.727182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:50.113545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722782+0000","last_change":"2024-09-10T22:00:34.498591+0000","last_active":"2024-09-10T22:02:35.722782+0000","last_peered":"2024-09-10T22:02:35.722782+0000","last_clean":"2024-09-10T22:02:35.722782+0000","last_became_active":"2024-09-10T22:00:34.496449+0000","last_became_peered":"2024-09-10T22:00:34.496449+0000","last_unstale":"2024-09-10T22:02:35.722782+0000","last_undegraded":"2024-09-10T22:02:35.722782+0000","last_fullsized":"2024-09-10T22:02:35.722782+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:54.815268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897905+0000","last_change":"2024-09-10T22:00:30.817201+0000","last_active":"2024-09-10T22:02:35.897905+0000","last_peered":"2024-09-10T22:02:35.897905+0000","last_clean":"2024-09-10T22:02:35.897905+0000","last_became_active":"2024-09-10T22:00:30.816792+0000","last_became_peered":"2024-09-10T22:00:30.816792+0000","last_unstale":"2024-09-10T22:02:35.897905+0000","last_undegraded":"2024-09-10T22:02:35.897905+0000","last_fullsized":"2024-09-10T22:02:35.897905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:48.239811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":120,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863107+0000","last_change":"2024-09-10T21:59:01.830452+0000","last_active":"2024-09-10T22:02:35.863107+0000","last_peered":"2024-09-10T22:02:35.863107+0000","last_clean":"2024-09-10T22:02:35.863107+0000","last_became_active":"2024-09-10T21:59:01.823744+0000","last_became_peered":"2024-09-10T21:59:01.823744+0000","last_unstale":"2024-09-10T22:02:35.863107+0000","last_undegraded":"2024-09-10T22:02:35.863107+0000","last_fullsized":"2024-09-10T22:02:35.863107+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_clean_scrub_stamp":"2024-09-10T21:54:48.621299+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:39.573257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863503+0000","last_change":"2024-09-10T22:00:43.713888+0000","last_active":"2024-09-10T22:02:35.863503+0000","last_peered":"2024-09-10T22:02:35.863503+0000","last_clean":"2024-09-10T22:02:35.863503+0000","last_became_active":"2024-09-10T22:00:43.713329+0000","last_became_peered":"2024-09-10T22:00:43.713329+0000","last_unstale":"2024-09-10T22:02:35.863503+0000","last_undegraded":"2024-09-10T22:02:35.863503+0000","last_fullsized":"2024-09-10T22:02:35.863503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:01.062089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863656+0000","last_change":"2024-09-10T22:00:34.502589+0000","last_active":"2024-09-10T22:02:35.863656+0000","last_peered":"2024-09-10T22:02:35.863656+0000","last_clean":"2024-09-10T22:02:35.863656+0000","last_became_active":"2024-09-10T22:00:34.502216+0000","last_became_peered":"2024-09-10T22:00:34.502216+0000","last_unstale":"2024-09-10T22:02:35.863656+0000","last_undegraded":"2024-09-10T22:02:35.863656+0000","last_fullsized":"2024-09-10T22:02:35.863656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:44.959375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:41.882598+0000","last_change":"2024-09-10T22:00:31.311807+0000","last_active":"2024-09-10T22:02:41.882598+0000","last_peered":"2024-09-10T22:02:41.882598+0000","last_clean":"2024-09-10T22:02:41.882598+0000","last_became_active":"2024-09-10T22:00:31.311424+0000","last_became_peered":"2024-09-10T22:00:31.311424+0000","last_unstale":"2024-09-10T22:02:41.882598+0000","last_undegraded":"2024-09-10T22:02:41.882598+0000","last_fullsized":"2024-09-10T22:02:41.882598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:09.754777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948128+0000","last_change":"2024-09-10T22:00:43.711882+0000","last_active":"2024-09-10T22:02:35.948128+0000","last_peered":"2024-09-10T22:02:35.948128+0000","last_clean":"2024-09-10T22:02:35.948128+0000","last_became_active":"2024-09-10T22:00:43.711408+0000","last_became_peered":"2024-09-10T22:00:43.711408+0000","last_unstale":"2024-09-10T22:02:35.948128+0000","last_undegraded":"2024-09-10T22:02:35.948128+0000","last_fullsized":"2024-09-10T22:02:35.948128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:31.155723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863336+0000","last_change":"2024-09-10T22:00:34.505653+0000","last_active":"2024-09-10T22:02:35.863336+0000","last_peered":"2024-09-10T22:02:35.863336+0000","last_clean":"2024-09-10T22:02:35.863336+0000","last_became_active":"2024-09-10T22:00:34.505317+0000","last_became_peered":"2024-09-10T22:00:34.505317+0000","last_unstale":"2024-09-10T22:02:35.863336+0000","last_undegraded":"2024-09-10T22:02:35.863336+0000","last_fullsized":"2024-09-10T22:02:35.863336+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:35.050892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727910+0000","last_change":"2024-09-10T22:00:30.817431+0000","last_active":"2024-09-10T22:02:35.727910+0000","last_peered":"2024-09-10T22:02:35.727910+0000","last_clean":"2024-09-10T22:02:35.727910+0000","last_became_active":"2024-09-10T22:00:30.816952+0000","last_became_peered":"2024-09-10T22:00:30.816952+0000","last_unstale":"2024-09-10T22:02:35.727910+0000","last_undegraded":"2024-09-10T22:02:35.727910+0000","last_fullsized":"2024-09-10T22:02:35.727910+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:37.881641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947445+0000","last_change":"2024-09-10T22:00:43.720627+0000","last_active":"2024-09-10T22:02:35.947445+0000","last_peered":"2024-09-10T22:02:35.947445+0000","last_clean":"2024-09-10T22:02:35.947445+0000","last_became_active":"2024-09-10T22:00:43.720235+0000","last_became_peered":"2024-09-10T22:00:43.720235+0000","last_unstale":"2024-09-10T22:02:35.947445+0000","last_undegraded":"2024-09-10T22:02:35.947445+0000","last_fullsized":"2024-09-10T22:02:35.947445+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:50.549536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863837+0000","last_change":"2024-09-10T22:00:34.501649+0000","last_active":"2024-09-10T22:02:35.863837+0000","last_peered":"2024-09-10T22:02:35.863837+0000","last_clean":"2024-09-10T22:02:35.863837+0000","last_became_active":"2024-09-10T22:00:34.498882+0000","last_became_peered":"2024-09-10T22:00:34.498882+0000","last_unstale":"2024-09-10T22:02:35.863837+0000","last_undegraded":"2024-09-10T22:02:35.863837+0000","last_fullsized":"2024-09-10T22:02:35.863837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894461+0000","last_change":"2024-09-10T22:00:30.821943+0000","last_active":"2024-09-10T22:02:35.894461+0000","last_peered":"2024-09-10T22:02:35.894461+0000","last_clean":"2024-09-10T22:02:35.894461+0000","last_became_active":"2024-09-10T22:00:30.821372+0000","last_became_peered":"2024-09-10T22:00:30.821372+0000","last_unstale":"2024-09-10T22:02:35.894461+0000","last_undegraded":"2024-09-10T22:02:35.894461+0000","last_fullsized":"2024-09-10T22:02:35.894461+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:04.360383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726948+0000","last_change":"2024-09-10T22:00:43.710045+0000","last_active":"2024-09-10T22:02:35.726948+0000","last_peered":"2024-09-10T22:02:35.726948+0000","last_clean":"2024-09-10T22:02:35.726948+0000","last_became_active":"2024-09-10T22:00:43.709416+0000","last_became_peered":"2024-09-10T22:00:43.709416+0000","last_unstale":"2024-09-10T22:02:35.726948+0000","last_undegraded":"2024-09-10T22:02:35.726948+0000","last_fullsized":"2024-09-10T22:02:35.726948+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:56.005549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948742+0000","last_change":"2024-09-10T22:00:34.496163+0000","last_active":"2024-09-10T22:02:35.948742+0000","last_peered":"2024-09-10T22:02:35.948742+0000","last_clean":"2024-09-10T22:02:35.948742+0000","last_became_active":"2024-09-10T22:00:34.495814+0000","last_became_peered":"2024-09-10T22:00:34.495814+0000","last_unstale":"2024-09-10T22:02:35.948742+0000","last_undegraded":"2024-09-10T22:02:35.948742+0000","last_fullsized":"2024-09-10T22:02:35.948742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:05.398154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726371+0000","last_change":"2024-09-10T22:00:30.812702+0000","last_active":"2024-09-10T22:02:35.726371+0000","last_peered":"2024-09-10T22:02:35.726371+0000","last_clean":"2024-09-10T22:02:35.726371+0000","last_became_active":"2024-09-10T22:00:30.812369+0000","last_became_peered":"2024-09-10T22:00:30.812369+0000","last_unstale":"2024-09-10T22:02:35.726371+0000","last_undegraded":"2024-09-10T22:02:35.726371+0000","last_fullsized":"2024-09-10T22:02:35.726371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:52.332876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727160+0000","last_change":"2024-09-10T22:00:43.715573+0000","last_active":"2024-09-10T22:02:35.727160+0000","last_peered":"2024-09-10T22:02:35.727160+0000","last_clean":"2024-09-10T22:02:35.727160+0000","last_became_active":"2024-09-10T22:00:43.714475+0000","last_became_peered":"2024-09-10T22:00:43.714475+0000","last_unstale":"2024-09-10T22:02:35.727160+0000","last_undegraded":"2024-09-10T22:02:35.727160+0000","last_fullsized":"2024-09-10T22:02:35.727160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:55.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.949841+0000","last_change":"2024-09-10T22:02:35.949769+0000","last_active":"2024-09-10T22:02:35.949841+0000","last_peered":"2024-09-10T22:02:35.949841+0000","last_clean":"2024-09-10T22:02:35.949841+0000","last_became_active":"2024-09-10T22:02:35.949440+0000","last_became_peered":"2024-09-10T22:02:35.949440+0000","last_unstale":"2024-09-10T22:02:35.949841+0000","last_undegraded":"2024-09-10T22:02:35.949841+0000","last_fullsized":"2024-09-10T22:02:35.949841+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:03.639412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948309+0000","last_change":"2024-09-10T22:01:35.903381+0000","last_active":"2024-09-10T22:02:35.948309+0000","last_peered":"2024-09-10T22:02:35.948309+0000","last_clean":"2024-09-10T22:02:35.948309+0000","last_became_active":"2024-09-10T22:01:35.878262+0000","last_became_peered":"2024-09-10T22:01:35.878262+0000","last_unstale":"2024-09-10T22:02:35.948309+0000","last_undegraded":"2024-09-10T22:02:35.948309+0000","last_fullsized":"2024-09-10T22:02:35.948309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:45.419449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893719+0000","last_change":"2024-09-10T22:00:43.720728+0000","last_active":"2024-09-10T22:02:35.893719+0000","last_peered":"2024-09-10T22:02:35.893719+0000","last_clean":"2024-09-10T22:02:35.893719+0000","last_became_active":"2024-09-10T22:00:43.720305+0000","last_became_peered":"2024-09-10T22:00:43.720305+0000","last_unstale":"2024-09-10T22:02:35.893719+0000","last_undegraded":"2024-09-10T22:02:35.893719+0000","last_fullsized":"2024-09-10T22:02:35.893719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:54.986822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.862895+0000","last_change":"2024-09-10T22:00:34.501571+0000","last_active":"2024-09-10T22:02:35.862895+0000","last_peered":"2024-09-10T22:02:35.862895+0000","last_clean":"2024-09-10T22:02:35.862895+0000","last_became_active":"2024-09-10T22:00:34.501211+0000","last_became_peered":"2024-09-10T22:00:34.501211+0000","last_unstale":"2024-09-10T22:02:35.862895+0000","last_undegraded":"2024-09-10T22:02:35.862895+0000","last_fullsized":"2024-09-10T22:02:35.862895+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:05.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":124,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:40.906640+0000","last_change":"2024-09-10T22:00:30.824296+0000","last_active":"2024-09-10T22:02:40.906640+0000","last_peered":"2024-09-10T22:02:40.906640+0000","last_clean":"2024-09-10T22:02:40.906640+0000","last_became_active":"2024-09-10T22:00:30.823886+0000","last_became_peered":"2024-09-10T22:00:30.823886+0000","last_unstale":"2024-09-10T22:02:40.906640+0000","last_undegraded":"2024-09-10T22:02:40.906640+0000","last_fullsized":"2024-09-10T22:02:40.906640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:26.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724313+0000","last_change":"2024-09-10T22:00:43.717580+0000","last_active":"2024-09-10T22:02:35.724313+0000","last_peered":"2024-09-10T22:02:35.724313+0000","last_clean":"2024-09-10T22:02:35.724313+0000","last_became_active":"2024-09-10T22:00:43.708377+0000","last_became_peered":"2024-09-10T22:00:43.708377+0000","last_unstale":"2024-09-10T22:02:35.724313+0000","last_undegraded":"2024-09-10T22:02:35.724313+0000","last_fullsized":"2024-09-10T22:02:35.724313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:54.682292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863307+0000","last_change":"2024-09-10T22:00:34.514348+0000","last_active":"2024-09-10T22:02:35.863307+0000","last_peered":"2024-09-10T22:02:35.863307+0000","last_clean":"2024-09-10T22:02:35.863307+0000","last_became_active":"2024-09-10T22:00:34.513797+0000","last_became_peered":"2024-09-10T22:00:34.513797+0000","last_unstale":"2024-09-10T22:02:35.863307+0000","last_undegraded":"2024-09-10T22:02:35.863307+0000","last_fullsized":"2024-09-10T22:02:35.863307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:12.797643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728017+0000","last_change":"2024-09-10T22:00:30.826486+0000","last_active":"2024-09-10T22:02:35.728017+0000","last_peered":"2024-09-10T22:02:35.728017+0000","last_clean":"2024-09-10T22:02:35.728017+0000","last_became_active":"2024-09-10T22:00:30.826111+0000","last_became_peered":"2024-09-10T22:00:30.826111+0000","last_unstale":"2024-09-10T22:02:35.728017+0000","last_undegraded":"2024-09-10T22:02:35.728017+0000","last_fullsized":"2024-09-10T22:02:35.728017+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:45.609925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948404+0000","last_change":"2024-09-10T22:00:43.709432+0000","last_active":"2024-09-10T22:02:35.948404+0000","last_peered":"2024-09-10T22:02:35.948404+0000","last_clean":"2024-09-10T22:02:35.948404+0000","last_became_active":"2024-09-10T22:00:43.708627+0000","last_became_peered":"2024-09-10T22:00:43.708627+0000","last_unstale":"2024-09-10T22:02:35.948404+0000","last_undegraded":"2024-09-10T22:02:35.948404+0000","last_fullsized":"2024-09-10T22:02:35.948404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:12.799976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894017+0000","last_change":"2024-09-10T22:00:34.499203+0000","last_active":"2024-09-10T22:02:35.894017+0000","last_peered":"2024-09-10T22:02:35.894017+0000","last_clean":"2024-09-10T22:02:35.894017+0000","last_became_active":"2024-09-10T22:00:34.497877+0000","last_became_peered":"2024-09-10T22:00:34.497877+0000","last_unstale":"2024-09-10T22:02:35.894017+0000","last_undegraded":"2024-09-10T22:02:35.894017+0000","last_fullsized":"2024-09-10T22:02:35.894017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:41.897603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723917+0000","last_change":"2024-09-10T22:00:30.820852+0000","last_active":"2024-09-10T22:02:35.723917+0000","last_peered":"2024-09-10T22:02:35.723917+0000","last_clean":"2024-09-10T22:02:35.723917+0000","last_became_active":"2024-09-10T22:00:30.820023+0000","last_became_peered":"2024-09-10T22:00:30.820023+0000","last_unstale":"2024-09-10T22:02:35.723917+0000","last_undegraded":"2024-09-10T22:02:35.723917+0000","last_fullsized":"2024-09-10T22:02:35.723917+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:54.245142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727756+0000","last_change":"2024-09-10T22:00:43.716323+0000","last_active":"2024-09-10T22:02:35.727756+0000","last_peered":"2024-09-10T22:02:35.727756+0000","last_clean":"2024-09-10T22:02:35.727756+0000","last_became_active":"2024-09-10T22:00:43.715827+0000","last_became_peered":"2024-09-10T22:00:43.715827+0000","last_unstale":"2024-09-10T22:02:35.727756+0000","last_undegraded":"2024-09-10T22:02:35.727756+0000","last_fullsized":"2024-09-10T22:02:35.727756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:10.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726443+0000","last_change":"2024-09-10T22:00:34.500114+0000","last_active":"2024-09-10T22:02:35.726443+0000","last_peered":"2024-09-10T22:02:35.726443+0000","last_clean":"2024-09-10T22:02:35.726443+0000","last_became_active":"2024-09-10T22:00:34.499334+0000","last_became_peered":"2024-09-10T22:00:34.499334+0000","last_unstale":"2024-09-10T22:02:35.726443+0000","last_undegraded":"2024-09-10T22:02:35.726443+0000","last_fullsized":"2024-09-10T22:02:35.726443+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:11.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":229,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:43.885670+0000","last_change":"2024-09-10T22:00:30.822561+0000","last_active":"2024-09-10T22:02:43.885670+0000","last_peered":"2024-09-10T22:02:43.885670+0000","last_clean":"2024-09-10T22:02:43.885670+0000","last_became_active":"2024-09-10T22:00:30.822201+0000","last_became_peered":"2024-09-10T22:00:30.822201+0000","last_unstale":"2024-09-10T22:02:43.885670+0000","last_undegraded":"2024-09-10T22:02:43.885670+0000","last_fullsized":"2024-09-10T22:02:43.885670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:41.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62657,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":152,"num_read_kb":137,"num_write":36,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871420+0000","last_change":"2024-09-10T22:01:35.444988+0000","last_active":"2024-09-10T22:02:35.871420+0000","last_peered":"2024-09-10T22:02:35.871420+0000","last_clean":"2024-09-10T22:02:35.871420+0000","last_became_active":"2024-09-10T22:01:35.444498+0000","last_became_peered":"2024-09-10T22:01:35.444498+0000","last_unstale":"2024-09-10T22:02:35.871420+0000","last_undegraded":"2024-09-10T22:02:35.871420+0000","last_fullsized":"2024-09-10T22:02:35.871420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:35.017660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948621+0000","last_change":"2024-09-10T22:00:34.503234+0000","last_active":"2024-09-10T22:02:35.948621+0000","last_peered":"2024-09-10T22:02:35.948621+0000","last_clean":"2024-09-10T22:02:35.948621+0000","last_became_active":"2024-09-10T22:00:34.501501+0000","last_became_peered":"2024-09-10T22:00:34.501501+0000","last_unstale":"2024-09-10T22:02:35.948621+0000","last_undegraded":"2024-09-10T22:02:35.948621+0000","last_fullsized":"2024-09-10T22:02:35.948621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:23.772491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728049+0000","last_change":"2024-09-10T22:00:31.082131+0000","last_active":"2024-09-10T22:02:35.728049+0000","last_peered":"2024-09-10T22:02:35.728049+0000","last_clean":"2024-09-10T22:02:35.728049+0000","last_became_active":"2024-09-10T22:00:31.081405+0000","last_became_peered":"2024-09-10T22:00:31.081405+0000","last_unstale":"2024-09-10T22:02:35.728049+0000","last_undegraded":"2024-09-10T22:02:35.728049+0000","last_fullsized":"2024-09-10T22:02:35.728049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:02.400004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897364+0000","last_change":"2024-09-10T22:00:43.705732+0000","last_active":"2024-09-10T22:02:35.897364+0000","last_peered":"2024-09-10T22:02:35.897364+0000","last_clean":"2024-09-10T22:02:35.897364+0000","last_became_active":"2024-09-10T22:00:43.705274+0000","last_became_peered":"2024-09-10T22:00:43.705274+0000","last_unstale":"2024-09-10T22:02:35.897364+0000","last_undegraded":"2024-09-10T22:02:35.897364+0000","last_fullsized":"2024-09-10T22:02:35.897364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:54.707636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948169+0000","last_change":"2024-09-10T22:00:34.499585+0000","last_active":"2024-09-10T22:02:35.948169+0000","last_peered":"2024-09-10T22:02:35.948169+0000","last_clean":"2024-09-10T22:02:35.948169+0000","last_became_active":"2024-09-10T22:00:34.499261+0000","last_became_peered":"2024-09-10T22:00:34.499261+0000","last_unstale":"2024-09-10T22:02:35.948169+0000","last_undegraded":"2024-09-10T22:02:35.948169+0000","last_fullsized":"2024-09-10T22:02:35.948169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:41.156093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863634+0000","last_change":"2024-09-10T22:00:30.823322+0000","last_active":"2024-09-10T22:02:35.863634+0000","last_peered":"2024-09-10T22:02:35.863634+0000","last_clean":"2024-09-10T22:02:35.863634+0000","last_became_active":"2024-09-10T22:00:30.822987+0000","last_became_peered":"2024-09-10T22:00:30.822987+0000","last_unstale":"2024-09-10T22:02:35.863634+0000","last_undegraded":"2024-09-10T22:02:35.863634+0000","last_fullsized":"2024-09-10T22:02:35.863634+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:33.605627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723207+0000","last_change":"2024-09-10T22:00:43.708746+0000","last_active":"2024-09-10T22:02:35.723207+0000","last_peered":"2024-09-10T22:02:35.723207+0000","last_clean":"2024-09-10T22:02:35.723207+0000","last_became_active":"2024-09-10T22:00:43.708283+0000","last_became_peered":"2024-09-10T22:00:43.708283+0000","last_unstale":"2024-09-10T22:02:35.723207+0000","last_undegraded":"2024-09-10T22:02:35.723207+0000","last_fullsized":"2024-09-10T22:02:35.723207+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:30.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948916+0000","last_change":"2024-09-10T22:00:34.501472+0000","last_active":"2024-09-10T22:02:35.948916+0000","last_peered":"2024-09-10T22:02:35.948916+0000","last_clean":"2024-09-10T22:02:35.948916+0000","last_became_active":"2024-09-10T22:00:34.501131+0000","last_became_peered":"2024-09-10T22:00:34.501131+0000","last_unstale":"2024-09-10T22:02:35.948916+0000","last_undegraded":"2024-09-10T22:02:35.948916+0000","last_fullsized":"2024-09-10T22:02:35.948916+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:11.281413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":191,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:43.885690+0000","last_change":"2024-09-10T22:00:30.828335+0000","last_active":"2024-09-10T22:02:43.885690+0000","last_peered":"2024-09-10T22:02:43.885690+0000","last_clean":"2024-09-10T22:02:43.885690+0000","last_became_active":"2024-09-10T22:00:30.828028+0000","last_became_peered":"2024-09-10T22:00:30.828028+0000","last_unstale":"2024-09-10T22:02:43.885690+0000","last_undegraded":"2024-09-10T22:02:43.885690+0000","last_fullsized":"2024-09-10T22:02:43.885690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:56.399002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":144,"num_read_kb":144,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722425+0000","last_change":"2024-09-10T22:00:43.710086+0000","last_active":"2024-09-10T22:02:35.722425+0000","last_peered":"2024-09-10T22:02:35.722425+0000","last_clean":"2024-09-10T22:02:35.722425+0000","last_became_active":"2024-09-10T22:00:43.709518+0000","last_became_peered":"2024-09-10T22:00:43.709518+0000","last_unstale":"2024-09-10T22:02:35.722425+0000","last_undegraded":"2024-09-10T22:02:35.722425+0000","last_fullsized":"2024-09-10T22:02:35.722425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:07.636887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947965+0000","last_change":"2024-09-10T22:00:43.711620+0000","last_active":"2024-09-10T22:02:35.947965+0000","last_peered":"2024-09-10T22:02:35.947965+0000","last_clean":"2024-09-10T22:02:35.947965+0000","last_became_active":"2024-09-10T22:00:43.711224+0000","last_became_peered":"2024-09-10T22:00:43.711224+0000","last_unstale":"2024-09-10T22:02:35.947965+0000","last_undegraded":"2024-09-10T22:02:35.947965+0000","last_fullsized":"2024-09-10T22:02:35.947965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:43.860286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722924+0000","last_change":"2024-09-10T22:00:34.494408+0000","last_active":"2024-09-10T22:02:35.722924+0000","last_peered":"2024-09-10T22:02:35.722924+0000","last_clean":"2024-09-10T22:02:35.722924+0000","last_became_active":"2024-09-10T22:00:34.494066+0000","last_became_peered":"2024-09-10T22:00:34.494066+0000","last_unstale":"2024-09-10T22:02:35.722924+0000","last_undegraded":"2024-09-10T22:02:35.722924+0000","last_fullsized":"2024-09-10T22:02:35.722924+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:49.565755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727659+0000","last_change":"2024-09-10T22:00:30.821842+0000","last_active":"2024-09-10T22:02:35.727659+0000","last_peered":"2024-09-10T22:02:35.727659+0000","last_clean":"2024-09-10T22:02:35.727659+0000","last_became_active":"2024-09-10T22:00:30.821499+0000","last_became_peered":"2024-09-10T22:00:30.821499+0000","last_unstale":"2024-09-10T22:02:35.727659+0000","last_undegraded":"2024-09-10T22:02:35.727659+0000","last_fullsized":"2024-09-10T22:02:35.727659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:19.125052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948671+0000","last_change":"2024-09-10T22:00:43.706689+0000","last_active":"2024-09-10T22:02:35.948671+0000","last_peered":"2024-09-10T22:02:35.948671+0000","last_clean":"2024-09-10T22:02:35.948671+0000","last_became_active":"2024-09-10T22:00:43.705954+0000","last_became_peered":"2024-09-10T22:00:43.705954+0000","last_unstale":"2024-09-10T22:02:35.948671+0000","last_undegraded":"2024-09-10T22:02:35.948671+0000","last_fullsized":"2024-09-10T22:02:35.948671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:30.800710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722387+0000","last_change":"2024-09-10T22:00:34.490853+0000","last_active":"2024-09-10T22:02:35.722387+0000","last_peered":"2024-09-10T22:02:35.722387+0000","last_clean":"2024-09-10T22:02:35.722387+0000","last_became_active":"2024-09-10T22:00:34.489908+0000","last_became_peered":"2024-09-10T22:00:34.489908+0000","last_unstale":"2024-09-10T22:02:35.722387+0000","last_undegraded":"2024-09-10T22:02:35.722387+0000","last_fullsized":"2024-09-10T22:02:35.722387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:57.105004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893871+0000","last_change":"2024-09-10T22:00:30.816497+0000","last_active":"2024-09-10T22:02:35.893871+0000","last_peered":"2024-09-10T22:02:35.893871+0000","last_clean":"2024-09-10T22:02:35.893871+0000","last_became_active":"2024-09-10T22:00:30.816181+0000","last_became_peered":"2024-09-10T22:00:30.816181+0000","last_unstale":"2024-09-10T22:02:35.893871+0000","last_undegraded":"2024-09-10T22:02:35.893871+0000","last_fullsized":"2024-09-10T22:02:35.893871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:12.779113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871729+0000","last_change":"2024-09-10T22:00:30.821402+0000","last_active":"2024-09-10T22:02:35.871729+0000","last_peered":"2024-09-10T22:02:35.871729+0000","last_clean":"2024-09-10T22:02:35.871729+0000","last_became_active":"2024-09-10T22:00:30.821105+0000","last_became_peered":"2024-09-10T22:00:30.821105+0000","last_unstale":"2024-09-10T22:02:35.871729+0000","last_undegraded":"2024-09-10T22:02:35.871729+0000","last_fullsized":"2024-09-10T22:02:35.871729+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:36.090579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727376+0000","last_change":"2024-09-10T22:00:34.511911+0000","last_active":"2024-09-10T22:02:35.727376+0000","last_peered":"2024-09-10T22:02:35.727376+0000","last_clean":"2024-09-10T22:02:35.727376+0000","last_became_active":"2024-09-10T22:00:34.511545+0000","last_became_peered":"2024-09-10T22:00:34.511545+0000","last_unstale":"2024-09-10T22:02:35.727376+0000","last_undegraded":"2024-09-10T22:02:35.727376+0000","last_fullsized":"2024-09-10T22:02:35.727376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:12.484733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723478+0000","last_change":"2024-09-10T22:01:35.750458+0000","last_active":"2024-09-10T22:02:35.723478+0000","last_peered":"2024-09-10T22:02:35.723478+0000","last_clean":"2024-09-10T22:02:35.723478+0000","last_became_active":"2024-09-10T22:01:35.749946+0000","last_became_peered":"2024-09-10T22:01:35.749946+0000","last_unstale":"2024-09-10T22:02:35.723478+0000","last_undegraded":"2024-09-10T22:02:35.723478+0000","last_fullsized":"2024-09-10T22:02:35.723478+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.362221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":123,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:40.881186+0000","last_change":"2024-09-10T22:00:30.826427+0000","last_active":"2024-09-10T22:02:40.881186+0000","last_peered":"2024-09-10T22:02:40.881186+0000","last_clean":"2024-09-10T22:02:40.881186+0000","last_became_active":"2024-09-10T22:00:30.826038+0000","last_became_peered":"2024-09-10T22:00:30.826038+0000","last_unstale":"2024-09-10T22:02:40.881186+0000","last_undegraded":"2024-09-10T22:02:40.881186+0000","last_fullsized":"2024-09-10T22:02:40.881186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:45.075208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726178+0000","last_change":"2024-09-10T22:00:34.494479+0000","last_active":"2024-09-10T22:02:35.726178+0000","last_peered":"2024-09-10T22:02:35.726178+0000","last_clean":"2024-09-10T22:02:35.726178+0000","last_became_active":"2024-09-10T22:00:34.494087+0000","last_became_peered":"2024-09-10T22:00:34.494087+0000","last_unstale":"2024-09-10T22:02:35.726178+0000","last_undegraded":"2024-09-10T22:02:35.726178+0000","last_fullsized":"2024-09-10T22:02:35.726178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:28.548999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723543+0000","last_change":"2024-09-10T22:00:43.714568+0000","last_active":"2024-09-10T22:02:35.723543+0000","last_peered":"2024-09-10T22:02:35.723543+0000","last_clean":"2024-09-10T22:02:35.723543+0000","last_became_active":"2024-09-10T22:00:43.712923+0000","last_became_peered":"2024-09-10T22:00:43.712923+0000","last_unstale":"2024-09-10T22:02:35.723543+0000","last_undegraded":"2024-09-10T22:02:35.723543+0000","last_fullsized":"2024-09-10T22:02:35.723543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:27.945799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948939+0000","last_change":"2024-09-10T22:00:34.496942+0000","last_active":"2024-09-10T22:02:35.948939+0000","last_peered":"2024-09-10T22:02:35.948939+0000","last_clean":"2024-09-10T22:02:35.948939+0000","last_became_active":"2024-09-10T22:00:34.496475+0000","last_became_peered":"2024-09-10T22:00:34.496475+0000","last_unstale":"2024-09-10T22:02:35.948939+0000","last_undegraded":"2024-09-10T22:02:35.948939+0000","last_fullsized":"2024-09-10T22:02:35.948939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:33.277346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723562+0000","last_change":"2024-09-10T22:00:34.506571+0000","last_active":"2024-09-10T22:02:35.723562+0000","last_peered":"2024-09-10T22:02:35.723562+0000","last_clean":"2024-09-10T22:02:35.723562+0000","last_became_active":"2024-09-10T22:00:34.505718+0000","last_became_peered":"2024-09-10T22:00:34.505718+0000","last_unstale":"2024-09-10T22:02:35.723562+0000","last_undegraded":"2024-09-10T22:02:35.723562+0000","last_fullsized":"2024-09-10T22:02:35.723562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:10.111415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723877+0000","last_change":"2024-09-10T22:00:34.502030+0000","last_active":"2024-09-10T22:02:35.723877+0000","last_peered":"2024-09-10T22:02:35.723877+0000","last_clean":"2024-09-10T22:02:35.723877+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:02:35.723877+0000","last_undegraded":"2024-09-10T22:02:35.723877+0000","last_fullsized":"2024-09-10T22:02:35.723877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:09.532523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947883+0000","last_change":"2024-09-10T22:00:34.513630+0000","last_active":"2024-09-10T22:02:35.947883+0000","last_peered":"2024-09-10T22:02:35.947883+0000","last_clean":"2024-09-10T22:02:35.947883+0000","last_became_active":"2024-09-10T22:00:34.513348+0000","last_became_peered":"2024-09-10T22:00:34.513348+0000","last_unstale":"2024-09-10T22:02:35.947883+0000","last_undegraded":"2024-09-10T22:02:35.947883+0000","last_fullsized":"2024-09-10T22:02:35.947883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:53.942004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894348+0000","last_change":"2024-09-10T22:00:34.508697+0000","last_active":"2024-09-10T22:02:35.894348+0000","last_peered":"2024-09-10T22:02:35.894348+0000","last_clean":"2024-09-10T22:02:35.894348+0000","last_became_active":"2024-09-10T22:00:34.508371+0000","last_became_peered":"2024-09-10T22:00:34.508371+0000","last_unstale":"2024-09-10T22:02:35.894348+0000","last_undegraded":"2024-09-10T22:02:35.894348+0000","last_fullsized":"2024-09-10T22:02:35.894348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:32.843517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948795+0000","last_change":"2024-09-10T22:00:34.496021+0000","last_active":"2024-09-10T22:02:35.948795+0000","last_peered":"2024-09-10T22:02:35.948795+0000","last_clean":"2024-09-10T22:02:35.948795+0000","last_became_active":"2024-09-10T22:00:34.495663+0000","last_became_peered":"2024-09-10T22:00:34.495663+0000","last_unstale":"2024-09-10T22:02:35.948795+0000","last_undegraded":"2024-09-10T22:02:35.948795+0000","last_fullsized":"2024-09-10T22:02:35.948795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:16.512182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724067+0000","last_change":"2024-09-10T22:00:34.506589+0000","last_active":"2024-09-10T22:02:35.724067+0000","last_peered":"2024-09-10T22:02:35.724067+0000","last_clean":"2024-09-10T22:02:35.724067+0000","last_became_active":"2024-09-10T22:00:34.506214+0000","last_became_peered":"2024-09-10T22:00:34.506214+0000","last_unstale":"2024-09-10T22:02:35.724067+0000","last_undegraded":"2024-09-10T22:02:35.724067+0000","last_fullsized":"2024-09-10T22:02:35.724067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:47.072458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723960+0000","last_change":"2024-09-10T22:00:34.503471+0000","last_active":"2024-09-10T22:02:35.723960+0000","last_peered":"2024-09-10T22:02:35.723960+0000","last_clean":"2024-09-10T22:02:35.723960+0000","last_became_active":"2024-09-10T22:00:34.501145+0000","last_became_peered":"2024-09-10T22:00:34.501145+0000","last_unstale":"2024-09-10T22:02:35.723960+0000","last_undegraded":"2024-09-10T22:02:35.723960+0000","last_fullsized":"2024-09-10T22:02:35.723960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:24.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727219+0000","last_change":"2024-09-10T22:00:34.490177+0000","last_active":"2024-09-10T22:02:35.727219+0000","last_peered":"2024-09-10T22:02:35.727219+0000","last_clean":"2024-09-10T22:02:35.727219+0000","last_became_active":"2024-09-10T22:00:34.488428+0000","last_became_peered":"2024-09-10T22:00:34.488428+0000","last_unstale":"2024-09-10T22:02:35.727219+0000","last_undegraded":"2024-09-10T22:02:35.727219+0000","last_fullsized":"2024-09-10T22:02:35.727219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:21.111826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722506+0000","last_change":"2024-09-10T22:00:34.499174+0000","last_active":"2024-09-10T22:02:35.722506+0000","last_peered":"2024-09-10T22:02:35.722506+0000","last_clean":"2024-09-10T22:02:35.722506+0000","last_became_active":"2024-09-10T22:00:34.498824+0000","last_became_peered":"2024-09-10T22:00:34.498824+0000","last_unstale":"2024-09-10T22:02:35.722506+0000","last_undegraded":"2024-09-10T22:02:35.722506+0000","last_fullsized":"2024-09-10T22:02:35.722506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:15.946270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948370+0000","last_change":"2024-09-10T22:00:34.505748+0000","last_active":"2024-09-10T22:02:35.948370+0000","last_peered":"2024-09-10T22:02:35.948370+0000","last_clean":"2024-09-10T22:02:35.948370+0000","last_became_active":"2024-09-10T22:00:34.500441+0000","last_became_peered":"2024-09-10T22:00:34.500441+0000","last_unstale":"2024-09-10T22:02:35.948370+0000","last_undegraded":"2024-09-10T22:02:35.948370+0000","last_fullsized":"2024-09-10T22:02:35.948370+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:22.177566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894288+0000","last_change":"2024-09-10T22:00:34.502319+0000","last_active":"2024-09-10T22:02:35.894288+0000","last_peered":"2024-09-10T22:02:35.894288+0000","last_clean":"2024-09-10T22:02:35.894288+0000","last_became_active":"2024-09-10T22:00:34.501427+0000","last_became_peered":"2024-09-10T22:00:34.501427+0000","last_unstale":"2024-09-10T22:02:35.894288+0000","last_undegraded":"2024-09-10T22:02:35.894288+0000","last_fullsized":"2024-09-10T22:02:35.894288+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:18.463319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948529+0000","last_change":"2024-09-10T22:00:34.497571+0000","last_active":"2024-09-10T22:02:35.948529+0000","last_peered":"2024-09-10T22:02:35.948529+0000","last_clean":"2024-09-10T22:02:35.948529+0000","last_became_active":"2024-09-10T22:00:34.497205+0000","last_became_peered":"2024-09-10T22:00:34.497205+0000","last_unstale":"2024-09-10T22:02:35.948529+0000","last_undegraded":"2024-09-10T22:02:35.948529+0000","last_fullsized":"2024-09-10T22:02:35.948529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:49.924518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723880+0000","last_change":"2024-09-10T22:00:34.504641+0000","last_active":"2024-09-10T22:02:35.723880+0000","last_peered":"2024-09-10T22:02:35.723880+0000","last_clean":"2024-09-10T22:02:35.723880+0000","last_became_active":"2024-09-10T22:00:34.504183+0000","last_became_peered":"2024-09-10T22:00:34.504183+0000","last_unstale":"2024-09-10T22:02:35.723880+0000","last_undegraded":"2024-09-10T22:02:35.723880+0000","last_fullsized":"2024-09-10T22:02:35.723880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:06.704972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723361+0000","last_change":"2024-09-10T22:00:34.503884+0000","last_active":"2024-09-10T22:02:35.723361+0000","last_peered":"2024-09-10T22:02:35.723361+0000","last_clean":"2024-09-10T22:02:35.723361+0000","last_became_active":"2024-09-10T22:00:34.502950+0000","last_became_peered":"2024-09-10T22:00:34.502950+0000","last_unstale":"2024-09-10T22:02:35.723361+0000","last_undegraded":"2024-09-10T22:02:35.723361+0000","last_fullsized":"2024-09-10T22:02:35.723361+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:45.560958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.949045+0000","last_change":"2024-09-10T22:00:34.504981+0000","last_active":"2024-09-10T22:02:35.949045+0000","last_peered":"2024-09-10T22:02:35.949045+0000","last_clean":"2024-09-10T22:02:35.949045+0000","last_became_active":"2024-09-10T22:00:34.500474+0000","last_became_peered":"2024-09-10T22:00:34.500474+0000","last_unstale":"2024-09-10T22:02:35.949045+0000","last_undegraded":"2024-09-10T22:02:35.949045+0000","last_fullsized":"2024-09-10T22:02:35.949045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:20.892605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948231+0000","last_change":"2024-09-10T22:00:34.506650+0000","last_active":"2024-09-10T22:02:35.948231+0000","last_peered":"2024-09-10T22:02:35.948231+0000","last_clean":"2024-09-10T22:02:35.948231+0000","last_became_active":"2024-09-10T22:00:34.505698+0000","last_became_peered":"2024-09-10T22:00:34.505698+0000","last_unstale":"2024-09-10T22:02:35.948231+0000","last_undegraded":"2024-09-10T22:02:35.948231+0000","last_fullsized":"2024-09-10T22:02:35.948231+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:29.221693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947606+0000","last_change":"2024-09-10T22:00:34.502238+0000","last_active":"2024-09-10T22:02:35.947606+0000","last_peered":"2024-09-10T22:02:35.947606+0000","last_clean":"2024-09-10T22:02:35.947606+0000","last_became_active":"2024-09-10T22:00:34.500049+0000","last_became_peered":"2024-09-10T22:00:34.500049+0000","last_unstale":"2024-09-10T22:02:35.947606+0000","last_undegraded":"2024-09-10T22:02:35.947606+0000","last_fullsized":"2024-09-10T22:02:35.947606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:05.669784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947630+0000","last_change":"2024-09-10T22:00:34.499340+0000","last_active":"2024-09-10T22:02:35.947630+0000","last_peered":"2024-09-10T22:02:35.947630+0000","last_clean":"2024-09-10T22:02:35.947630+0000","last_became_active":"2024-09-10T22:00:34.497655+0000","last_became_peered":"2024-09-10T22:00:34.497655+0000","last_unstale":"2024-09-10T22:02:35.947630+0000","last_undegraded":"2024-09-10T22:02:35.947630+0000","last_fullsized":"2024-09-10T22:02:35.947630+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:34.115829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894275+0000","last_change":"2024-09-10T22:00:34.500705+0000","last_active":"2024-09-10T22:02:35.894275+0000","last_peered":"2024-09-10T22:02:35.894275+0000","last_clean":"2024-09-10T22:02:35.894275+0000","last_became_active":"2024-09-10T22:00:34.500071+0000","last_became_peered":"2024-09-10T22:00:34.500071+0000","last_unstale":"2024-09-10T22:02:35.894275+0000","last_undegraded":"2024-09-10T22:02:35.894275+0000","last_fullsized":"2024-09-10T22:02:35.894275+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:40.519976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897606+0000","last_change":"2024-09-10T22:00:34.510570+0000","last_active":"2024-09-10T22:02:35.897606+0000","last_peered":"2024-09-10T22:02:35.897606+0000","last_clean":"2024-09-10T22:02:35.897606+0000","last_became_active":"2024-09-10T22:00:34.506922+0000","last_became_peered":"2024-09-10T22:00:34.506922+0000","last_unstale":"2024-09-10T22:02:35.897606+0000","last_undegraded":"2024-09-10T22:02:35.897606+0000","last_fullsized":"2024-09-10T22:02:35.897606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.489031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863526+0000","last_change":"2024-09-10T22:00:34.509985+0000","last_active":"2024-09-10T22:02:35.863526+0000","last_peered":"2024-09-10T22:02:35.863526+0000","last_clean":"2024-09-10T22:02:35.863526+0000","last_became_active":"2024-09-10T22:00:34.509527+0000","last_became_peered":"2024-09-10T22:00:34.509527+0000","last_unstale":"2024-09-10T22:02:35.863526+0000","last_undegraded":"2024-09-10T22:02:35.863526+0000","last_fullsized":"2024-09-10T22:02:35.863526+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:49.463334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.872121+0000","last_change":"2024-09-10T22:00:34.497986+0000","last_active":"2024-09-10T22:02:35.872121+0000","last_peered":"2024-09-10T22:02:35.872121+0000","last_clean":"2024-09-10T22:02:35.872121+0000","last_became_active":"2024-09-10T22:00:34.497612+0000","last_became_peered":"2024-09-10T22:00:34.497612+0000","last_unstale":"2024-09-10T22:02:35.872121+0000","last_undegraded":"2024-09-10T22:02:35.872121+0000","last_fullsized":"2024-09-10T22:02:35.872121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:15.696988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.949002+0000","last_change":"2024-09-10T22:00:34.498482+0000","last_active":"2024-09-10T22:02:35.949002+0000","last_peered":"2024-09-10T22:02:35.949002+0000","last_clean":"2024-09-10T22:02:35.949002+0000","last_became_active":"2024-09-10T22:00:34.498139+0000","last_became_peered":"2024-09-10T22:00:34.498139+0000","last_unstale":"2024-09-10T22:02:35.949002+0000","last_undegraded":"2024-09-10T22:02:35.949002+0000","last_fullsized":"2024-09-10T22:02:35.949002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:23.427253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722383+0000","last_change":"2024-09-10T22:00:34.491325+0000","last_active":"2024-09-10T22:02:35.722383+0000","last_peered":"2024-09-10T22:02:35.722383+0000","last_clean":"2024-09-10T22:02:35.722383+0000","last_became_active":"2024-09-10T22:00:34.491026+0000","last_became_peered":"2024-09-10T22:00:34.491026+0000","last_unstale":"2024-09-10T22:02:35.722383+0000","last_undegraded":"2024-09-10T22:02:35.722383+0000","last_fullsized":"2024-09-10T22:02:35.722383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:07.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.862981+0000","last_change":"2024-09-10T22:00:34.503975+0000","last_active":"2024-09-10T22:02:35.862981+0000","last_peered":"2024-09-10T22:02:35.862981+0000","last_clean":"2024-09-10T22:02:35.862981+0000","last_became_active":"2024-09-10T22:00:34.503624+0000","last_became_peered":"2024-09-10T22:00:34.503624+0000","last_unstale":"2024-09-10T22:02:35.862981+0000","last_undegraded":"2024-09-10T22:02:35.862981+0000","last_fullsized":"2024-09-10T22:02:35.862981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:17.678811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871637+0000","last_change":"2024-09-10T22:01:35.596824+0000","last_active":"2024-09-10T22:02:35.871637+0000","last_peered":"2024-09-10T22:02:35.871637+0000","last_clean":"2024-09-10T22:02:35.871637+0000","last_became_active":"2024-09-10T22:01:35.596550+0000","last_became_peered":"2024-09-10T22:01:35.596550+0000","last_unstale":"2024-09-10T22:02:35.871637+0000","last_undegraded":"2024-09-10T22:02:35.871637+0000","last_fullsized":"2024-09-10T22:02:35.871637+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:19.617649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.895388+0000","last_change":"2024-09-10T22:02:35.895282+0000","last_active":"2024-09-10T22:02:35.895388+0000","last_peered":"2024-09-10T22:02:35.895388+0000","last_clean":"2024-09-10T22:02:35.895388+0000","last_became_active":"2024-09-10T22:02:35.894850+0000","last_became_peered":"2024-09-10T22:02:35.894850+0000","last_unstale":"2024-09-10T22:02:35.895388+0000","last_undegraded":"2024-09-10T22:02:35.895388+0000","last_fullsized":"2024-09-10T22:02:35.895388+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:50.109012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897539+0000","last_change":"2024-09-10T22:00:34.513134+0000","last_active":"2024-09-10T22:02:35.897539+0000","last_peered":"2024-09-10T22:02:35.897539+0000","last_clean":"2024-09-10T22:02:35.897539+0000","last_became_active":"2024-09-10T22:00:34.507578+0000","last_became_peered":"2024-09-10T22:00:34.507578+0000","last_unstale":"2024-09-10T22:02:35.897539+0000","last_undegraded":"2024-09-10T22:02:35.897539+0000","last_fullsized":"2024-09-10T22:02:35.897539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:33.387337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947814+0000","last_change":"2024-09-10T22:00:34.507645+0000","last_active":"2024-09-10T22:02:35.947814+0000","last_peered":"2024-09-10T22:02:35.947814+0000","last_clean":"2024-09-10T22:02:35.947814+0000","last_became_active":"2024-09-10T22:00:34.503614+0000","last_became_peered":"2024-09-10T22:00:34.503614+0000","last_unstale":"2024-09-10T22:02:35.947814+0000","last_undegraded":"2024-09-10T22:02:35.947814+0000","last_fullsized":"2024-09-10T22:02:35.947814+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:53.033523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947997+0000","last_change":"2024-09-10T22:00:34.501891+0000","last_active":"2024-09-10T22:02:35.947997+0000","last_peered":"2024-09-10T22:02:35.947997+0000","last_clean":"2024-09-10T22:02:35.947997+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:02:35.947997+0000","last_undegraded":"2024-09-10T22:02:35.947997+0000","last_fullsized":"2024-09-10T22:02:35.947997+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:30.068461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.949048+0000","last_change":"2024-09-10T22:00:34.501475+0000","last_active":"2024-09-10T22:02:35.949048+0000","last_peered":"2024-09-10T22:02:35.949048+0000","last_clean":"2024-09-10T22:02:35.949048+0000","last_became_active":"2024-09-10T22:00:34.500909+0000","last_became_peered":"2024-09-10T22:00:34.500909+0000","last_unstale":"2024-09-10T22:02:35.949048+0000","last_undegraded":"2024-09-10T22:02:35.949048+0000","last_fullsized":"2024-09-10T22:02:35.949048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:39.616773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70643,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":751,"num_read_kb":453,"num_write":249,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246272,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678058,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1011712,"data_stored":536256,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874287,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1155072,"data_stored":1185617,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037813,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":539779,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234044,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":535949,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430275,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1077248,"data_stored":1133340,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593802,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1003520,"data_stored":531578,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790032,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1085440,"data_stored":1128140,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953558,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":535652,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1015808,"data_stored":530895,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378722,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1085440,"data_stored":595764,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542249,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1081344,"data_stored":595377,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705774,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1028096,"data_stored":533034,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:02:47.464 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:02:47.464 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T22:02:47.464 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T22:02:47.464 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T22:02:47.836 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:47.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:47 smithi090 ceph-mon[30350]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:02:47.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:47 smithi154 ceph-mon[34495]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:02:47.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:47 smithi171 ceph-mon[34138]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:02:48.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.396+0000 7f4f5a400640 1 -- 172.21.15.90:0/135718995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 msgr2=0x7f4f5410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:48.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.396+0000 7f4f5a400640 1 --2- 172.21.15.90:0/135718995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 0x7f4f5410f930 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009900 tx=0x7f4f4c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.397+0000 7f4f5a400640 1 -- 172.21.15.90:0/135718995 shutdown_connections 2024-09-10T22:02:48.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.397+0000 7f4f5a400640 1 --2- 172.21.15.90:0/135718995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 0x7f4f5410f930 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.397+0000 7f4f5a400640 1 --2- 172.21.15.90:0/135718995 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f541087b0 0x7f4f54108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.397+0000 7f4f5a400640 1 --2- 172.21.15.90:0/135718995 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f54102da0 0x7f4f54103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.397+0000 7f4f5a400640 1 -- 172.21.15.90:0/135718995 >> 172.21.15.90:0/135718995 conn(0x7f4f540fe810 msgr2=0x7f4f54100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.397+0000 7f4f5a400640 1 -- 172.21.15.90:0/135718995 shutdown_connections 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.397+0000 7f4f5a400640 1 -- 172.21.15.90:0/135718995 wait complete. 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f5a400640 1 Processor -- start 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f5a400640 1 -- start start 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f5a400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f54102da0 0x7f4f5419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f5a400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f541087b0 0x7f4f5419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f5a400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 0x7f4f541a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f5a400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54112cf0 con 0x7f4f54109130 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f5a400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54112b70 con 0x7f4f541087b0 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f5a400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54112e70 con 0x7f4f54102da0 2024-09-10T22:02:48.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f53e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 0x7f4f541a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f53e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 0x7f4f541a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52402/0 (socket says 172.21.15.90:52402) 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.398+0000 7f4f53e00640 1 -- 172.21.15.90:0/358489106 learned_addr learned my addr 172.21.15.90:0/358489106 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f53400640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f54102da0 0x7f4f5419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f52a00640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f541087b0 0x7f4f5419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f53e00640 1 -- 172.21.15.90:0/358489106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f54102da0 msgr2=0x7f4f5419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f53e00640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f54102da0 0x7f4f5419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f53e00640 1 -- 172.21.15.90:0/358489106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f541087b0 msgr2=0x7f4f5419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f53e00640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f541087b0 0x7f4f5419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f53e00640 1 -- 172.21.15.90:0/358489106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f541a1360 con 0x7f4f54109130 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f53e00640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 0x7f4f541a0b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c004770 tx=0x7f4f4c003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f4be00640 1 -- 172.21.15.90:0/358489106 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c03f070 con 0x7f4f54109130 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f4be00640 1 -- 172.21.15.90:0/358489106 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f4c03bd00 con 0x7f4f54109130 2024-09-10T22:02:48.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f4be00640 1 -- 172.21.15.90:0/358489106 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c035030 con 0x7f4f54109130 2024-09-10T22:02:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.399+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f541a15f0 con 0x7f4f54109130 2024-09-10T22:02:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.400+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541a7d10 con 0x7f4f54109130 2024-09-10T22:02:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.401+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f54103180 con 0x7f4f54109130 2024-09-10T22:02:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.403+0000 7f4f4be00640 1 -- 172.21.15.90:0/358489106 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f4f4c031bd0 con 0x7f4f54109130 2024-09-10T22:02:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.404+0000 7f4f4be00640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4f380b53c0 0x7f4f380b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.404+0000 7f4f4be00640 1 -- 172.21.15.90:0/358489106 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4f4c106cb0 con 0x7f4f54109130 2024-09-10T22:02:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.404+0000 7f4f53400640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4f380b53c0 0x7f4f380b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.405+0000 7f4f53400640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4f380b53c0 0x7f4f380b7880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f44007ec0 tx=0x7f4f440075b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:48.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.408+0000 7f4f4be00640 1 -- 172.21.15.90:0/358489106 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4f4c0cced0 con 0x7f4f54109130 2024-09-10T22:02:48.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.630+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4f5402e5c0 con 0x7f4f54109130 2024-09-10T22:02:48.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.630+0000 7f4f4be00640 1 -- 172.21.15.90:0/358489106 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4f4c0d1900 con 0x7f4f54109130 2024-09-10T22:02:48.632 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:48.632 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T22:02:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4f380b53c0 msgr2=0x7f4f380b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4f380b53c0 0x7f4f380b7880 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f44007ec0 tx=0x7f4f440075b0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 msgr2=0x7f4f541a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 0x7f4f541a0b90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c004770 tx=0x7f4f4c003cd0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 shutdown_connections 2024-09-10T22:02:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4f380b53c0 0x7f4f380b7880 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f54109130 0x7f4f541a0b90 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4f541087b0 0x7f4f5419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 --2- 172.21.15.90:0/358489106 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4f54102da0 0x7f4f5419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:48.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 >> 172.21.15.90:0/358489106 conn(0x7f4f540fe810 msgr2=0x7f4f54104d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:48.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 shutdown_connections 2024-09-10T22:02:48.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:48.633+0000 7f4f5a400640 1 -- 172.21.15.90:0/358489106 wait complete. 2024-09-10T22:02:48.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:48 smithi090 ceph-mon[30350]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:02:48.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:48 smithi154 ceph-mon[34495]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:02:48.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:48 smithi171 ceph-mon[34138]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:02:49.417 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T22:02:49.417 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:02:49.788 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:49.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:49 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/358489106' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:02:49.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:49 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/358489106' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:02:49.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:49 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/358489106' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T22:02:50.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.409+0000 7fe75a800640 1 -- 172.21.15.90:0/666802770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754102c10 msgr2=0x7fe754102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.409+0000 7fe75a800640 1 --2- 172.21.15.90:0/666802770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754102c10 0x7fe754102ff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe748009900 tx=0x7fe74802a410 comp rx=0 tx=0).stop 2024-09-10T22:02:50.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.409+0000 7fe75a800640 1 -- 172.21.15.90:0/666802770 shutdown_connections 2024-09-10T22:02:50.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.409+0000 7fe75a800640 1 --2- 172.21.15.90:0/666802770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe754108fa0 0x7fe75410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.409+0000 7fe75a800640 1 --2- 172.21.15.90:0/666802770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe754108620 0x7fe754108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.409+0000 7fe75a800640 1 --2- 172.21.15.90:0/666802770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754102c10 0x7fe754102ff0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.409+0000 7fe75a800640 1 -- 172.21.15.90:0/666802770 >> 172.21.15.90:0/666802770 conn(0x7fe7540fe6c0 msgr2=0x7fe754100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:50.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.409+0000 7fe75a800640 1 -- 172.21.15.90:0/666802770 shutdown_connections 2024-09-10T22:02:50.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.409+0000 7fe75a800640 1 -- 172.21.15.90:0/666802770 wait complete. 2024-09-10T22:02:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.410+0000 7fe75a800640 1 Processor -- start 2024-09-10T22:02:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.410+0000 7fe75a800640 1 -- start start 2024-09-10T22:02:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe75a800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe754102c10 0x7fe75419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe75a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe754108620 0x7fe75419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe75a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754108fa0 0x7fe7541a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe75a800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe754112b20 con 0x7fe754108fa0 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe75a800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541129a0 con 0x7fe754102c10 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe75a800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe754112ca0 con 0x7fe754108620 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe759e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754108fa0 0x7fe7541a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe759e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754108fa0 0x7fe7541a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52416/0 (socket says 172.21.15.90:52416) 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe759e00640 1 -- 172.21.15.90:0/2289094598 learned_addr learned my addr 172.21.15.90:0/2289094598 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe759400640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe754102c10 0x7fe75419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.411+0000 7fe758a00640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe754108620 0x7fe75419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe759e00640 1 -- 172.21.15.90:0/2289094598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe754108620 msgr2=0x7fe75419b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe759e00640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe754108620 0x7fe75419b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe759e00640 1 -- 172.21.15.90:0/2289094598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe754102c10 msgr2=0x7fe75419b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe759e00640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe754102c10 0x7fe75419b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe759e00640 1 -- 172.21.15.90:0/2289094598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7541a1230 con 0x7fe754108fa0 2024-09-10T22:02:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe759e00640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754108fa0 0x7fe7541a0a60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00eb10 tx=0x7fe74c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe752000640 1 -- 172.21.15.90:0/2289094598 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c004b20 con 0x7fe754108fa0 2024-09-10T22:02:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe752000640 1 -- 172.21.15.90:0/2289094598 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe74c004cc0 con 0x7fe754108fa0 2024-09-10T22:02:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7541a1520 con 0x7fe754108fa0 2024-09-10T22:02:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe752000640 1 -- 172.21.15.90:0/2289094598 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c010690 con 0x7fe754108fa0 2024-09-10T22:02:50.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.412+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7541a7be0 con 0x7fe754108fa0 2024-09-10T22:02:50.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.414+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe754102ff0 con 0x7fe754108fa0 2024-09-10T22:02:50.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.416+0000 7fe752000640 1 -- 172.21.15.90:0/2289094598 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fe74c005020 con 0x7fe754108fa0 2024-09-10T22:02:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.417+0000 7fe752000640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe73c0b54e0 0x7fe73c0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.417+0000 7fe752000640 1 -- 172.21.15.90:0/2289094598 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe74c0e67f0 con 0x7fe754108fa0 2024-09-10T22:02:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.417+0000 7fe759400640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe73c0b54e0 0x7fe73c0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:50.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.418+0000 7fe759400640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe73c0b54e0 0x7fe73c0b79a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe748006010 tx=0x7fe748005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:50.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.421+0000 7fe752000640 1 -- 172.21.15.90:0/2289094598 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe74c0ad060 con 0x7fe754108fa0 2024-09-10T22:02:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.648+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe75402e5c0 con 0x7fe754108fa0 2024-09-10T22:02:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.649+0000 7fe752000640 1 -- 172.21.15.90:0/2289094598 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7fe74c0ac960 con 0x7fe754108fa0 2024-09-10T22:02:50.650 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:50.650 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"btime":"2024-09-10T22:01:34:736152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:01:34.015563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:02:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-09-10T22:02:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe73c0b54e0 msgr2=0x7fe73c0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe73c0b54e0 0x7fe73c0b79a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe748006010 tx=0x7fe748005e80 comp rx=0 tx=0).stop 2024-09-10T22:02:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754108fa0 msgr2=0x7fe7541a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754108fa0 0x7fe7541a0a60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00eb10 tx=0x7fe74c00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 shutdown_connections 2024-09-10T22:02:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe73c0b54e0 0x7fe73c0b79a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe754108fa0 0x7fe7541a0a60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe754108620 0x7fe75419b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 --2- 172.21.15.90:0/2289094598 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe754102c10 0x7fe75419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 >> 172.21.15.90:0/2289094598 conn(0x7fe7540fe6c0 msgr2=0x7fe754104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 shutdown_connections 2024-09-10T22:02:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:50.651+0000 7fe75a800640 1 -- 172.21.15.90:0/2289094598 wait complete. 2024-09-10T22:02:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:50 smithi090 ceph-mon[30350]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:02:50.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:50 smithi154 ceph-mon[34495]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:02:50.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:50 smithi171 ceph-mon[34138]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:02:51.438 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:02:51.811 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:51.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:51 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2289094598' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:02:51.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:51 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2289094598' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:02:51.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:51 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2289094598' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.364+0000 7fbf06c00640 1 -- 172.21.15.90:0/3493390409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 msgr2=0x7fbf0010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.364+0000 7fbf06c00640 1 --2- 172.21.15.90:0/3493390409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 0x7fbf0010adf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009900 tx=0x7fbee802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.364+0000 7fbf06c00640 1 -- 172.21.15.90:0/3493390409 shutdown_connections 2024-09-10T22:02:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.364+0000 7fbf06c00640 1 --2- 172.21.15.90:0/3493390409 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf0010b330 0x7fbf00111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.364+0000 7fbf06c00640 1 --2- 172.21.15.90:0/3493390409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 0x7fbf0010adf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.364+0000 7fbf06c00640 1 --2- 172.21.15.90:0/3493390409 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf00104fa0 0x7fbf00105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.364+0000 7fbf06c00640 1 -- 172.21.15.90:0/3493390409 >> 172.21.15.90:0/3493390409 conn(0x7fbf00100a50 msgr2=0x7fbf00102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.364+0000 7fbf06c00640 1 -- 172.21.15.90:0/3493390409 shutdown_connections 2024-09-10T22:02:52.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.364+0000 7fbf06c00640 1 -- 172.21.15.90:0/3493390409 wait complete. 2024-09-10T22:02:52.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.365+0000 7fbf06c00640 1 Processor -- start 2024-09-10T22:02:52.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbf06c00640 1 -- start start 2024-09-10T22:02:52.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbf06c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf00104fa0 0x7fbf0019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbf06c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 0x7fbf0019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbf06c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf0010b330 0x7fbf001a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbf06c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00114e60 con 0x7fbf0010a9b0 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbf06c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00114ce0 con 0x7fbf0010b330 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbf06c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00114fe0 con 0x7fbf00104fa0 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbef7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 0x7fbf0019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbef7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 0x7fbf0019b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52434/0 (socket says 172.21.15.90:52434) 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.366+0000 7fbef7e00640 1 -- 172.21.15.90:0/73601269 learned_addr learned my addr 172.21.15.90:0/73601269 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.367+0000 7fbeffe00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf00104fa0 0x7fbf0019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.367+0000 7fbf04c00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf0010b330 0x7fbf001a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.367+0000 7fbef7e00640 1 -- 172.21.15.90:0/73601269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf00104fa0 msgr2=0x7fbf0019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.367+0000 7fbef7e00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf00104fa0 0x7fbf0019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.367+0000 7fbef7e00640 1 -- 172.21.15.90:0/73601269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf0010b330 msgr2=0x7fbf001a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.367+0000 7fbef7e00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf0010b330 0x7fbf001a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.367+0000 7fbef7e00640 1 -- 172.21.15.90:0/73601269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf001a1480 con 0x7fbf0010a9b0 2024-09-10T22:02:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.367+0000 7fbef7e00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 0x7fbf0019b910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbee802a910 tx=0x7fbee8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.368+0000 7fbefd600640 1 -- 172.21.15.90:0/73601269 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee803f070 con 0x7fbf0010a9b0 2024-09-10T22:02:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.368+0000 7fbefd600640 1 -- 172.21.15.90:0/73601269 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbee8003e40 con 0x7fbf0010a9b0 2024-09-10T22:02:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.368+0000 7fbefd600640 1 -- 172.21.15.90:0/73601269 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8035030 con 0x7fbf0010a9b0 2024-09-10T22:02:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.368+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf001a1710 con 0x7fbf0010a9b0 2024-09-10T22:02:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.368+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf001a7e30 con 0x7fbf0010a9b0 2024-09-10T22:02:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.369+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf00105380 con 0x7fbf0010a9b0 2024-09-10T22:02:52.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.371+0000 7fbefd600640 1 -- 172.21.15.90:0/73601269 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fbee8031400 con 0x7fbf0010a9b0 2024-09-10T22:02:52.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.372+0000 7fbefd600640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbee40b5440 0x7fbee40b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:52.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.373+0000 7fbefd600640 1 -- 172.21.15.90:0/73601269 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fbee8106fe0 con 0x7fbf0010a9b0 2024-09-10T22:02:52.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.373+0000 7fbeffe00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbee40b5440 0x7fbee40b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:52.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.373+0000 7fbeffe00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbee40b5440 0x7fbee40b7900 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbef000bbe0 tx=0x7fbef000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:52.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.374+0000 7fbefd600640 1 -- 172.21.15.90:0/73601269 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbee80cd240 con 0x7fbf0010a9b0 2024-09-10T22:02:52.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.594+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf0019c500 con 0x7fbf0010a9b0 2024-09-10T22:02:52.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.594+0000 7fbefd600640 1 -- 172.21.15.90:0/73601269 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7fbee80d1c70 con 0x7fbf0010a9b0 2024-09-10T22:02:52.595 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:52.596 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"btime":"2024-09-10T22:01:34:736152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:01:34.015563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:02:52.596 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-09-10T22:02:52.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbee40b5440 msgr2=0x7fbee40b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbee40b5440 0x7fbee40b7900 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbef000bbe0 tx=0x7fbef000b390 comp rx=0 tx=0).stop 2024-09-10T22:02:52.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 msgr2=0x7fbf0019b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:52.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 0x7fbf0019b910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbee802a910 tx=0x7fbee8003c20 comp rx=0 tx=0).stop 2024-09-10T22:02:52.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 shutdown_connections 2024-09-10T22:02:52.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fbee40b5440 0x7fbee40b7900 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fbf0010b330 0x7fbf001a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf0010a9b0 0x7fbf0019b910 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 --2- 172.21.15.90:0/73601269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fbf00104fa0 0x7fbf0019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:52.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.597+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 >> 172.21.15.90:0/73601269 conn(0x7fbf00100a50 msgr2=0x7fbf00102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:52.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.598+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 shutdown_connections 2024-09-10T22:02:52.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:52.598+0000 7fbf06c00640 1 -- 172.21.15.90:0/73601269 wait complete. 2024-09-10T22:02:52.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:52 smithi090 ceph-mon[30350]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:02:52.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:52 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/73601269' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:02:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:52 smithi154 ceph-mon[34495]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:02:52.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:52 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/73601269' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:02:52.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:52 smithi171 ceph-mon[34138]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:02:52.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:52 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/73601269' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:02:53.536 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:02:53.908 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:54.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.505+0000 7f22a4a00640 1 -- 172.21.15.90:0/2194719216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22a010b330 msgr2=0x7f22a0111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.505+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2194719216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22a010b330 0x7f22a0111b30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2298009be0 tx=0x7f229802a6c0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.505+0000 7f22a4a00640 1 -- 172.21.15.90:0/2194719216 shutdown_connections 2024-09-10T22:02:54.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.505+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2194719216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22a010b330 0x7f22a0111b30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.505+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2194719216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22a010a9b0 0x7f22a010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.505+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2194719216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22a0104fa0 0x7f22a0105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.505+0000 7f22a4a00640 1 -- 172.21.15.90:0/2194719216 >> 172.21.15.90:0/2194719216 conn(0x7f22a0100a50 msgr2=0x7f22a0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:54.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.505+0000 7f22a4a00640 1 -- 172.21.15.90:0/2194719216 shutdown_connections 2024-09-10T22:02:54.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.505+0000 7f22a4a00640 1 -- 172.21.15.90:0/2194719216 wait complete. 2024-09-10T22:02:54.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.506+0000 7f22a4a00640 1 Processor -- start 2024-09-10T22:02:54.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.506+0000 7f22a4a00640 1 -- start start 2024-09-10T22:02:54.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f22a4a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22a0104fa0 0x7f22a0197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:54.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f22a4a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22a010a9b0 0x7f22a0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:54.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f22a4a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22a010b330 0x7f22a01995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:54.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f22a4a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0079c80 con 0x7f22a0104fa0 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f22a4a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0079b00 con 0x7f22a010a9b0 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f22a4a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0079e00 con 0x7f22a010b330 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f229de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22a0104fa0 0x7f22a0197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f229de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22a0104fa0 0x7f22a0197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52446/0 (socket says 172.21.15.90:52446) 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f229de00640 1 -- 172.21.15.90:0/2193981362 learned_addr learned my addr 172.21.15.90:0/2193981362 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f229e800640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22a010b330 0x7f22a01995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.507+0000 7f229d400640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22a010a9b0 0x7f22a0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f229de00640 1 -- 172.21.15.90:0/2193981362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22a010b330 msgr2=0x7f22a01995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f229de00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22a010b330 0x7f22a01995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f229de00640 1 -- 172.21.15.90:0/2193981362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22a010a9b0 msgr2=0x7f22a0197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f229de00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22a010a9b0 0x7f22a0197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f229de00640 1 -- 172.21.15.90:0/2193981362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a0199d70 con 0x7f22a0104fa0 2024-09-10T22:02:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f229d400640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22a010a9b0 0x7f22a0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f229de00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22a0104fa0 0x7f22a0197690 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f229000e9f0 tx=0x7f229000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f2296a00640 1 -- 172.21.15.90:0/2193981362 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229000cd80 con 0x7f22a0104fa0 2024-09-10T22:02:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f2296a00640 1 -- 172.21.15.90:0/2193981362 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f22900054b0 con 0x7f22a0104fa0 2024-09-10T22:02:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f2296a00640 1 -- 172.21.15.90:0/2193981362 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290010430 con 0x7f22a0104fa0 2024-09-10T22:02:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a019a060 con 0x7f22a0104fa0 2024-09-10T22:02:54.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.508+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a01a39d0 con 0x7f22a0104fa0 2024-09-10T22:02:54.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.510+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a0105380 con 0x7f22a0104fa0 2024-09-10T22:02:54.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.512+0000 7f2296a00640 1 -- 172.21.15.90:0/2193981362 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f2290005020 con 0x7f22a0104fa0 2024-09-10T22:02:54.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.513+0000 7f2296a00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f22840b53c0 0x7f22840b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:54.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.513+0000 7f2296a00640 1 -- 172.21.15.90:0/2193981362 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f22900e6720 con 0x7f22a0104fa0 2024-09-10T22:02:54.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.513+0000 7f229d400640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f22840b53c0 0x7f22840b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:54.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.514+0000 7f229d400640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f22840b53c0 0x7f22840b7880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f22a010a080 tx=0x7f228c015040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:54.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.517+0000 7f2296a00640 1 -- 172.21.15.90:0/2193981362 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f22900acf90 con 0x7f22a0104fa0 2024-09-10T22:02:54.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.706+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22a0111c10 con 0x7f22a0104fa0 2024-09-10T22:02:54.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.708+0000 7f2296a00640 1 -- 172.21.15.90:0/2193981362 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25227 (secure 0 0 0) 0x7f22900ac890 con 0x7f22a0104fa0 2024-09-10T22:02:54.709 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:54.709 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:02:35.713572+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:02:54.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f22840b53c0 msgr2=0x7f22840b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f22840b53c0 0x7f22840b7880 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f22a010a080 tx=0x7f228c015040 comp rx=0 tx=0).stop 2024-09-10T22:02:54.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22a0104fa0 msgr2=0x7f22a0197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:54.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22a0104fa0 0x7f22a0197690 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f229000e9f0 tx=0x7f229000edf0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 shutdown_connections 2024-09-10T22:02:54.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f22840b53c0 0x7f22840b7880 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f22a010b330 0x7f22a01995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f22a010a9b0 0x7f22a0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 --2- 172.21.15.90:0/2193981362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22a0104fa0 0x7f22a0197690 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:54.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.710+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 >> 172.21.15.90:0/2193981362 conn(0x7f22a0100a50 msgr2=0x7f22a007a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:54.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.711+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 shutdown_connections 2024-09-10T22:02:54.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:54.711+0000 7f22a4a00640 1 -- 172.21.15.90:0/2193981362 wait complete. 2024-09-10T22:02:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:54 smithi171 ceph-mon[34138]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:02:55.005 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:54 smithi090 ceph-mon[30350]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:02:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:54 smithi154 ceph-mon[34495]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:02:55.611 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:02:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:55 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2193981362' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:02:55.992 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:55.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:55 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2193981362' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:02:56.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:55 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2193981362' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:02:56.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.581+0000 7fb724800640 1 -- 172.21.15.90:0/3477407238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c1087d0 msgr2=0x7fb71c108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.581+0000 7fb724800640 1 --2- 172.21.15.90:0/3477407238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c1087d0 0x7fb71c108c10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009900 tx=0x7fb70c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.582+0000 7fb724800640 1 -- 172.21.15.90:0/3477407238 shutdown_connections 2024-09-10T22:02:56.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.582+0000 7fb724800640 1 --2- 172.21.15.90:0/3477407238 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb71c109150 0x7fb71c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.582+0000 7fb724800640 1 --2- 172.21.15.90:0/3477407238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c1087d0 0x7fb71c108c10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.582+0000 7fb724800640 1 --2- 172.21.15.90:0/3477407238 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb71c102dc0 0x7fb71c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.582+0000 7fb724800640 1 -- 172.21.15.90:0/3477407238 >> 172.21.15.90:0/3477407238 conn(0x7fb71c0fe830 msgr2=0x7fb71c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:56.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.582+0000 7fb724800640 1 -- 172.21.15.90:0/3477407238 shutdown_connections 2024-09-10T22:02:56.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.582+0000 7fb724800640 1 -- 172.21.15.90:0/3477407238 wait complete. 2024-09-10T22:02:56.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.583+0000 7fb724800640 1 Processor -- start 2024-09-10T22:02:56.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.583+0000 7fb724800640 1 -- start start 2024-09-10T22:02:56.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.583+0000 7fb724800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb71c102dc0 0x7fb71c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.583+0000 7fb724800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb71c1087d0 0x7fb71c19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.583+0000 7fb724800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c109150 0x7fb71c1a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.583+0000 7fb724800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c112ca0 con 0x7fb71c109150 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.583+0000 7fb724800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c112b20 con 0x7fb71c1087d0 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.583+0000 7fb724800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c112e20 con 0x7fb71c102dc0 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.583+0000 7fb722800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c109150 0x7fb71c1a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb722800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c109150 0x7fb71c1a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52460/0 (socket says 172.21.15.90:52460) 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb722800640 1 -- 172.21.15.90:0/821724833 learned_addr learned my addr 172.21.15.90:0/821724833 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb721e00640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb71c102dc0 0x7fb71c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb721400640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb71c1087d0 0x7fb71c19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb722800640 1 -- 172.21.15.90:0/821724833 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb71c102dc0 msgr2=0x7fb71c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb722800640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb71c102dc0 0x7fb71c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb722800640 1 -- 172.21.15.90:0/821724833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb71c1087d0 msgr2=0x7fb71c19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb722800640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb71c1087d0 0x7fb71c19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb722800640 1 -- 172.21.15.90:0/821724833 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c1a14c0 con 0x7fb71c109150 2024-09-10T22:02:56.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb721400640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb71c1087d0 0x7fb71c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:56.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.584+0000 7fb722800640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c109150 0x7fb71c1a0cf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb71800eaa0 tx=0x7fb71800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:56.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.585+0000 7fb716a00640 1 -- 172.21.15.90:0/821724833 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718004d20 con 0x7fb71c109150 2024-09-10T22:02:56.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.585+0000 7fb716a00640 1 -- 172.21.15.90:0/821724833 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb718005020 con 0x7fb71c109150 2024-09-10T22:02:56.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.585+0000 7fb716a00640 1 -- 172.21.15.90:0/821724833 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7180116d0 con 0x7fb71c109150 2024-09-10T22:02:56.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.585+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c1a17b0 con 0x7fb71c109150 2024-09-10T22:02:56.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.585+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c1a7e70 con 0x7fb71c109150 2024-09-10T22:02:56.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.586+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c1031a0 con 0x7fb71c109150 2024-09-10T22:02:56.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.588+0000 7fb716a00640 1 -- 172.21.15.90:0/821724833 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fb718022060 con 0x7fb71c109150 2024-09-10T22:02:56.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.589+0000 7fb716a00640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7040b5440 0x7fb7040b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:56.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.589+0000 7fb716a00640 1 -- 172.21.15.90:0/821724833 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb7180e5cd0 con 0x7fb71c109150 2024-09-10T22:02:56.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.589+0000 7fb721e00640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7040b5440 0x7fb7040b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:56.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.590+0000 7fb721e00640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7040b5440 0x7fb7040b7900 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb71000abe0 tx=0x7fb71000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:56.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.593+0000 7fb716a00640 1 -- 172.21.15.90:0/821724833 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb718016030 con 0x7fb71c109150 2024-09-10T22:02:56.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.820+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb71c02e5c0 con 0x7fb71c109150 2024-09-10T22:02:56.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:56 smithi090 ceph-mon[30350]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:02:56.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.821+0000 7fb716a00640 1 -- 172.21.15.90:0/821724833 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7fb7180ac5a0 con 0x7fb71c109150 2024-09-10T22:02:56.822 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:02:56.822 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"btime":"2024-09-10T22:01:34:736152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:01:34.015563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:02:56.823 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-09-10T22:02:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.824+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7040b5440 msgr2=0x7fb7040b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.824+0000 7fb724800640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7040b5440 0x7fb7040b7900 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb71000abe0 tx=0x7fb71000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.824+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c109150 msgr2=0x7fb71c1a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.824+0000 7fb724800640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c109150 0x7fb71c1a0cf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb71800eaa0 tx=0x7fb71800eea0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.824+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 shutdown_connections 2024-09-10T22:02:56.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.825+0000 7fb724800640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb7040b5440 0x7fb7040b7900 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.825+0000 7fb724800640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb71c109150 0x7fb71c1a0cf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.825+0000 7fb724800640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb71c1087d0 0x7fb71c19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.825+0000 7fb724800640 1 --2- 172.21.15.90:0/821724833 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb71c102dc0 0x7fb71c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:56.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.825+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 >> 172.21.15.90:0/821724833 conn(0x7fb71c0fe830 msgr2=0x7fb71c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:56.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.825+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 shutdown_connections 2024-09-10T22:02:56.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:56.825+0000 7fb724800640 1 -- 172.21.15.90:0/821724833 wait complete. 2024-09-10T22:02:56.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:56 smithi171 ceph-mon[34138]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:02:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:56 smithi154 ceph-mon[34495]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:02:57.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:00:35.595646+0000', 'modified': '2024-09-10T22:01:34.015563+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34703, 'mds_1': 34670, 'mds_2': 34646, 'mds_3': 34640, 'mds_4': 34607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14841': {'gid': 14841, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6839/3596805819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 3596805819}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 3596805819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14853': {'gid': 14853, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6841/3169114114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6840', 'nonce': 3169114114}, {'type': 'v1', 'addr': '172.21.15.90:6841', 'nonce': 3169114114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24598': {'gid': 24598, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6833/3906453887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6832', 'nonce': 3906453887}, {'type': 'v1', 'addr': '172.21.15.171:6833', 'nonce': 3906453887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24655': {'gid': 24655, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/2889350064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 2889350064}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 2889350064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24676': {'gid': 24676, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6839/3372214298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 3372214298}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 3372214298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34607': {'gid': 34607, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.154:6833/2781539957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6832', 'nonce': 2781539957}, {'type': 'v1', 'addr': '172.21.15.154:6833', 'nonce': 2781539957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34640': {'gid': 34640, 'name': 'e', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.154:6835/622088003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 622088003}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 622088003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34646': {'gid': 34646, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.171:6835/2969925217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 2969925217}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 2969925217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34670': {'gid': 34670, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.154:6837/3907253955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 3907253955}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 3907253955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34703': {'gid': 34703, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.154:6839/703390433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 703390433}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 703390433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34703, 'qdb_cluster': [34607, 34646, 34640, 34670, 34703]} 2024-09-10T22:02:57.761 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T22:02:57.761 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T22:02:57.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:57 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/821724833' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:02:58.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:57 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/821724833' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:02:58.129 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:02:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:57 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/821724833' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:02:58.773 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:02:58 smithi090 ceph-mon[30350]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:02:58.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.755+0000 7f0ec2400640 1 -- 172.21.15.90:0/2384854419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 msgr2=0x7f0ebc10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.755+0000 7f0ec2400640 1 --2- 172.21.15.90:0/2384854419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 0x7f0ebc10f7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009ca0 tx=0x7f0eb402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.755+0000 7f0ec2400640 1 -- 172.21.15.90:0/2384854419 shutdown_connections 2024-09-10T22:02:58.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.755+0000 7f0ec2400640 1 --2- 172.21.15.90:0/2384854419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 0x7f0ebc10f7a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.755+0000 7f0ec2400640 1 --2- 172.21.15.90:0/2384854419 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc108620 0x7f0ebc108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.755+0000 7f0ec2400640 1 --2- 172.21.15.90:0/2384854419 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ebc102c10 0x7f0ebc102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.755+0000 7f0ec2400640 1 -- 172.21.15.90:0/2384854419 >> 172.21.15.90:0/2384854419 conn(0x7f0ebc0fe660 msgr2=0x7f0ebc100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.755+0000 7f0ec2400640 1 -- 172.21.15.90:0/2384854419 shutdown_connections 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.755+0000 7f0ec2400640 1 -- 172.21.15.90:0/2384854419 wait complete. 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.756+0000 7f0ec2400640 1 Processor -- start 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.756+0000 7f0ec2400640 1 -- start start 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.756+0000 7f0ec2400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc102c10 0x7f0ebc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.756+0000 7f0ec2400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ebc108620 0x7f0ebc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.756+0000 7f0ec2400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 0x7f0ebc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.756+0000 7f0ec2400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc112b60 con 0x7f0ebc108fa0 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.756+0000 7f0ec2400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc1129e0 con 0x7f0ebc108620 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.756+0000 7f0ec2400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc112ce0 con 0x7f0ebc102c10 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0ec1a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 0x7f0ebc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0ec1a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 0x7f0ebc075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47850/0 (socket says 172.21.15.90:47850) 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0ec1a00640 1 -- 172.21.15.90:0/837363586 learned_addr learned my addr 172.21.15.90:0/837363586 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0ec1000640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc102c10 0x7f0ebc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ebc108620 0x7f0ebc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0ec1a00640 1 -- 172.21.15.90:0/837363586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc102c10 msgr2=0x7f0ebc0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0ec1a00640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc102c10 0x7f0ebc0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0ec1a00640 1 -- 172.21.15.90:0/837363586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ebc108620 msgr2=0x7f0ebc0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0ec1a00640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ebc108620 0x7f0ebc0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.757+0000 7f0ec1a00640 1 -- 172.21.15.90:0/837363586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc1a7310 con 0x7f0ebc108fa0 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.758+0000 7f0ec1a00640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 0x7f0ebc075f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4031040 tx=0x7f0eb4003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.758+0000 7f0eba000640 1 -- 172.21.15.90:0/837363586 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4032c90 con 0x7f0ebc108fa0 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.758+0000 7f0eba000640 1 -- 172.21.15.90:0/837363586 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0eb4032e30 con 0x7f0ebc108fa0 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.758+0000 7f0ec2400640 1 -- 172.21.15.90:0/837363586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc1a75a0 con 0x7f0ebc108fa0 2024-09-10T22:02:58.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.758+0000 7f0eba000640 1 -- 172.21.15.90:0/837363586 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4030af0 con 0x7f0ebc108fa0 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.758+0000 7f0ec2400640 1 -- 172.21.15.90:0/837363586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc1a7ae0 con 0x7f0ebc108fa0 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.761+0000 7f0ec63dd740 1 Processor -- start 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.761+0000 7f0eba000640 1 -- 172.21.15.90:0/837363586 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f0eb4030c90 con 0x7f0ebc108fa0 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0ec63dd740 1 -- start start 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9a400 0x55d48ed9a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 0x55d48eda6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48eda90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d48ed11d50 con 0x55d48ed9a400 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d48ed01180 con 0x55d48ed9ad20 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d48ed04f90 con 0x55d48eda6cb0 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0eb3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9a400 0x55d48ed9a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0eb3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9a400 0x55d48ed9a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47864/0 (socket says 172.21.15.90:47864) 2024-09-10T22:02:58.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0eb3e00640 1 -- 172.21.15.90:0/3576615411 learned_addr learned my addr 172.21.15.90:0/3576615411 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.762+0000 7f0ec1000640 1 --2- 172.21.15.90:0/3576615411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 0x55d48eda6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/3576615411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48eda90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb3e00640 1 -- 172.21.15.90:0/3576615411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 msgr2=0x55d48eda90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/3576615411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48eda90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb3e00640 1 -- 172.21.15.90:0/3576615411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 msgr2=0x55d48eda6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/3576615411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 0x55d48eda6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb3e00640 1 -- 172.21.15.90:0/3576615411 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d48eccba70 con 0x55d48ed9a400 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eba000640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0ea00b5460 0x7f0ea00b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/3576615411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9a400 0x55d48ed9a7e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400bfd0 tx=0x7f0ea400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49c07ccad1a7be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eba000640 1 -- 172.21.15.90:0/837363586 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f0eb41073b0 con 0x7f0ebc108fa0 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb2000640 1 -- 172.21.15.90:0/3576615411 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4004030 con 0x55d48ed9a400 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0ec1000640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0ea00b5460 0x7f0ea00b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb2000640 1 -- 172.21.15.90:0/3576615411 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ea40035f0 con 0x55d48ed9a400 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.763+0000 7f0eb2000640 1 -- 172.21.15.90:0/3576615411 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4010660 con 0x55d48ed9a400 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3576615411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9a400 msgr2=0x55d48ed9a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec1000640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0ea00b5460 0x7f0ea00b7920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55d48ed9d580 tx=0x7f0eac005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/3576615411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9a400 0x55d48ed9a7e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0ea400bfd0 tx=0x7f0ea400b840 comp rx=0 tx=0).stop 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3576615411 shutdown_connections 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/3576615411 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48eda90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/3576615411 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 0x55d48eda6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/3576615411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9a400 0x55d48ed9a7e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3576615411 >> 172.21.15.90:0/3576615411 conn(0x55d48ed98230 msgr2=0x55d48ed993d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3576615411 shutdown_connections 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.764+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3576615411 wait complete. 2024-09-10T22:02:58.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.765+0000 7f0ec63dd740 1 -- start start 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.765+0000 7f0ec63dd740 1 Processor -- start 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.765+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48ed9a400 0x55d48eda3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.765+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 0x55d48eda4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.765+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48eda6cb0 0x55d48eda47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.765+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d48eb625f0 con 0x55d48eda6cb0 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.765+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d48ec9c460 con 0x55d48ed9ad20 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.765+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d48eda9a00 con 0x55d48ed9a400 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.765+0000 7f0ec63dd740 10 client.0 _ll_register_callbacks cb 0x55d48ecac210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48eda6cb0 0x55d48eda47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48eda6cb0 0x55d48eda47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47874/0 (socket says 172.21.15.90:47874) 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1a00640 1 -- 172.21.15.90:0/4254067680 learned_addr learned my addr 172.21.15.90:0/4254067680 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:58.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 0x55d48eda4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1000640 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48ed9a400 0x55d48eda3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1a00640 1 -- 172.21.15.90:0/4254067680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48ed9a400 msgr2=0x55d48eda3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1a00640 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48ed9a400 0x55d48eda3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1a00640 1 -- 172.21.15.90:0/4254067680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 msgr2=0x55d48eda4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1a00640 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 0x55d48eda4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1a00640 1 -- 172.21.15.90:0/4254067680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d48ea5f540 con 0x55d48eda6cb0 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1000640 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48ed9a400 0x55d48eda3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0ec1a00640 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48eda6cb0 0x55d48eda47a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40cd4b0 tx=0x7f0eb4104cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a63533b9456654 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.766+0000 7f0eaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0eaaa00640 1 -- 172.21.15.90:0/4254067680 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40cd980 con 0x55d48eda6cb0 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0eaaa00640 1 -- 172.21.15.90:0/4254067680 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f0eb40d6620 con 0x55d48eda6cb0 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0ec63dd740 1 -- 172.21.15.90:0/4254067680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d48ec9ea70 con 0x55d48eda6cb0 2024-09-10T22:02:58.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0eaaa00640 1 -- 172.21.15.90:0/4254067680 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0eb40cdc20 con 0x55d48eda6cb0 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0ec63dd740 10 client.15309 fetch_fsmap learned FSMap version 29 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0ec63dd740 1 -- 172.21.15.90:0/4254067680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d48ecb96f0 con 0x55d48eda6cb0 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0eaaa00640 1 -- 172.21.15.90:0/4254067680 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d48ec9ea70 con 0x55d48eda6cb0 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0eaaa00640 1 -- 172.21.15.90:0/4254067680 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0eb40cc780 con 0x55d48eda6cb0 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0ec63dd740 10 client.15309 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.767+0000 7f0ec63dd740 10 client.15309 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.768+0000 7f0ec63dd740 20 client.15309 populate_metadata read hostname 'smithi090' 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.768+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x55d48edb7b40 0x55d48edb8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.768+0000 7f0ec63dd740 4 client.15309 mds_command: new command op to 34607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.768+0000 7f0ec63dd740 1 -- 172.21.15.90:0/4254067680 --> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d48ed13ca0 con 0x55d48edb7b40 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.768+0000 7f0ec1000640 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x55d48edb7b40 0x55d48edb8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.769+0000 7f0ec1000640 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x55d48edb7b40 0x55d48edb8fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=74fade283034b172 server_cookie=207b9f30a635cfa in_seq=0 out_seq=0 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.769+0000 7f0eaaa00640 10 client.15309 ms_handle_connect on v2:172.21.15.154:6832/2781539957 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0eaaa00640 1 -- 172.21.15.90:0/4254067680 <== mds.4 v2:172.21.15.154:6832/2781539957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d48ed13ca0 con 0x55d48edb7b40 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0eaaa00640 10 client.15309 handle_command_reply: tid=0 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 client.15309 shutdown 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 -- 172.21.15.90:0/4254067680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48eda6cb0 msgr2=0x55d48eda47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48eda6cb0 0x55d48eda47a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40cd4b0 tx=0x7f0eb4104cf0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 -- 172.21.15.90:0/4254067680 shutdown_connections 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x55d48edb7b40 0x55d48edb8fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48eda6cb0 0x55d48eda47a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9ad20 0x55d48eda4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/4254067680 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48ed9a400 0x55d48eda3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0eaaa00640 0 client.15309 ms_handle_reset on v2:172.21.15.154:6832/2781539957 2024-09-10T22:02:58.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 -- 172.21.15.90:0/4254067680 >> 172.21.15.90:0/4254067680 conn(0x55d48ed98230 msgr2=0x55d48ed9ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 -- 172.21.15.90:0/4254067680 shutdown_connections 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.770+0000 7f0ec63dd740 1 -- 172.21.15.90:0/4254067680 wait complete. 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.771+0000 7f0ec63dd740 20 client.15309 trim_cache size 0 max 16384 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.789+0000 7f0ec63dd740 1 Processor -- start 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.789+0000 7f0ec63dd740 1 -- start start 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.789+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48ed9e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.789+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edb67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48edb6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d48edb72f0 con 0x55d48ed9ad20 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d48edb79c0 con 0x55d48ed9a400 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d48edb8090 con 0x55d48eda6cb0 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0eb3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edb67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0eb3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edb67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47876/0 (socket says 172.21.15.90:47876) 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0eb3e00640 1 -- 172.21.15.90:0/3643507201 learned_addr learned my addr 172.21.15.90:0/3643507201 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0eb3e00640 1 -- 172.21.15.90:0/3643507201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 msgr2=0x55d48edb6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/3643507201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48edb6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0ec1000640 1 --2- 172.21.15.90:0/3643507201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48ed9e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0eb3e00640 1 -- 172.21.15.90:0/3643507201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 msgr2=0x55d48ed9e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/3643507201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48ed9e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0eb3e00640 1 -- 172.21.15.90:0/3643507201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d48ed98cb0 con 0x55d48ed9ad20 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0ec1000640 1 --2- 172.21.15.90:0/3643507201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48ed9e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.790+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/3643507201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edb67f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4012ec0 tx=0x7f0ea400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5288e9386aecaf17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0eb1600640 1 -- 172.21.15.90:0/3643507201 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40050d0 con 0x55d48ed9ad20 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0eb1600640 1 -- 172.21.15.90:0/3643507201 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ea40035f0 con 0x55d48ed9ad20 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0eb1600640 1 -- 172.21.15.90:0/3643507201 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea40106a0 con 0x55d48ed9ad20 2024-09-10T22:02:58.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3643507201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 msgr2=0x55d48edb67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/3643507201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edb67f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0ea4012ec0 tx=0x7f0ea400b390 comp rx=0 tx=0).stop 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3643507201 shutdown_connections 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/3643507201 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48edb6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/3643507201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edb67f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/3643507201 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48ed9e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3643507201 >> 172.21.15.90:0/3643507201 conn(0x55d48ed98230 msgr2=0x55d48ed9efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.791+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3643507201 shutdown_connections 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.792+0000 7f0ec63dd740 1 -- 172.21.15.90:0/3643507201 wait complete. 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.792+0000 7f0ec63dd740 1 -- start start 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.792+0000 7f0ec63dd740 1 Processor -- start 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48edc1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edbaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0ec63dd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48edbb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d48edb8e30 con 0x55d48ed9ad20 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d48edb8760 con 0x55d48ed9a400 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0ec63dd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d48edb9500 con 0x55d48eda6cb0 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0ec63dd740 10 client.0 _ll_register_callbacks cb 0x7f0eb40c93c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0eb3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edbaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0eb3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edbaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47888/0 (socket says 172.21.15.90:47888) 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0eb3e00640 1 -- 172.21.15.90:0/2715949906 learned_addr learned my addr 172.21.15.90:0/2715949906 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0ec1000640 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48edc1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0eb3e00640 1 -- 172.21.15.90:0/2715949906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 msgr2=0x55d48edbb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:02:58.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48edbb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0eb3e00640 1 -- 172.21.15.90:0/2715949906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 msgr2=0x55d48edc1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:02:58.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48edc1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0eb3e00640 1 -- 172.21.15.90:0/2715949906 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d48ed938c0 con 0x55d48ed9ad20 2024-09-10T22:02:58.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.793+0000 7f0ec1000640 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48edc1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:02:58.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.794+0000 7f0eb3e00640 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edbaff0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40122f0 tx=0x7f0ea4012320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8103653187c5e4b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:02:58.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.794+0000 7f0eaaa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:02:58.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.794+0000 7f0eaaa00640 1 -- 172.21.15.90:0/2715949906 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea4003b70 con 0x55d48ed9ad20 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.794+0000 7f0eaaa00640 1 -- 172.21.15.90:0/2715949906 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f0ea40050d0 con 0x55d48ed9ad20 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.794+0000 7f0ec63dd740 1 -- 172.21.15.90:0/2715949906 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ea40035f0 con 0x55d48ed9ad20 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.794+0000 7f0eaaa00640 1 -- 172.21.15.90:0/2715949906 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ea403e050 con 0x55d48ed9ad20 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.794+0000 7f0ec63dd740 10 client.15315 fetch_fsmap learned FSMap version 29 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.794+0000 7f0ec63dd740 1 -- 172.21.15.90:0/2715949906 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d48edc14d0 con 0x55d48ed9ad20 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.794+0000 7f0eaaa00640 1 -- 172.21.15.90:0/2715949906 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0ea40035f0 con 0x55d48ed9ad20 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.795+0000 7f0eaaa00640 1 -- 172.21.15.90:0/2715949906 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f0ea4010a10 con 0x55d48ed9ad20 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.795+0000 7f0ec63dd740 10 client.15315 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.795+0000 7f0ec63dd740 10 client.15315 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.795+0000 7f0ec63dd740 20 client.15315 populate_metadata read hostname 'smithi090' 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.795+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x55d48ee287a0 0x55d48ee274b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:02:58.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.795+0000 7f0ec63dd740 4 client.15315 mds_command: new command op to 34607 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.795+0000 7f0ec63dd740 1 -- 172.21.15.90:0/2715949906 --> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] -- command(tid 0: {"prefix": "status"}) -- 0x55d48ed0f750 con 0x55d48ee287a0 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.795+0000 7f0ec1000640 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x55d48ee287a0 0x55d48ee274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.796+0000 7f0ec1000640 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x55d48ee287a0 0x55d48ee274b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ff0c39ad7b104c67 server_cookie=68156ea48dd1dc0f in_seq=0 out_seq=0 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.796+0000 7f0eaaa00640 10 client.15315 ms_handle_connect on v2:172.21.15.154:6832/2781539957 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.796+0000 7f0eaaa00640 1 -- 172.21.15.90:0/2715949906 <== mds.4 v2:172.21.15.154:6832/2781539957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d48ed0f750 con 0x55d48ee287a0 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.796+0000 7f0eaaa00640 10 client.15315 handle_command_reply: tid=0 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.796+0000 7f0ec63dd740 1 client.15315 shutdown 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.796+0000 7f0ec63dd740 1 -- 172.21.15.90:0/2715949906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 msgr2=0x55d48edbaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.796+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edbaff0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40122f0 tx=0x7f0ea4012320 comp rx=0 tx=0).stop 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec63dd740 1 -- 172.21.15.90:0/2715949906 shutdown_connections 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x55d48ee287a0 0x55d48ee274b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d48eda6cb0 0x55d48edbb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d48ed9ad20 0x55d48edbaff0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec63dd740 1 --2- 172.21.15.90:0/2715949906 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d48ed9a400 0x55d48edc1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec63dd740 1 -- 172.21.15.90:0/2715949906 >> 172.21.15.90:0/2715949906 conn(0x55d48ed98230 msgr2=0x55d48ed9efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0eaaa00640 0 client.15315 ms_handle_reset on v2:172.21.15.154:6832/2781539957 2024-09-10T22:02:58.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec63dd740 1 -- 172.21.15.90:0/2715949906 shutdown_connections 2024-09-10T22:02:58.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec63dd740 1 -- 172.21.15.90:0/2715949906 wait complete. 2024-09-10T22:02:58.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec63dd740 20 client.15315 trim_cache size 0 max 16384 2024-09-10T22:02:58.859 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:02:58.859 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:02:58.859 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 4, 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34607, 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 87.063791030000004, 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 195.30171830500001 2024-09-10T22:02:58.860 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec2400640 1 -- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0ea00b5460 msgr2=0x7f0ea00b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec2400640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0ea00b5460 0x7f0ea00b7920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55d48ed9d580 tx=0x7f0eac005f90 comp rx=0 tx=0).stop 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec2400640 1 -- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 msgr2=0x7f0ebc075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.797+0000 7f0ec2400640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 0x7f0ebc075f20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4031040 tx=0x7f0eb4003f80 comp rx=0 tx=0).stop 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.798+0000 7f0ec2400640 1 -- 172.21.15.90:0/837363586 shutdown_connections 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.798+0000 7f0ec2400640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0ea00b5460 0x7f0ea00b7920 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.798+0000 7f0ec2400640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ebc108fa0 0x7f0ebc075f20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.798+0000 7f0ec2400640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0ebc108620 0x7f0ebc0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.798+0000 7f0ec2400640 1 --2- 172.21.15.90:0/837363586 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0ebc102c10 0x7f0ebc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.798+0000 7f0ec2400640 1 -- 172.21.15.90:0/837363586 >> 172.21.15.90:0/837363586 conn(0x7f0ebc0fe660 msgr2=0x7f0ebc100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:02:58.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.798+0000 7f0ec2400640 1 -- 172.21.15.90:0/837363586 shutdown_connections 2024-09-10T22:02:58.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:02:58.798+0000 7f0ec2400640 1 -- 172.21.15.90:0/837363586 wait complete. 2024-09-10T22:02:58.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:02:58 smithi171 ceph-mon[34138]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:02:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:02:58 smithi154 ceph-mon[34495]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:00.000 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-10T22:03:00.373 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:00.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:00 smithi090 ceph-mon[30350]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:03:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:00 smithi171 ceph-mon[34138]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:03:01.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.975+0000 7fe892800640 1 -- 172.21.15.90:0/2138470679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10a800 msgr2=0x7fe88c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.975+0000 7fe892800640 1 --2- 172.21.15.90:0/2138470679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10a800 0x7fe88c10ac40 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe878009900 tx=0x7fe87802a410 comp rx=0 tx=0).stop 2024-09-10T22:03:01.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.975+0000 7fe892800640 1 -- 172.21.15.90:0/2138470679 shutdown_connections 2024-09-10T22:03:01.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.975+0000 7fe892800640 1 --2- 172.21.15.90:0/2138470679 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe88c10b180 0x7fe88c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.975+0000 7fe892800640 1 --2- 172.21.15.90:0/2138470679 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10a800 0x7fe88c10ac40 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.975+0000 7fe892800640 1 --2- 172.21.15.90:0/2138470679 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe88c104df0 0x7fe88c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.975+0000 7fe892800640 1 -- 172.21.15.90:0/2138470679 >> 172.21.15.90:0/2138470679 conn(0x7fe88c100840 msgr2=0x7fe88c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.975+0000 7fe892800640 1 -- 172.21.15.90:0/2138470679 shutdown_connections 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.975+0000 7fe892800640 1 -- 172.21.15.90:0/2138470679 wait complete. 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.976+0000 7fe892800640 1 Processor -- start 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.976+0000 7fe892800640 1 -- start start 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.976+0000 7fe892800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe88c104df0 0x7fe88c19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.976+0000 7fe892800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe88c10a800 0x7fe88c19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.976+0000 7fe892800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10b180 0x7fe88c1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.976+0000 7fe892800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c114cd0 con 0x7fe88c10b180 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.976+0000 7fe892800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c114b50 con 0x7fe88c104df0 2024-09-10T22:03:01.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.976+0000 7fe892800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c114e50 con 0x7fe88c10a800 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10b180 0x7fe88c1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10b180 0x7fe88c1a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47894/0 (socket says 172.21.15.90:47894) 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891e00640 1 -- 172.21.15.90:0/260251878 learned_addr learned my addr 172.21.15.90:0/260251878 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891400640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe88c104df0 0x7fe88c19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe890a00640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe88c10a800 0x7fe88c19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891e00640 1 -- 172.21.15.90:0/260251878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe88c10a800 msgr2=0x7fe88c19b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891e00640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe88c10a800 0x7fe88c19b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891e00640 1 -- 172.21.15.90:0/260251878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe88c104df0 msgr2=0x7fe88c19b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891e00640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe88c104df0 0x7fe88c19b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891e00640 1 -- 172.21.15.90:0/260251878 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c1a11b0 con 0x7fe88c10b180 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe890a00640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe88c10a800 0x7fe88c19b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:01.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.977+0000 7fe891e00640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10b180 0x7fe88c1a09e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe88400eaa0 tx=0x7fe88400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.978+0000 7fe88a000640 1 -- 172.21.15.90:0/260251878 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88400cdb0 con 0x7fe88c10b180 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.978+0000 7fe88a000640 1 -- 172.21.15.90:0/260251878 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe884005020 con 0x7fe88c10b180 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.978+0000 7fe892800640 1 -- 172.21.15.90:0/260251878 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c1a14a0 con 0x7fe88c10b180 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.978+0000 7fe88a000640 1 -- 172.21.15.90:0/260251878 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8840116d0 con 0x7fe88c10b180 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.978+0000 7fe892800640 1 -- 172.21.15.90:0/260251878 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c1a7b60 con 0x7fe88c10b180 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.981+0000 7fe8967f2740 1 Processor -- start 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.981+0000 7fe88a000640 1 -- 172.21.15.90:0/260251878 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fe88400f040 con 0x7fe88c10b180 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.981+0000 7fe8967f2740 1 -- start start 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5400 0x559084ac57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5d20 0x559084ad1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 0x559084ad40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe8967f2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559084a3cd50 con 0x559084ac5d20 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe8967f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559084a2c180 con 0x559084ad1cb0 2024-09-10T22:03:01.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe8967f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559084a2ff90 con 0x559084ac5400 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe891400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5d20 0x559084ad1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe891400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5d20 0x559084ad1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47910/0 (socket says 172.21.15.90:47910) 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe891400640 1 -- 172.21.15.90:0/2599682131 learned_addr learned my addr 172.21.15.90:0/2599682131 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe890a00640 1 --2- 172.21.15.90:0/2599682131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5400 0x559084ac57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe890a00640 1 --2- 172.21.15.90:0/2599682131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 0x559084ad40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe891400640 1 -- 172.21.15.90:0/2599682131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5400 msgr2=0x559084ac57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe891400640 1 --2- 172.21.15.90:0/2599682131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5400 0x559084ac57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe891400640 1 -- 172.21.15.90:0/2599682131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 msgr2=0x559084ad40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe891400640 1 --2- 172.21.15.90:0/2599682131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 0x559084ad40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.982+0000 7fe891400640 1 -- 172.21.15.90:0/2599682131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590849f6a70 con 0x559084ac5d20 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.983+0000 7fe891400640 1 --2- 172.21.15.90:0/2599682131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5d20 0x559084ad1770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe88000bd40 tx=0x7fe8800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8f2d445b8dd348 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.983+0000 7fe87e600640 1 -- 172.21.15.90:0/2599682131 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880004230 con 0x559084ac5d20 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.983+0000 7fe88a000640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe8740b5460 0x7fe8740b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.983+0000 7fe87e600640 1 -- 172.21.15.90:0/2599682131 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe88000be30 con 0x559084ac5d20 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.983+0000 7fe87e600640 1 -- 172.21.15.90:0/2599682131 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88001a7e0 con 0x559084ac5d20 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.983+0000 7fe88a000640 1 -- 172.21.15.90:0/260251878 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe8840e6160 con 0x7fe88c10b180 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.983+0000 7fe890a00640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe8740b5460 0x7fe8740b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe8967f2740 1 -- 172.21.15.90:0/2599682131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5d20 msgr2=0x559084ad1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2599682131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5d20 0x559084ad1770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe88000bd40 tx=0x7fe8800107a0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe890a00640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe8740b5460 0x7fe8740b7920 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x559084ac53b0 tx=0x7fe878005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe8967f2740 1 -- 172.21.15.90:0/2599682131 shutdown_connections 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2599682131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 0x559084ad40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2599682131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5d20 0x559084ad1770 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2599682131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5400 0x559084ac57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe8967f2740 1 -- 172.21.15.90:0/2599682131 >> 172.21.15.90:0/2599682131 conn(0x559084ac3230 msgr2=0x559084ac43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe8967f2740 1 -- 172.21.15.90:0/2599682131 shutdown_connections 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.984+0000 7fe8967f2740 1 -- 172.21.15.90:0/2599682131 wait complete. 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe8967f2740 1 -- start start 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe8967f2740 1 Processor -- start 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5400 0x559084acec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5d20 0x559084acf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 0x559084acf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe8967f2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55908488d5f0 con 0x559084ac5400 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe8967f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5590849c7460 con 0x559084ad1cb0 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe8967f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559084ad4a00 con 0x559084ac5d20 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe8967f2740 10 client.0 _ll_register_callbacks cb 0x5590849d7210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.985+0000 7fe891400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5400 0x559084acec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe891400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5400 0x559084acec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47920/0 (socket says 172.21.15.90:47920) 2024-09-10T22:03:01.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe891400640 1 -- 172.21.15.90:0/1380557882 learned_addr learned my addr 172.21.15.90:0/1380557882 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe891e00640 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 0x559084acf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe890a00640 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5d20 0x559084acf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe891400640 1 -- 172.21.15.90:0/1380557882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5d20 msgr2=0x559084acf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe891400640 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5d20 0x559084acf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe891400640 1 -- 172.21.15.90:0/1380557882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 msgr2=0x559084acf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe891400640 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 0x559084acf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe891400640 1 -- 172.21.15.90:0/1380557882 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908478a540 con 0x559084ac5400 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe891400640 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5400 0x559084acec00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe880003e20 tx=0x7fe88000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79334564458e288 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.986+0000 7fe872a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe872a00640 1 -- 172.21.15.90:0/1380557882 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880029060 con 0x559084ac5400 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe872a00640 1 -- 172.21.15.90:0/1380557882 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe880003800 con 0x559084ac5400 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe8967f2740 1 -- 172.21.15.90:0/1380557882 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590849c9a70 con 0x559084ac5400 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe872a00640 1 -- 172.21.15.90:0/1380557882 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe880003a00 con 0x559084ac5400 2024-09-10T22:03:01.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe8967f2740 10 client.15327 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe8967f2740 1 -- 172.21.15.90:0/1380557882 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590849e46f0 con 0x559084ac5400 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe872a00640 1 -- 172.21.15.90:0/1380557882 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5590849c9a70 con 0x559084ac5400 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe872a00640 1 -- 172.21.15.90:0/1380557882 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe880005440 con 0x559084ac5400 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe8967f2740 10 client.15327 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe8967f2740 10 client.15327 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.987+0000 7fe8967f2740 20 client.15327 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.988+0000 7fe8967f2740 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559084ae2630 0x559084ae3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.988+0000 7fe8967f2740 4 client.15327 mds_command: new command op to 34640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.988+0000 7fe8967f2740 1 -- 172.21.15.90:0/1380557882 --> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559084a3eca0 con 0x559084ae2630 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.988+0000 7fe890a00640 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559084ae2630 0x559084ae3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.989+0000 7fe890a00640 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559084ae2630 0x559084ae3aa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=96700a295198925a server_cookie=ebb957547b473fdb in_seq=0 out_seq=0 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.989+0000 7fe872a00640 10 client.15327 ms_handle_connect on v2:172.21.15.154:6834/622088003 2024-09-10T22:03:01.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe872a00640 1 -- 172.21.15.90:0/1380557882 <== mds.3 v2:172.21.15.154:6834/622088003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559084a3eca0 con 0x559084ae2630 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe872a00640 10 client.15327 handle_command_reply: tid=0 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 client.15327 shutdown 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 -- 172.21.15.90:0/1380557882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5400 msgr2=0x559084acec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5400 0x559084acec00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe880003e20 tx=0x7fe88000ba40 comp rx=0 tx=0).stop 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 -- 172.21.15.90:0/1380557882 shutdown_connections 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ad1cb0 0x559084acf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559084ae2630 0x559084ae3aa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac5d20 0x559084acf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 --2- 172.21.15.90:0/1380557882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac5400 0x559084acec00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe872a00640 0 client.15327 ms_handle_reset on v2:172.21.15.154:6834/622088003 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 -- 172.21.15.90:0/1380557882 >> 172.21.15.90:0/1380557882 conn(0x559084ac3230 msgr2=0x559084ac8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 -- 172.21.15.90:0/1380557882 shutdown_connections 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 1 -- 172.21.15.90:0/1380557882 wait complete. 2024-09-10T22:03:01.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:00.990+0000 7fe8967f2740 20 client.15327 trim_cache size 0 max 16384 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.009+0000 7fe8967f2740 1 Processor -- start 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.009+0000 7fe8967f2740 1 -- start start 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.009+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 0x559084ac9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ad1cb0 0x559084ac4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac47c0 0x559084acbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe8967f2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559084ae1de0 con 0x559084ad1cb0 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe8967f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559084ae24b0 con 0x559084ac5d20 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe8967f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559084ae2b80 con 0x559084ac47c0 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe890a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ad1cb0 0x559084ac4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe890a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ad1cb0 0x559084ac4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47930/0 (socket says 172.21.15.90:47930) 2024-09-10T22:03:01.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe890a00640 1 -- 172.21.15.90:0/4135718767 learned_addr learned my addr 172.21.15.90:0/4135718767 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe891400640 1 --2- 172.21.15.90:0/4135718767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 0x559084ac9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe891e00640 1 --2- 172.21.15.90:0/4135718767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac47c0 0x559084acbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe890a00640 1 -- 172.21.15.90:0/4135718767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac47c0 msgr2=0x559084acbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe890a00640 1 --2- 172.21.15.90:0/4135718767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac47c0 0x559084acbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe890a00640 1 -- 172.21.15.90:0/4135718767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 msgr2=0x559084ac9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe890a00640 1 --2- 172.21.15.90:0/4135718767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 0x559084ac9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.010+0000 7fe890a00640 1 -- 172.21.15.90:0/4135718767 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559084ac3cb0 con 0x559084ad1cb0 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.011+0000 7fe890a00640 1 --2- 172.21.15.90:0/4135718767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ad1cb0 0x559084ac4280 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe88c107000 tx=0x7fe87803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa30d5bccf815267 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.011+0000 7fe87dc00640 1 -- 172.21.15.90:0/4135718767 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878059070 con 0x559084ad1cb0 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.011+0000 7fe87dc00640 1 -- 172.21.15.90:0/4135718767 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8780054c0 con 0x559084ad1cb0 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.011+0000 7fe87dc00640 1 -- 172.21.15.90:0/4135718767 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878035040 con 0x559084ad1cb0 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 -- 172.21.15.90:0/4135718767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ad1cb0 msgr2=0x559084ac4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 --2- 172.21.15.90:0/4135718767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ad1cb0 0x559084ac4280 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe88c107000 tx=0x7fe87803f000 comp rx=0 tx=0).stop 2024-09-10T22:03:01.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 -- 172.21.15.90:0/4135718767 shutdown_connections 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 --2- 172.21.15.90:0/4135718767 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ac47c0 0x559084acbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 --2- 172.21.15.90:0/4135718767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ad1cb0 0x559084ac4280 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 --2- 172.21.15.90:0/4135718767 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 0x559084ac9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 -- 172.21.15.90:0/4135718767 >> 172.21.15.90:0/4135718767 conn(0x559084ac3230 msgr2=0x559084ac9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 -- 172.21.15.90:0/4135718767 shutdown_connections 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 -- 172.21.15.90:0/4135718767 wait complete. 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 -- start start 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.012+0000 7fe8967f2740 1 Processor -- start 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac47c0 0x559084aec0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 0x559084aec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe8967f2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ad1cb0 0x559084ae6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe8967f2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559084ae3920 con 0x559084ac47c0 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe8967f2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559084ae3250 con 0x559084ac5d20 2024-09-10T22:03:01.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe8967f2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559084ae3ff0 con 0x559084ad1cb0 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe8967f2740 10 client.0 _ll_register_callbacks cb 0x7fe88000b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe891400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac47c0 0x559084aec0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe891400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac47c0 0x559084aec0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47938/0 (socket says 172.21.15.90:47938) 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe891400640 1 -- 172.21.15.90:0/2612938279 learned_addr learned my addr 172.21.15.90:0/2612938279 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe890a00640 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 0x559084aec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.013+0000 7fe891e00640 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ad1cb0 0x559084ae6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe891400640 1 -- 172.21.15.90:0/2612938279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ad1cb0 msgr2=0x559084ae6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe891400640 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ad1cb0 0x559084ae6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe891400640 1 -- 172.21.15.90:0/2612938279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 msgr2=0x559084aec640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe891400640 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 0x559084aec640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe891400640 1 -- 172.21.15.90:0/2612938279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559084ae6ea0 con 0x559084ac47c0 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe891400640 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac47c0 0x559084aec0c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe8800048d0 tx=0x7fe880003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a9a5ba14da6756 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe872a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:01.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe872a00640 1 -- 172.21.15.90:0/2612938279 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880038060 con 0x559084ac47c0 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe872a00640 1 -- 172.21.15.90:0/2612938279 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe880003ba0 con 0x559084ac47c0 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe8967f2740 1 -- 172.21.15.90:0/2612938279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8780054c0 con 0x559084ac47c0 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe872a00640 1 -- 172.21.15.90:0/2612938279 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe88003f050 con 0x559084ac47c0 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe8967f2740 10 client.15333 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.014+0000 7fe8967f2740 1 -- 172.21.15.90:0/2612938279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559084b4fb90 con 0x559084ac47c0 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.015+0000 7fe872a00640 1 -- 172.21.15.90:0/2612938279 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8780054c0 con 0x559084ac47c0 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.015+0000 7fe872a00640 1 -- 172.21.15.90:0/2612938279 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe880005060 con 0x559084ac47c0 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.015+0000 7fe8967f2740 10 client.15333 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.015+0000 7fe8967f2740 10 client.15333 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.015+0000 7fe8967f2740 20 client.15333 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.015+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559084b54720 0x559084b53430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.015+0000 7fe8967f2740 4 client.15333 mds_command: new command op to 34640 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:01.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.015+0000 7fe8967f2740 1 -- 172.21.15.90:0/2612938279 --> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] -- command(tid 0: {"prefix": "status"}) -- 0x559084a3a750 con 0x559084b54720 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.016+0000 7fe890a00640 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559084b54720 0x559084b53430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.016+0000 7fe890a00640 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559084b54720 0x559084b53430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4b85b5a0c098f5f5 server_cookie=fa3ec612f584cd51 in_seq=0 out_seq=0 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.016+0000 7fe872a00640 10 client.15333 ms_handle_connect on v2:172.21.15.154:6834/622088003 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.016+0000 7fe872a00640 1 -- 172.21.15.90:0/2612938279 <== mds.3 v2:172.21.15.154:6834/622088003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559084a3a750 con 0x559084b54720 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.016+0000 7fe872a00640 10 client.15333 handle_command_reply: tid=0 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 client.15333 shutdown 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 -- 172.21.15.90:0/2612938279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac47c0 msgr2=0x559084aec0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac47c0 0x559084aec0c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe8800048d0 tx=0x7fe880003870 comp rx=0 tx=0).stop 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 -- 172.21.15.90:0/2612938279 shutdown_connections 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559084b54720 0x559084b53430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559084ad1cb0 0x559084ae6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559084ac5d20 0x559084aec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 --2- 172.21.15.90:0/2612938279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559084ac47c0 0x559084aec0c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 -- 172.21.15.90:0/2612938279 >> 172.21.15.90:0/2612938279 conn(0x559084ac3230 msgr2=0x559084ad0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe872a00640 0 client.15333 ms_handle_reset on v2:172.21.15.154:6834/622088003 2024-09-10T22:03:01.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 -- 172.21.15.90:0/2612938279 shutdown_connections 2024-09-10T22:03:01.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.017+0000 7fe8967f2740 1 -- 172.21.15.90:0/2612938279 wait complete. 2024-09-10T22:03:01.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe8967f2740 20 client.15333 trim_cache size 0 max 16384 2024-09-10T22:03:01.030 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:01.030 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 3, 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34640, 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 90.299144487999996, 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 183.48305902600001 2024-09-10T22:03:01.031 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 -- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe8740b5460 msgr2=0x7fe8740b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe8740b5460 0x7fe8740b7920 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x559084ac53b0 tx=0x7fe878005fd0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 -- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10b180 msgr2=0x7fe88c1a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10b180 0x7fe88c1a09e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe88400eaa0 tx=0x7fe88400ef60 comp rx=0 tx=0).stop 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 -- 172.21.15.90:0/260251878 shutdown_connections 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe8740b5460 0x7fe8740b7920 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe88c10b180 0x7fe88c1a09e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe88c10a800 0x7fe88c19b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 --2- 172.21.15.90:0/260251878 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe88c104df0 0x7fe88c19b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 -- 172.21.15.90:0/260251878 >> 172.21.15.90:0/260251878 conn(0x7fe88c100840 msgr2=0x7fe88c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 -- 172.21.15.90:0/260251878 shutdown_connections 2024-09-10T22:03:01.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:01.018+0000 7fe892800640 1 -- 172.21.15.90:0/260251878 wait complete. 2024-09-10T22:03:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:00 smithi154 ceph-mon[34495]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:03:01.901 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T22:03:02.284 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:02.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.915+0000 7f9417200640 1 -- 172.21.15.90:0/6511228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 msgr2=0x7f9418102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.915+0000 7f9417200640 1 --2- 172.21.15.90:0/6511228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 0x7f9418102ff0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9408009900 tx=0x7f940802a410 comp rx=0 tx=0).stop 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.915+0000 7f9417200640 1 -- 172.21.15.90:0/6511228 shutdown_connections 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.915+0000 7f9417200640 1 --2- 172.21.15.90:0/6511228 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9418108fa0 0x7f941810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.915+0000 7f9417200640 1 --2- 172.21.15.90:0/6511228 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9418108620 0x7f9418108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.915+0000 7f9417200640 1 --2- 172.21.15.90:0/6511228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 0x7f9418102ff0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.915+0000 7f9417200640 1 -- 172.21.15.90:0/6511228 >> 172.21.15.90:0/6511228 conn(0x7f94180fe660 msgr2=0x7f9418100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.915+0000 7f9417200640 1 -- 172.21.15.90:0/6511228 shutdown_connections 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.915+0000 7f9417200640 1 -- 172.21.15.90:0/6511228 wait complete. 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.916+0000 7f9417200640 1 Processor -- start 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.916+0000 7f9417200640 1 -- start start 2024-09-10T22:03:02.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9417200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 0x7f941819b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9417200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9418108620 0x7f941819b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9417200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9418108fa0 0x7f94181a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9417200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9418112b60 con 0x7f9418102c10 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9417200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94181129e0 con 0x7f9418108620 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9417200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9418112ce0 con 0x7f9418108fa0 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9415e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 0x7f941819b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9415e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 0x7f941819b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47966/0 (socket says 172.21.15.90:47966) 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9415e00640 1 -- 172.21.15.90:0/1568473585 learned_addr learned my addr 172.21.15.90:0/1568473585 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9415400640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9418108620 0x7f941819b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.917+0000 7f9416800640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9418108fa0 0x7f94181a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f9415e00640 1 -- 172.21.15.90:0/1568473585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9418108fa0 msgr2=0x7f94181a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f9415e00640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9418108fa0 0x7f94181a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f9415e00640 1 -- 172.21.15.90:0/1568473585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9418108620 msgr2=0x7f941819b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f9415e00640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9418108620 0x7f941819b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f9415e00640 1 -- 172.21.15.90:0/1568473585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94181a11e0 con 0x7f9418102c10 2024-09-10T22:03:02.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f9415e00640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 0x7f941819b130 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f940802a930 tx=0x7f9408003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:02 smithi090 ceph-mon[30350]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:02.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f940ea00640 1 -- 172.21.15.90:0/1568473585 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940803f070 con 0x7f9418102c10 2024-09-10T22:03:02.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f940ea00640 1 -- 172.21.15.90:0/1568473585 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9408003e40 con 0x7f9418102c10 2024-09-10T22:03:02.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f940ea00640 1 -- 172.21.15.90:0/1568473585 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408035030 con 0x7f9418102c10 2024-09-10T22:03:02.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f9417200640 1 -- 172.21.15.90:0/1568473585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94181a1470 con 0x7f9418102c10 2024-09-10T22:03:02.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.918+0000 7f9417200640 1 -- 172.21.15.90:0/1568473585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94181a7b90 con 0x7f9418102c10 2024-09-10T22:03:02.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.922+0000 7f941fbd5740 1 Processor -- start 2024-09-10T22:03:02.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f941fbd5740 1 -- start start 2024-09-10T22:03:02.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761400 0x55e8887617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 0x55e88876d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e88876dcb0 0x55e8887700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f941fbd5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e8886d8d50 con 0x55e888761400 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f941fbd5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e8886c8180 con 0x55e888761d20 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f941fbd5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e8886cbf90 con 0x55e88876dcb0 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f940ea00640 1 -- 172.21.15.90:0/1568473585 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f9408031400 con 0x7f9418102c10 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f9416800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761400 0x55e8887617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f9416800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761400 0x55e8887617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47972/0 (socket says 172.21.15.90:47972) 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f9416800640 1 -- 172.21.15.90:0/2016440299 learned_addr learned my addr 172.21.15.90:0/2016440299 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.923+0000 7f9415400640 1 --2- 172.21.15.90:0/2016440299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e88876dcb0 0x55e8887700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.924+0000 7f9416800640 1 -- 172.21.15.90:0/2016440299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e88876dcb0 msgr2=0x55e8887700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.924+0000 7f9416800640 1 --2- 172.21.15.90:0/2016440299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e88876dcb0 0x55e8887700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.924+0000 7f9416800640 1 -- 172.21.15.90:0/2016440299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 msgr2=0x55e88876d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.924+0000 7f9416800640 1 --2- 172.21.15.90:0/2016440299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 0x55e88876d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.924+0000 7f9416800640 1 -- 172.21.15.90:0/2016440299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e888692a70 con 0x55e888761400 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.924+0000 7f9416800640 1 --2- 172.21.15.90:0/2016440299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761400 0x55e8887617e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9410007f70 tx=0x7f941000f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac52de5037ed3ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.924+0000 7f9403400640 1 -- 172.21.15.90:0/2016440299 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410010900 con 0x55e888761400 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.924+0000 7f9403400640 1 -- 172.21.15.90:0/2016440299 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9410014070 con 0x55e888761400 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.924+0000 7f9403400640 1 -- 172.21.15.90:0/2016440299 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410017cd0 con 0x55e888761400 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f940ea00640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93fc0b54e0 0x7f93fc0b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f941fbd5740 1 -- 172.21.15.90:0/2016440299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761400 msgr2=0x55e8887617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f941fbd5740 1 --2- 172.21.15.90:0/2016440299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761400 0x55e8887617e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9410007f70 tx=0x7f941000f850 comp rx=0 tx=0).stop 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f940ea00640 1 -- 172.21.15.90:0/1568473585 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9408106c50 con 0x7f9418102c10 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f9415e00640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93fc0b54e0 0x7f93fc0b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f941fbd5740 1 -- 172.21.15.90:0/2016440299 shutdown_connections 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f941fbd5740 1 --2- 172.21.15.90:0/2016440299 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e88876dcb0 0x55e8887700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f941fbd5740 1 --2- 172.21.15.90:0/2016440299 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 0x55e88876d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f941fbd5740 1 --2- 172.21.15.90:0/2016440299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761400 0x55e8887617e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f941fbd5740 1 -- 172.21.15.90:0/2016440299 >> 172.21.15.90:0/2016440299 conn(0x55e88875f230 msgr2=0x55e8887603d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f941fbd5740 1 -- 172.21.15.90:0/2016440299 shutdown_connections 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.925+0000 7f941fbd5740 1 -- 172.21.15.90:0/2016440299 wait complete. 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.926+0000 7f9415e00640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93fc0b54e0 0x7f93fc0b79a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f940803bfd0 tx=0x7f9408009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.926+0000 7f941fbd5740 1 -- start start 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.926+0000 7f941fbd5740 1 Processor -- start 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 0x55e88876ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 0x55e88876b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e88876dcb0 0x55e88876b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f941fbd5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e8885295f0 con 0x55e88876dcb0 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f941fbd5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e888663460 con 0x55e888761d20 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f941fbd5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e888770a00 con 0x55e888761400 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f941fbd5740 10 client.0 _ll_register_callbacks cb 0x55e888673210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:02.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f9416800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 0x55e88876ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f9415400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 0x55e88876b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f9415400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 0x55e88876b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:47360/0 (socket says 172.21.15.90:47360) 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f9415400640 1 -- 172.21.15.90:0/3320284973 learned_addr learned my addr 172.21.15.90:0/3320284973 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.927+0000 7f9416800640 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e88876dcb0 0x55e88876b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f9415400640 1 -- 172.21.15.90:0/3320284973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 msgr2=0x55e88876ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f9415400640 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 0x55e88876ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f9415400640 1 -- 172.21.15.90:0/3320284973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e88876dcb0 msgr2=0x55e88876b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f9415400640 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e88876dcb0 0x55e88876b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f9415400640 1 -- 172.21.15.90:0/3320284973 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e888426540 con 0x55e888761d20 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f9415400640 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 0x55e88876b1d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f940400bad0 tx=0x7f940400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f862f3846f3f1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f93f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f93f7e00640 1 -- 172.21.15.90:0/3320284973 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404003e20 con 0x55e888761d20 2024-09-10T22:03:02.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f941fbd5740 1 -- 172.21.15.90:0/3320284973 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e888665a70 con 0x55e888761d20 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.928+0000 7f93f7e00640 1 -- 172.21.15.90:0/3320284973 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9404011440 con 0x55e888761d20 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.929+0000 7f93f7e00640 1 -- 172.21.15.90:0/3320284973 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9404011640 con 0x55e888761d20 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.929+0000 7f941fbd5740 10 client.35102 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.929+0000 7f941fbd5740 1 -- 172.21.15.90:0/3320284973 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e8886806f0 con 0x55e888761d20 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.929+0000 7f93f7e00640 1 -- 172.21.15.90:0/3320284973 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55e888665a70 con 0x55e888761d20 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.929+0000 7f93f7e00640 1 -- 172.21.15.90:0/3320284973 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f940401b8b0 con 0x55e888761d20 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.929+0000 7f941fbd5740 10 client.35102 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.930+0000 7f941fbd5740 10 client.35102 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.930+0000 7f941fbd5740 20 client.35102 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.930+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55e88877eb20 0x55e88877ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.930+0000 7f941fbd5740 4 client.35102 mds_command: new command op to 34646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.930+0000 7f941fbd5740 1 -- 172.21.15.90:0/3320284973 --> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8886daca0 con 0x55e88877eb20 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.930+0000 7f9416800640 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55e88877eb20 0x55e88877ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.931+0000 7f9416800640 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55e88877eb20 0x55e88877ff90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b0720ad15c8a871 server_cookie=844d1f6db5627167 in_seq=0 out_seq=0 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.931+0000 7f93f7e00640 10 client.35102 ms_handle_connect on v2:172.21.15.171:6834/2969925217 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f93f7e00640 1 -- 172.21.15.90:0/3320284973 <== mds.2 v2:172.21.15.171:6834/2969925217 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e8886daca0 con 0x55e88877eb20 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f93f7e00640 10 client.35102 handle_command_reply: tid=0 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 client.35102 shutdown 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 -- 172.21.15.90:0/3320284973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 msgr2=0x55e88876b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 0x55e88876b1d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f940400bad0 tx=0x7f940400bf90 comp rx=0 tx=0).stop 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 -- 172.21.15.90:0/3320284973 shutdown_connections 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55e88877eb20 0x55e88877ff90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e88876dcb0 0x55e88876b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761d20 0x55e88876b1d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f93f7e00640 0 client.35102 ms_handle_reset on v2:172.21.15.171:6834/2969925217 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3320284973 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 0x55e88876ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 -- 172.21.15.90:0/3320284973 >> 172.21.15.90:0/3320284973 conn(0x55e88875f230 msgr2=0x55e888764dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 -- 172.21.15.90:0/3320284973 shutdown_connections 2024-09-10T22:03:02.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.932+0000 7f941fbd5740 1 -- 172.21.15.90:0/3320284973 wait complete. 2024-09-10T22:03:02.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.933+0000 7f941fbd5740 20 client.35102 trim_cache size 0 max 16384 2024-09-10T22:03:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.951+0000 7f941fbd5740 1 Processor -- start 2024-09-10T22:03:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.951+0000 7f941fbd5740 1 -- start start 2024-09-10T22:03:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761400 0x55e8887655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888769510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e88876dcb0 0x55e888769a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f941fbd5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e88877e2d0 con 0x55e888761d20 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f941fbd5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e88877e9a0 con 0x55e888761400 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f941fbd5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e88877f070 con 0x55e88876dcb0 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f9415400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888769510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f9415400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888769510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47992/0 (socket says 172.21.15.90:47992) 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f9415400640 1 -- 172.21.15.90:0/3202614652 learned_addr learned my addr 172.21.15.90:0/3202614652 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f9416800640 1 --2- 172.21.15.90:0/3202614652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761400 0x55e8887655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f9415400640 1 -- 172.21.15.90:0/3202614652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e88876dcb0 msgr2=0x55e888769a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f9415400640 1 --2- 172.21.15.90:0/3202614652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e88876dcb0 0x55e888769a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.952+0000 7f9415400640 1 -- 172.21.15.90:0/3202614652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761400 msgr2=0x55e8887655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.953+0000 7f9415400640 1 --2- 172.21.15.90:0/3202614652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761400 0x55e8887655c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.953+0000 7f9415400640 1 -- 172.21.15.90:0/3202614652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88875fcb0 con 0x55e888761d20 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.953+0000 7f9416800640 1 --2- 172.21.15.90:0/3202614652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761400 0x55e8887655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.953+0000 7f9415400640 1 --2- 172.21.15.90:0/3202614652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888769510 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f940400baa0 tx=0x7f9404003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c348790208e25961 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.953+0000 7f9402a00640 1 -- 172.21.15.90:0/3202614652 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940402e070 con 0x55e888761d20 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.953+0000 7f9402a00640 1 -- 172.21.15.90:0/3202614652 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9404005500 con 0x55e888761d20 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.953+0000 7f9402a00640 1 -- 172.21.15.90:0/3202614652 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940401b420 con 0x55e888761d20 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 -- 172.21.15.90:0/3202614652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 msgr2=0x55e888769510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3202614652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888769510 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f940400baa0 tx=0x7f9404003a40 comp rx=0 tx=0).stop 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 -- 172.21.15.90:0/3202614652 shutdown_connections 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3202614652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e88876dcb0 0x55e888769a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3202614652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888769510 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3202614652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e888761400 0x55e8887655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 -- 172.21.15.90:0/3202614652 >> 172.21.15.90:0/3202614652 conn(0x55e88875f230 msgr2=0x55e888765f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 -- 172.21.15.90:0/3202614652 shutdown_connections 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 -- 172.21.15.90:0/3202614652 wait complete. 2024-09-10T22:03:02.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.954+0000 7f941fbd5740 1 -- start start 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f941fbd5740 1 Processor -- start 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 0x55e8887821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888782730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f941fbd5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e88876dcb0 0x55e888782c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f941fbd5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e88877fe10 con 0x55e888761d20 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f941fbd5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e88877f740 con 0x55e88876dcb0 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f941fbd5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e8887804e0 con 0x55e888761400 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f941fbd5740 10 client.0 _ll_register_callbacks cb 0x7f94040043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f9415400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888782730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f9415400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888782730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48008/0 (socket says 172.21.15.90:48008) 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.955+0000 7f9415400640 1 -- 172.21.15.90:0/3558196551 learned_addr learned my addr 172.21.15.90:0/3558196551 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f9416800640 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 0x55e8887821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f9416800640 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e88876dcb0 0x55e888782c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f9415400640 1 -- 172.21.15.90:0/3558196551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 msgr2=0x55e8887821f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f9415400640 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 0x55e8887821f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f9415400640 1 -- 172.21.15.90:0/3558196551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e88876dcb0 msgr2=0x55e888782c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f9415400640 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e88876dcb0 0x55e888782c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f9415400640 1 -- 172.21.15.90:0/3558196551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88875a8c0 con 0x55e888761d20 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f9416800640 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e88876dcb0 0x55e888782c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f9415400640 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888782730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f940400bad0 tx=0x7f9404011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffe742994ff8506 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.956+0000 7f93f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.957+0000 7f93f7e00640 1 -- 172.21.15.90:0/3558196551 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404037020 con 0x55e888761d20 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.957+0000 7f941fbd5740 1 -- 172.21.15.90:0/3558196551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9404005500 con 0x55e888761d20 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.957+0000 7f93f7e00640 1 -- 172.21.15.90:0/3558196551 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f940401b420 con 0x55e888761d20 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.957+0000 7f93f7e00640 1 -- 172.21.15.90:0/3558196551 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f940402e040 con 0x55e888761d20 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.957+0000 7f941fbd5740 10 client.15351 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:02.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.957+0000 7f941fbd5740 1 -- 172.21.15.90:0/3558196551 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e8887eb5b0 con 0x55e888761d20 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.957+0000 7f93f7e00640 1 -- 172.21.15.90:0/3558196551 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9404005500 con 0x55e888761d20 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.957+0000 7f93f7e00640 1 -- 172.21.15.90:0/3558196551 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9404033940 con 0x55e888761d20 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.957+0000 7f941fbd5740 10 client.15351 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.958+0000 7f941fbd5740 10 client.15351 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.958+0000 7f941fbd5740 20 client.15351 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.958+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55e8887efbd0 0x55e8887ee8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.958+0000 7f941fbd5740 4 client.15351 mds_command: new command op to 34646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.958+0000 7f941fbd5740 1 -- 172.21.15.90:0/3558196551 --> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] -- command(tid 0: {"prefix": "status"}) -- 0x55e8886d6750 con 0x55e8887efbd0 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.958+0000 7f9416800640 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55e8887efbd0 0x55e8887ee8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f9416800640 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55e8887efbd0 0x55e8887ee8e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e590f6e38a516d server_cookie=f43e4341f66e03bf in_seq=0 out_seq=0 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f93f7e00640 10 client.15351 ms_handle_connect on v2:172.21.15.171:6834/2969925217 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f93f7e00640 1 -- 172.21.15.90:0/3558196551 <== mds.2 v2:172.21.15.171:6834/2969925217 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e8886d6750 con 0x55e8887efbd0 2024-09-10T22:03:02.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f93f7e00640 10 client.15351 handle_command_reply: tid=0 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f941fbd5740 1 client.15351 shutdown 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f941fbd5740 1 -- 172.21.15.90:0/3558196551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 msgr2=0x55e888782730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888782730 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f940400bad0 tx=0x7f9404011b50 comp rx=0 tx=0).stop 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f941fbd5740 1 -- 172.21.15.90:0/3558196551 shutdown_connections 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55e8887efbd0 0x55e8887ee8e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e88876dcb0 0x55e888782c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e888761d20 0x55e888782730 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f941fbd5740 1 --2- 172.21.15.90:0/3558196551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e888761400 0x55e8887821f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f941fbd5740 1 -- 172.21.15.90:0/3558196551 >> 172.21.15.90:0/3558196551 conn(0x55e88875f230 msgr2=0x55e8887694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.959+0000 7f93f7e00640 0 client.15351 ms_handle_reset on v2:172.21.15.171:6834/2969925217 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.960+0000 7f941fbd5740 1 -- 172.21.15.90:0/3558196551 shutdown_connections 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.960+0000 7f941fbd5740 1 -- 172.21.15.90:0/3558196551 wait complete. 2024-09-10T22:03:02.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.960+0000 7f941fbd5740 20 client.15351 trim_cache size 0 max 16384 2024-09-10T22:03:02.972 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:02.972 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:03:02.972 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-09-10T22:03:02.972 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34646, 2024-09-10T22:03:02.972 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:03:02.972 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:03:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:03:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 93.245387829999999, 2024-09-10T22:03:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:03:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:03:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:03:02.973 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 182.04481934099999 2024-09-10T22:03:02.973 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.960+0000 7f9417200640 1 -- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93fc0b54e0 msgr2=0x7f93fc0b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.960+0000 7f9417200640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93fc0b54e0 0x7f93fc0b79a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f940803bfd0 tx=0x7f9408009260 comp rx=0 tx=0).stop 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.960+0000 7f9417200640 1 -- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 msgr2=0x7f941819b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.960+0000 7f9417200640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 0x7f941819b130 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f940802a930 tx=0x7f9408003c20 comp rx=0 tx=0).stop 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.961+0000 7f9417200640 1 -- 172.21.15.90:0/1568473585 shutdown_connections 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.961+0000 7f9417200640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f93fc0b54e0 0x7f93fc0b79a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.961+0000 7f9417200640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9418108fa0 0x7f94181a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.961+0000 7f9417200640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9418108620 0x7f941819b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.961+0000 7f9417200640 1 --2- 172.21.15.90:0/1568473585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9418102c10 0x7f941819b130 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:02.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.961+0000 7f9417200640 1 -- 172.21.15.90:0/1568473585 >> 172.21.15.90:0/1568473585 conn(0x7f94180fe660 msgr2=0x7f94181043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.961+0000 7f9417200640 1 -- 172.21.15.90:0/1568473585 shutdown_connections 2024-09-10T22:03:02.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:02.961+0000 7f9417200640 1 -- 172.21.15.90:0/1568473585 wait complete. 2024-09-10T22:03:02.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:02 smithi171 ceph-mon[34138]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:02 smithi154 ceph-mon[34495]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:03.796 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T22:03:04.193 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:04.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.793+0000 7fd631e00640 1 -- 172.21.15.90:0/859798898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c104df0 msgr2=0x7fd62c1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.793+0000 7fd631e00640 1 --2- 172.21.15.90:0/859798898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c104df0 0x7fd62c1051d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009ca0 tx=0x7fd61c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.793+0000 7fd631e00640 1 -- 172.21.15.90:0/859798898 shutdown_connections 2024-09-10T22:03:04.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.793+0000 7fd631e00640 1 --2- 172.21.15.90:0/859798898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62c10b180 0x7fd62c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.793+0000 7fd631e00640 1 --2- 172.21.15.90:0/859798898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd62c10a800 0x7fd62c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.793+0000 7fd631e00640 1 --2- 172.21.15.90:0/859798898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c104df0 0x7fd62c1051d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.793+0000 7fd631e00640 1 -- 172.21.15.90:0/859798898 >> 172.21.15.90:0/859798898 conn(0x7fd62c100840 msgr2=0x7fd62c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:04.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.793+0000 7fd631e00640 1 -- 172.21.15.90:0/859798898 shutdown_connections 2024-09-10T22:03:04.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.793+0000 7fd631e00640 1 -- 172.21.15.90:0/859798898 wait complete. 2024-09-10T22:03:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.794+0000 7fd631e00640 1 Processor -- start 2024-09-10T22:03:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.794+0000 7fd631e00640 1 -- start start 2024-09-10T22:03:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd631e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62c104df0 0x7fd62c19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd631e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c10a800 0x7fd62c19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd631e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd62c10b180 0x7fd62c1a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd631e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c114d00 con 0x7fd62c10a800 2024-09-10T22:03:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd631e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c114b80 con 0x7fd62c104df0 2024-09-10T22:03:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd631e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd62c114e80 con 0x7fd62c10b180 2024-09-10T22:03:04.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd62be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c10a800 0x7fd62c19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd62be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c10a800 0x7fd62c19b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48038/0 (socket says 172.21.15.90:48038) 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd62be00640 1 -- 172.21.15.90:0/74229847 learned_addr learned my addr 172.21.15.90:0/74229847 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.795+0000 7fd630a00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62c104df0 0x7fd62c19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd631400640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd62c10b180 0x7fd62c1a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd62be00640 1 -- 172.21.15.90:0/74229847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd62c10b180 msgr2=0x7fd62c1a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd62be00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd62c10b180 0x7fd62c1a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd62be00640 1 -- 172.21.15.90:0/74229847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62c104df0 msgr2=0x7fd62c19b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd62be00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62c104df0 0x7fd62c19b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd62be00640 1 -- 172.21.15.90:0/74229847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62c1a11c0 con 0x7fd62c10a800 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd630a00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62c104df0 0x7fd62c19b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd631400640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd62c10b180 0x7fd62c1a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd62be00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c10a800 0x7fd62c19b650 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd61400d950 tx=0x7fd61400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd629600640 1 -- 172.21.15.90:0/74229847 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614017070 con 0x7fd62c10a800 2024-09-10T22:03:04.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd629600640 1 -- 172.21.15.90:0/74229847 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd614009c30 con 0x7fd62c10a800 2024-09-10T22:03:04.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd629600640 1 -- 172.21.15.90:0/74229847 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61400f620 con 0x7fd62c10a800 2024-09-10T22:03:04.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.796+0000 7fd631e00640 1 -- 172.21.15.90:0/74229847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62c1a14b0 con 0x7fd62c10a800 2024-09-10T22:03:04.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.797+0000 7fd631e00640 1 -- 172.21.15.90:0/74229847 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62c1a7d40 con 0x7fd62c10a800 2024-09-10T22:03:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.800+0000 7fd635e01740 1 Processor -- start 2024-09-10T22:03:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.800+0000 7fd635e01740 1 -- start start 2024-09-10T22:03:04.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.800+0000 7fd629600640 1 -- 172.21.15.90:0/74229847 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7fd614010460 con 0x7fd62c10a800 2024-09-10T22:03:04.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.800+0000 7fd635e01740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fda87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.800+0000 7fd635e01740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdb4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.800+0000 7fd635e01740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdb70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.800+0000 7fd635e01740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56139fd1fd50 con 0x56139fda8400 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.800+0000 7fd635e01740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56139fd0f180 con 0x56139fdb4cb0 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.800+0000 7fd635e01740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56139fd12f90 con 0x56139fda8d20 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd631400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fda87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd631400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fda87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48054/0 (socket says 172.21.15.90:48054) 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd631400640 1 -- 172.21.15.90:0/1489113619 learned_addr learned my addr 172.21.15.90:0/1489113619 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd630a00640 1 --2- 172.21.15.90:0/1489113619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdb4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd62be00640 1 --2- 172.21.15.90:0/1489113619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdb70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd631400640 1 -- 172.21.15.90:0/1489113619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 msgr2=0x56139fdb4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd631400640 1 --2- 172.21.15.90:0/1489113619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdb4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd631400640 1 -- 172.21.15.90:0/1489113619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 msgr2=0x56139fdb70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:04.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd631400640 1 --2- 172.21.15.90:0/1489113619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdb70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd631400640 1 -- 172.21.15.90:0/1489113619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56139fcd9a70 con 0x56139fda8400 2024-09-10T22:03:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.801+0000 7fd631400640 1 --2- 172.21.15.90:0/1489113619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fda87e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd624009be0 tx=0x7fd624010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785901b3559b1c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.802+0000 7fd621c00640 1 -- 172.21.15.90:0/1489113619 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd624004020 con 0x56139fda8400 2024-09-10T22:03:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.802+0000 7fd621c00640 1 -- 172.21.15.90:0/1489113619 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6240041c0 con 0x56139fda8400 2024-09-10T22:03:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.802+0000 7fd621c00640 1 -- 172.21.15.90:0/1489113619 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6240126b0 con 0x56139fda8400 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.802+0000 7fd629600640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6100b54e0 0x7fd6100b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.802+0000 7fd629600640 1 -- 172.21.15.90:0/74229847 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd6140e5f20 con 0x7fd62c10a800 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.802+0000 7fd630a00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6100b54e0 0x7fd6100b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd635e01740 1 -- 172.21.15.90:0/1489113619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 msgr2=0x56139fda87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd635e01740 1 --2- 172.21.15.90:0/1489113619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fda87e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd624009be0 tx=0x7fd624010740 comp rx=0 tx=0).stop 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd635e01740 1 -- 172.21.15.90:0/1489113619 shutdown_connections 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd635e01740 1 --2- 172.21.15.90:0/1489113619 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdb70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd635e01740 1 --2- 172.21.15.90:0/1489113619 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdb4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd635e01740 1 --2- 172.21.15.90:0/1489113619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fda87e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd635e01740 1 -- 172.21.15.90:0/1489113619 >> 172.21.15.90:0/1489113619 conn(0x56139fda6230 msgr2=0x56139fda73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd635e01740 1 -- 172.21.15.90:0/1489113619 shutdown_connections 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd635e01740 1 -- 172.21.15.90:0/1489113619 wait complete. 2024-09-10T22:03:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.803+0000 7fd630a00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6100b54e0 0x7fd6100b79a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x56139fdab580 tx=0x7fd61c0098a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.804+0000 7fd635e01740 1 -- start start 2024-09-10T22:03:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.804+0000 7fd635e01740 1 Processor -- start 2024-09-10T22:03:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.804+0000 7fd635e01740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdb1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.804+0000 7fd635e01740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdb21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.804+0000 7fd635e01740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdb2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.804+0000 7fd635e01740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56139fb705f0 con 0x56139fda8400 2024-09-10T22:03:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.804+0000 7fd635e01740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56139fcaa460 con 0x56139fdb4cb0 2024-09-10T22:03:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.804+0000 7fd635e01740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56139fdb7a00 con 0x56139fda8d20 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.804+0000 7fd635e01740 10 client.0 _ll_register_callbacks cb 0x56139fcba210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd630a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdb1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd62be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdb21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd630a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdb1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48058/0 (socket says 172.21.15.90:48058) 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd630a00640 1 -- 172.21.15.90:0/2247637485 learned_addr learned my addr 172.21.15.90:0/2247637485 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd631400640 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdb2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd630a00640 1 -- 172.21.15.90:0/2247637485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 msgr2=0x56139fdb21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd630a00640 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdb21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd630a00640 1 -- 172.21.15.90:0/2247637485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 msgr2=0x56139fdb2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd630a00640 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdb2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd630a00640 1 -- 172.21.15.90:0/2247637485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56139fa6d540 con 0x56139fda8400 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd630a00640 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdb1c90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd61c003f20 tx=0x7fd61c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494de7400ddbea4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.805+0000 7fd61a000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.806+0000 7fd61a000640 1 -- 172.21.15.90:0/2247637485 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c031780 con 0x56139fda8400 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.806+0000 7fd61a000640 1 -- 172.21.15.90:0/2247637485 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd61c004250 con 0x56139fda8400 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.806+0000 7fd635e01740 1 -- 172.21.15.90:0/2247637485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56139fcaca70 con 0x56139fda8400 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.806+0000 7fd61a000640 1 -- 172.21.15.90:0/2247637485 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd61c004450 con 0x56139fda8400 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.806+0000 7fd635e01740 10 client.15363 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.806+0000 7fd635e01740 1 -- 172.21.15.90:0/2247637485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56139fcc76f0 con 0x56139fda8400 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.806+0000 7fd61a000640 1 -- 172.21.15.90:0/2247637485 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56139fcaca70 con 0x56139fda8400 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.807+0000 7fd61a000640 1 -- 172.21.15.90:0/2247637485 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd61c051020 con 0x56139fda8400 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.807+0000 7fd635e01740 10 client.15363 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.807+0000 7fd635e01740 10 client.15363 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.807+0000 7fd635e01740 20 client.15363 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.807+0000 7fd635e01740 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56139fdc5ae0 0x56139fdc6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.807+0000 7fd635e01740 4 client.15363 mds_command: new command op to 34670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:04.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.807+0000 7fd635e01740 1 -- 172.21.15.90:0/2247637485 --> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56139fd21ca0 con 0x56139fdc5ae0 2024-09-10T22:03:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.807+0000 7fd62be00640 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56139fdc5ae0 0x56139fdc6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.808+0000 7fd62be00640 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56139fdc5ae0 0x56139fdc6f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6332604373fc0d3 server_cookie=b493b700495aee10 in_seq=0 out_seq=0 2024-09-10T22:03:04.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.808+0000 7fd61a000640 10 client.15363 ms_handle_connect on v2:172.21.15.154:6836/3907253955 2024-09-10T22:03:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd61a000640 1 -- 172.21.15.90:0/2247637485 <== mds.1 v2:172.21.15.154:6836/3907253955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56139fd21ca0 con 0x56139fdc5ae0 2024-09-10T22:03:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd61a000640 10 client.15363 handle_command_reply: tid=0 2024-09-10T22:03:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd635e01740 1 client.15363 shutdown 2024-09-10T22:03:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd635e01740 1 -- 172.21.15.90:0/2247637485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 msgr2=0x56139fdb1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd635e01740 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdb1c90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd61c003f20 tx=0x7fd61c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd635e01740 1 -- 172.21.15.90:0/2247637485 shutdown_connections 2024-09-10T22:03:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd635e01740 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56139fdc5ae0 0x56139fdc6f50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd635e01740 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdb2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd635e01740 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdb21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd635e01740 1 --2- 172.21.15.90:0/2247637485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdb1c90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd635e01740 1 -- 172.21.15.90:0/2247637485 >> 172.21.15.90:0/2247637485 conn(0x56139fda6230 msgr2=0x56139fdabdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.809+0000 7fd61a000640 0 client.15363 ms_handle_reset on v2:172.21.15.154:6836/3907253955 2024-09-10T22:03:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.810+0000 7fd635e01740 1 -- 172.21.15.90:0/2247637485 shutdown_connections 2024-09-10T22:03:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.810+0000 7fd635e01740 1 -- 172.21.15.90:0/2247637485 wait complete. 2024-09-10T22:03:04.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.810+0000 7fd635e01740 20 client.15363 trim_cache size 0 max 16384 2024-09-10T22:03:04.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.828+0000 7fd635e01740 1 Processor -- start 2024-09-10T22:03:04.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.828+0000 7fd635e01740 1 -- start start 2024-09-10T22:03:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd635e01740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fda8400 0x56139fdc6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd635e01740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8d20 0x56139fdac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd635e01740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fdb4cb0 0x56139fdb0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd635e01740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56139fdc5290 con 0x56139fda8d20 2024-09-10T22:03:04.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd635e01740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56139fdc5960 con 0x56139fda8400 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd635e01740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56139fdc6030 con 0x56139fdb4cb0 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd62be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8d20 0x56139fdac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd62be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8d20 0x56139fdac5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48060/0 (socket says 172.21.15.90:48060) 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd62be00640 1 -- 172.21.15.90:0/1116366197 learned_addr learned my addr 172.21.15.90:0/1116366197 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd630a00640 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fda8400 0x56139fdc6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd631400640 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fdb4cb0 0x56139fdb0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd62be00640 1 -- 172.21.15.90:0/1116366197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fdb4cb0 msgr2=0x56139fdb0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd62be00640 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fdb4cb0 0x56139fdb0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd62be00640 1 -- 172.21.15.90:0/1116366197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fda8400 msgr2=0x56139fdc6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd62be00640 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fda8400 0x56139fdc6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd62be00640 1 -- 172.21.15.90:0/1116366197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56139fda8b50 con 0x56139fda8d20 2024-09-10T22:03:04.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd631400640 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fdb4cb0 0x56139fdb0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:04.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.829+0000 7fd630a00640 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fda8400 0x56139fdc6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:03:04.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.830+0000 7fd62be00640 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8d20 0x56139fdac5c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd6140ac020 tx=0x7fd6140e8370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5911c5f6d772005c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.830+0000 7fd621200640 1 -- 172.21.15.90:0/1116366197 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614020040 con 0x56139fda8d20 2024-09-10T22:03:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.830+0000 7fd621200640 1 -- 172.21.15.90:0/1116366197 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6140ac790 con 0x56139fda8d20 2024-09-10T22:03:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.830+0000 7fd621200640 1 -- 172.21.15.90:0/1116366197 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614091e60 con 0x56139fda8d20 2024-09-10T22:03:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 -- 172.21.15.90:0/1116366197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8d20 msgr2=0x56139fdac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8d20 0x56139fdac5c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fd6140ac020 tx=0x7fd6140e8370 comp rx=0 tx=0).stop 2024-09-10T22:03:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 -- 172.21.15.90:0/1116366197 shutdown_connections 2024-09-10T22:03:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fdb4cb0 0x56139fdb0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8d20 0x56139fdac5c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 --2- 172.21.15.90:0/1116366197 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fda8400 0x56139fdc6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 -- 172.21.15.90:0/1116366197 >> 172.21.15.90:0/1116366197 conn(0x56139fda6230 msgr2=0x56139fdacee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 -- 172.21.15.90:0/1116366197 shutdown_connections 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 -- 172.21.15.90:0/1116366197 wait complete. 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 -- start start 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.831+0000 7fd635e01740 1 Processor -- start 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd635e01740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdc9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd635e01740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdc9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd635e01740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd635e01740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56139fdc6dd0 con 0x56139fda8400 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd635e01740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56139fdc6700 con 0x56139fdb4cb0 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd635e01740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56139fdc74a0 con 0x56139fda8d20 2024-09-10T22:03:04.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd635e01740 10 client.0 _ll_register_callbacks cb 0x7fd61c003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd630a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdc9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd630a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdc9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48068/0 (socket says 172.21.15.90:48068) 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd630a00640 1 -- 172.21.15.90:0/941020913 learned_addr learned my addr 172.21.15.90:0/941020913 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.832+0000 7fd62be00640 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdc9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd631400640 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd630a00640 1 -- 172.21.15.90:0/941020913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 msgr2=0x56139fdc9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd630a00640 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdc9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd630a00640 1 -- 172.21.15.90:0/941020913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 msgr2=0x56139fdceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd630a00640 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd630a00640 1 -- 172.21.15.90:0/941020913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56139fdcf510 con 0x56139fda8400 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd631400640 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd630a00640 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdc9920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd61c030e30 tx=0x7fd61c004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a559427978d41a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:04.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd61a000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd61a000640 1 -- 172.21.15.90:0/941020913 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c031780 con 0x56139fda8400 2024-09-10T22:03:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd61a000640 1 -- 172.21.15.90:0/941020913 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd61c0307d0 con 0x56139fda8400 2024-09-10T22:03:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.833+0000 7fd635e01740 1 -- 172.21.15.90:0/941020913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6140ac790 con 0x56139fda8400 2024-09-10T22:03:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd61a000640 1 -- 172.21.15.90:0/941020913 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd61c0309d0 con 0x56139fda8400 2024-09-10T22:03:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd635e01740 10 client.15369 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:04.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd635e01740 1 -- 172.21.15.90:0/941020913 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56139fe32d60 con 0x56139fda8400 2024-09-10T22:03:04.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd61a000640 1 -- 172.21.15.90:0/941020913 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd6140ac790 con 0x56139fda8400 2024-09-10T22:03:04.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd61a000640 1 -- 172.21.15.90:0/941020913 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd61c051070 con 0x56139fda8400 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd635e01740 10 client.15369 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd635e01740 10 client.15369 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd635e01740 20 client.15369 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd635e01740 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56139fe37820 0x56139fe36530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd635e01740 4 client.15369 mds_command: new command op to 34670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.834+0000 7fd635e01740 1 -- 172.21.15.90:0/941020913 --> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] -- command(tid 0: {"prefix": "status"}) -- 0x56139fd1d750 con 0x56139fe37820 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.835+0000 7fd62be00640 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56139fe37820 0x56139fe36530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.835+0000 7fd62be00640 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56139fe37820 0x56139fe36530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b73517b1a07b76ec server_cookie=5cd6911a2247e6fd in_seq=0 out_seq=0 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.835+0000 7fd61a000640 10 client.15369 ms_handle_connect on v2:172.21.15.154:6836/3907253955 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd61a000640 1 -- 172.21.15.90:0/941020913 <== mds.1 v2:172.21.15.154:6836/3907253955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56139fd1d750 con 0x56139fe37820 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd61a000640 10 client.15369 handle_command_reply: tid=0 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 client.15369 shutdown 2024-09-10T22:03:04.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 -- 172.21.15.90:0/941020913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 msgr2=0x56139fdc9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdc9920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd61c030e30 tx=0x7fd61c004820 comp rx=0 tx=0).stop 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 -- 172.21.15.90:0/941020913 shutdown_connections 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56139fe37820 0x56139fe36530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56139fdb4cb0 0x56139fdceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56139fda8d20 0x56139fdc9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 --2- 172.21.15.90:0/941020913 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56139fda8400 0x56139fdc9920 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 -- 172.21.15.90:0/941020913 >> 172.21.15.90:0/941020913 conn(0x56139fda6230 msgr2=0x56139fdb00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd61a000640 0 client.15369 ms_handle_reset on v2:172.21.15.154:6836/3907253955 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 -- 172.21.15.90:0/941020913 shutdown_connections 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 1 -- 172.21.15.90:0/941020913 wait complete. 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.836+0000 7fd635e01740 20 client.15369 trim_cache size 0 max 16384 2024-09-10T22:03:04.841 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34670, 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 96.137408144000005, 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 177.246154641 2024-09-10T22:03:04.842 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 -- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6100b54e0 msgr2=0x7fd6100b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6100b54e0 0x7fd6100b79a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x56139fdab580 tx=0x7fd61c0098a0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 -- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c10a800 msgr2=0x7fd62c19b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c10a800 0x7fd62c19b650 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd61400d950 tx=0x7fd61400de10 comp rx=0 tx=0).stop 2024-09-10T22:03:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 -- 172.21.15.90:0/74229847 shutdown_connections 2024-09-10T22:03:04.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd6100b54e0 0x7fd6100b79a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd62c10b180 0x7fd62c1a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd62c10a800 0x7fd62c19b650 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 --2- 172.21.15.90:0/74229847 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd62c104df0 0x7fd62c19b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 -- 172.21.15.90:0/74229847 >> 172.21.15.90:0/74229847 conn(0x7fd62c100840 msgr2=0x7fd62c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 -- 172.21.15.90:0/74229847 shutdown_connections 2024-09-10T22:03:04.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:04.837+0000 7fd631e00640 1 -- 172.21.15.90:0/74229847 wait complete. 2024-09-10T22:03:04.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:04 smithi171 ceph-mon[34138]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:04 smithi090 ceph-mon[30350]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:04 smithi154 ceph-mon[34495]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:05.646 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:03:06.017 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:06.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.599+0000 7f04b5800640 1 -- 172.21.15.90:0/622189300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b010cf70 msgr2=0x7f04b010f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.599+0000 7f04b5800640 1 --2- 172.21.15.90:0/622189300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b010cf70 0x7f04b010f3d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009900 tx=0x7f04a802a410 comp rx=0 tx=0).stop 2024-09-10T22:03:06.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.599+0000 7f04b5800640 1 -- 172.21.15.90:0/622189300 shutdown_connections 2024-09-10T22:03:06.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.599+0000 7f04b5800640 1 --2- 172.21.15.90:0/622189300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b010cf70 0x7f04b010f3d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.599+0000 7f04b5800640 1 --2- 172.21.15.90:0/622189300 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04b0104430 0x7f04b010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.599+0000 7f04b5800640 1 --2- 172.21.15.90:0/622189300 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04b0103b10 0x7f04b0103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.599+0000 7f04b5800640 1 -- 172.21.15.90:0/622189300 >> 172.21.15.90:0/622189300 conn(0x7f04b00fd5b0 msgr2=0x7f04b00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.599+0000 7f04b5800640 1 -- 172.21.15.90:0/622189300 shutdown_connections 2024-09-10T22:03:06.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.599+0000 7f04b5800640 1 -- 172.21.15.90:0/622189300 wait complete. 2024-09-10T22:03:06.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.600+0000 7f04b5800640 1 Processor -- start 2024-09-10T22:03:06.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.600+0000 7f04b5800640 1 -- start start 2024-09-10T22:03:06.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04b5800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b0103b10 0x7f04b019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04b5800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04b0104430 0x7f04b019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04b5800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04b010cf70 0x7f04b01a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04b5800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0112ae0 con 0x7f04b0103b10 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04b5800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0112960 con 0x7f04b0104430 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04b5800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0112c60 con 0x7f04b010cf70 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b0103b10 0x7f04b019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b0103b10 0x7f04b019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48084/0 (socket says 172.21.15.90:48084) 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04afe00640 1 -- 172.21.15.90:0/4255435444 learned_addr learned my addr 172.21.15.90:0/4255435444 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04b4e00640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04b010cf70 0x7f04b01a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04a7400640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04b0104430 0x7f04b019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04afe00640 1 -- 172.21.15.90:0/4255435444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04b010cf70 msgr2=0x7f04b01a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04afe00640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04b010cf70 0x7f04b01a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04afe00640 1 -- 172.21.15.90:0/4255435444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04b0104430 msgr2=0x7f04b019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04afe00640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04b0104430 0x7f04b019b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.601+0000 7f04afe00640 1 -- 172.21.15.90:0/4255435444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b01a11d0 con 0x7f04b0103b10 2024-09-10T22:03:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.602+0000 7f04afe00640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b0103b10 0x7f04b019b120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f04a000ea80 tx=0x7f04a000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.602+0000 7f04ad600640 1 -- 172.21.15.90:0/4255435444 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a000ce40 con 0x7f04b0103b10 2024-09-10T22:03:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.602+0000 7f04ad600640 1 -- 172.21.15.90:0/4255435444 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f04a0005020 con 0x7f04b0103b10 2024-09-10T22:03:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.602+0000 7f04b5800640 1 -- 172.21.15.90:0/4255435444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b01a14c0 con 0x7f04b0103b10 2024-09-10T22:03:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.602+0000 7f04ad600640 1 -- 172.21.15.90:0/4255435444 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a00116d0 con 0x7f04b0103b10 2024-09-10T22:03:06.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.602+0000 7f04b5800640 1 -- 172.21.15.90:0/4255435444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b01a7b80 con 0x7f04b0103b10 2024-09-10T22:03:06.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.605+0000 7f04b99c3740 1 Processor -- start 2024-09-10T22:03:06.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04b99c3740 1 -- start start 2024-09-10T22:03:06.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3b400 0x556c16c3b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04ad600640 1 -- 172.21.15.90:0/4255435444 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 153318+0+0 (secure 0 0 0) 0x7f04a000f040 con 0x7f04b0103b10 2024-09-10T22:03:06.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 0x556c16c47770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c47cb0 0x556c16c4a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04b99c3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c16bb2d50 con 0x556c16c47cb0 2024-09-10T22:03:06.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04b99c3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556c16ba2180 con 0x556c16c3bd20 2024-09-10T22:03:06.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04b99c3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556c16ba5f90 con 0x556c16c3b400 2024-09-10T22:03:06.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04a7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c47cb0 0x556c16c4a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04a7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c47cb0 0x556c16c4a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48086/0 (socket says 172.21.15.90:48086) 2024-09-10T22:03:06.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04a7400640 1 -- 172.21.15.90:0/3178799244 learned_addr learned my addr 172.21.15.90:0/3178799244 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:06.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.606+0000 7f04afe00640 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 0x556c16c47770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.607+0000 7f04b4e00640 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3b400 0x556c16c3b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.607+0000 7f04a7400640 1 -- 172.21.15.90:0/3178799244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3b400 msgr2=0x556c16c3b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.607+0000 7f04a7400640 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3b400 0x556c16c3b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.607+0000 7f04a7400640 1 -- 172.21.15.90:0/3178799244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 msgr2=0x556c16c47770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.607+0000 7f04a7400640 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 0x556c16c47770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.607+0000 7f04a7400640 1 -- 172.21.15.90:0/3178799244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c16b6ca70 con 0x556c16c47cb0 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.607+0000 7f04b4e00640 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3b400 0x556c16c3b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.607+0000 7f04a7400640 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c47cb0 0x556c16c4a0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0498004610 tx=0x7f049800bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcbb0b4a44f496 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.607+0000 7f04a5600640 1 -- 172.21.15.90:0/3178799244 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498003dd0 con 0x556c16c47cb0 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04a5600640 1 -- 172.21.15.90:0/3178799244 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0498015070 con 0x556c16c47cb0 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04a5600640 1 -- 172.21.15.90:0/3178799244 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04980116a0 con 0x556c16c47cb0 2024-09-10T22:03:06.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04ad600640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04940b5460 0x7f04940b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04ad600640 1 -- 172.21.15.90:0/4255435444 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f04a00e6490 con 0x7f04b0103b10 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04b4e00640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04940b5460 0x7f04940b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04b99c3740 1 -- 172.21.15.90:0/3178799244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c47cb0 msgr2=0x556c16c4a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04b99c3740 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c47cb0 0x556c16c4a0a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0498004610 tx=0x7f049800bbd0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04b99c3740 1 -- 172.21.15.90:0/3178799244 shutdown_connections 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04b99c3740 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c47cb0 0x556c16c4a0a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.608+0000 7f04b99c3740 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 0x556c16c47770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.609+0000 7f04b99c3740 1 --2- 172.21.15.90:0/3178799244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3b400 0x556c16c3b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.609+0000 7f04b99c3740 1 -- 172.21.15.90:0/3178799244 >> 172.21.15.90:0/3178799244 conn(0x556c16c39230 msgr2=0x556c16c3a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.609+0000 7f04b99c3740 1 -- 172.21.15.90:0/3178799244 shutdown_connections 2024-09-10T22:03:06.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.609+0000 7f04b99c3740 1 -- 172.21.15.90:0/3178799244 wait complete. 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.609+0000 7f04b4e00640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04940b5460 0x7f04940b7920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f04a8004770 tx=0x7f04a80046c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04b99c3740 1 -- start start 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04b99c3740 1 Processor -- start 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c44ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3bd20 0x556c16c451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c47cb0 0x556c16c45720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04b99c3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c16a035f0 con 0x556c16c3b400 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04b99c3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556c16b3d460 con 0x556c16c47cb0 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04b99c3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556c16c4aa00 con 0x556c16c3bd20 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04b99c3740 10 client.0 _ll_register_callbacks cb 0x556c16b4d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.610+0000 7f04afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c44ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c44ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48094/0 (socket says 172.21.15.90:48094) 2024-09-10T22:03:06.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04afe00640 1 -- 172.21.15.90:0/2825540424 learned_addr learned my addr 172.21.15.90:0/2825540424 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:06.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04a7400640 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3bd20 0x556c16c451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04b4e00640 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c47cb0 0x556c16c45720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04afe00640 1 -- 172.21.15.90:0/2825540424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3bd20 msgr2=0x556c16c451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:06.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04afe00640 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3bd20 0x556c16c451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04afe00640 1 -- 172.21.15.90:0/2825540424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c47cb0 msgr2=0x556c16c45720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:06.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04afe00640 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c47cb0 0x556c16c45720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04afe00640 1 -- 172.21.15.90:0/2825540424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c16900540 con 0x556c16c3b400 2024-09-10T22:03:06.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04afe00640 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c44ca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f04a00a9920 tx=0x7f04a00a8df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6809c76914aa67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f049e000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f049e000640 1 -- 172.21.15.90:0/2825540424 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a001f040 con 0x556c16c3b400 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f049e000640 1 -- 172.21.15.90:0/2825540424 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f04a00e3ba0 con 0x556c16c3b400 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.611+0000 7f04b99c3740 1 -- 172.21.15.90:0/2825540424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c16b3fa70 con 0x556c16c3b400 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f049e000640 1 -- 172.21.15.90:0/2825540424 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f04a00abe20 con 0x556c16c3b400 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f04b99c3740 10 client.15381 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f04b99c3740 1 -- 172.21.15.90:0/2825540424 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c16b5a6f0 con 0x556c16c3b400 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f049e000640 1 -- 172.21.15.90:0/2825540424 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556c16b3fa70 con 0x556c16c3b400 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f049e000640 1 -- 172.21.15.90:0/2825540424 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f04a00b5700 con 0x556c16c3b400 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f04b99c3740 10 client.15381 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f04b99c3740 10 client.15381 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f04b99c3740 20 client.15381 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:06.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f04b99c3740 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556c16c589e0 0x556c16c59e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f04b99c3740 4 client.15381 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.612+0000 7f04b99c3740 1 -- 172.21.15.90:0/2825540424 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c16bb4ca0 con 0x556c16c589e0 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.613+0000 7f04a7400640 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556c16c589e0 0x556c16c59e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.613+0000 7f04a7400640 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556c16c589e0 0x556c16c59e50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336d61d8d6b10bc7 server_cookie=d7b64793215498e5 in_seq=0 out_seq=0 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.613+0000 7f049e000640 10 client.15381 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.614+0000 7f049e000640 1 -- 172.21.15.90:0/2825540424 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c16bb4ca0 con 0x556c16c589e0 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.614+0000 7f049e000640 10 client.15381 handle_command_reply: tid=0 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.614+0000 7f04b99c3740 1 client.15381 shutdown 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 -- 172.21.15.90:0/2825540424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 msgr2=0x556c16c44ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c44ca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f04a00a9920 tx=0x7f04a00a8df0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 -- 172.21.15.90:0/2825540424 shutdown_connections 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556c16c589e0 0x556c16c59e50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c47cb0 0x556c16c45720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c3bd20 0x556c16c451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 --2- 172.21.15.90:0/2825540424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c44ca0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f049e000640 0 client.15381 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:03:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 -- 172.21.15.90:0/2825540424 >> 172.21.15.90:0/2825540424 conn(0x556c16c39230 msgr2=0x556c16c3edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 -- 172.21.15.90:0/2825540424 shutdown_connections 2024-09-10T22:03:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 1 -- 172.21.15.90:0/2825540424 wait complete. 2024-09-10T22:03:06.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.615+0000 7f04b99c3740 20 client.15381 trim_cache size 0 max 16384 2024-09-10T22:03:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.633+0000 7f04b99c3740 1 Processor -- start 2024-09-10T22:03:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04b99c3740 1 -- start start 2024-09-10T22:03:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3b400 0x556c16c3f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3bd20 0x556c16c3a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 0x556c16c41eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04b99c3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c16c58190 con 0x556c16c3bd20 2024-09-10T22:03:06.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04b99c3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556c16c58860 con 0x556c16c3b400 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04b99c3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556c16c58f30 con 0x556c16c47cb0 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04a7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3bd20 0x556c16c3a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04a7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3bd20 0x556c16c3a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48098/0 (socket says 172.21.15.90:48098) 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04a7400640 1 -- 172.21.15.90:0/3288587798 learned_addr learned my addr 172.21.15.90:0/3288587798 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04afe00640 1 --2- 172.21.15.90:0/3288587798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3b400 0x556c16c3f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.634+0000 7f04b4e00640 1 --2- 172.21.15.90:0/3288587798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 0x556c16c41eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.635+0000 7f04a7400640 1 -- 172.21.15.90:0/3288587798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 msgr2=0x556c16c41eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.635+0000 7f04a7400640 1 --2- 172.21.15.90:0/3288587798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 0x556c16c41eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.635+0000 7f04a7400640 1 -- 172.21.15.90:0/3288587798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3b400 msgr2=0x556c16c3f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.635+0000 7f04a7400640 1 --2- 172.21.15.90:0/3288587798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3b400 0x556c16c3f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.635+0000 7f04a7400640 1 -- 172.21.15.90:0/3288587798 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c16c39cb0 con 0x556c16c3bd20 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.635+0000 7f04a7400640 1 --2- 172.21.15.90:0/3288587798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3bd20 0x556c16c3a620 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0498005cc0 tx=0x7f0498004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126c4beabe29d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.635+0000 7f04a4c00640 1 -- 172.21.15.90:0/3288587798 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498031070 con 0x556c16c3bd20 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.635+0000 7f04a4c00640 1 -- 172.21.15.90:0/3288587798 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0498015040 con 0x556c16c3bd20 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.635+0000 7f04a4c00640 1 -- 172.21.15.90:0/3288587798 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049801a6d0 con 0x556c16c3bd20 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 -- 172.21.15.90:0/3288587798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3bd20 msgr2=0x556c16c3a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 --2- 172.21.15.90:0/3288587798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3bd20 0x556c16c3a620 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0498005cc0 tx=0x7f0498004560 comp rx=0 tx=0).stop 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 -- 172.21.15.90:0/3288587798 shutdown_connections 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 --2- 172.21.15.90:0/3288587798 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 0x556c16c41eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 --2- 172.21.15.90:0/3288587798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3bd20 0x556c16c3a620 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 --2- 172.21.15.90:0/3288587798 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3b400 0x556c16c3f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 -- 172.21.15.90:0/3288587798 >> 172.21.15.90:0/3288587798 conn(0x556c16c39230 msgr2=0x556c16c39f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 -- 172.21.15.90:0/3288587798 shutdown_connections 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 -- 172.21.15.90:0/3288587798 wait complete. 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.636+0000 7f04b99c3740 1 -- start start 2024-09-10T22:03:06.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04b99c3740 1 Processor -- start 2024-09-10T22:03:06.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c5c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 0x556c16c5c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04b99c3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 0x556c16c5cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04b99c3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556c16c59cd0 con 0x556c16c3b400 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04b99c3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556c16c59600 con 0x556c16c3bd20 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04b99c3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556c16c5a3a0 con 0x556c16c47cb0 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04b99c3740 10 client.0 _ll_register_callbacks cb 0x7f04a00e9c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c5c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04afe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c5c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48108/0 (socket says 172.21.15.90:48108) 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04afe00640 1 -- 172.21.15.90:0/1303911234 learned_addr learned my addr 172.21.15.90:0/1303911234 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.637+0000 7f04b4e00640 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 0x556c16c5cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04a7400640 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 0x556c16c5c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04afe00640 1 -- 172.21.15.90:0/1303911234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 msgr2=0x556c16c5cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04afe00640 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 0x556c16c5cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04afe00640 1 -- 172.21.15.90:0/1303911234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 msgr2=0x556c16c5c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:06.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04afe00640 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 0x556c16c5c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04afe00640 1 -- 172.21.15.90:0/1303911234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c16c61d20 con 0x556c16c3b400 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04b4e00640 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 0x556c16c5cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04a7400640 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 0x556c16c5c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04afe00640 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c5c2d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f04a00b5a70 tx=0x7f04a00e6390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c94ceca690a98a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f049e000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f049e000640 1 -- 172.21.15.90:0/1303911234 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a001f070 con 0x556c16c3b400 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f049e000640 1 -- 172.21.15.90:0/1303911234 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f04a00ab910 con 0x556c16c3b400 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04b99c3740 1 -- 172.21.15.90:0/1303911234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0498015040 con 0x556c16c3b400 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f049e000640 1 -- 172.21.15.90:0/1303911234 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f04a00ac480 con 0x556c16c3b400 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04b99c3740 10 client.15387 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.638+0000 7f04b99c3740 1 -- 172.21.15.90:0/1303911234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c16cc5910 con 0x556c16c3b400 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.639+0000 7f049e000640 1 -- 172.21.15.90:0/1303911234 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0498015040 con 0x556c16c3b400 2024-09-10T22:03:06.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.639+0000 7f049e000640 1 -- 172.21.15.90:0/1303911234 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f04a00b2270 con 0x556c16c3b400 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.639+0000 7f04b99c3740 10 client.15387 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.639+0000 7f04b99c3740 10 client.15387 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.639+0000 7f04b99c3740 20 client.15387 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.639+0000 7f04b99c3740 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556c16cca580 0x556c16cc9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.639+0000 7f04b99c3740 4 client.15387 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.639+0000 7f04b99c3740 1 -- 172.21.15.90:0/1303911234 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "status"}) -- 0x556c16bb0750 con 0x556c16cca580 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.640+0000 7f04a7400640 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556c16cca580 0x556c16cc9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.640+0000 7f049e000640 10 client.15387 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.640+0000 7f04a7400640 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556c16cca580 0x556c16cc9290 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74d471187b8911 server_cookie=929ee5d8b18e9c05 in_seq=0 out_seq=0 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f049e000640 1 -- 172.21.15.90:0/1303911234 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556c16bb0750 con 0x556c16cca580 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f049e000640 10 client.15387 handle_command_reply: tid=0 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 client.15387 shutdown 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 -- 172.21.15.90:0/1303911234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 msgr2=0x556c16c5c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c5c2d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f04a00b5a70 tx=0x7f04a00e6390 comp rx=0 tx=0).stop 2024-09-10T22:03:06.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 -- 172.21.15.90:0/1303911234 shutdown_connections 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556c16cca580 0x556c16cc9290 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556c16c47cb0 0x556c16c5cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556c16c3bd20 0x556c16c5c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f049e000640 0 client.15387 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 --2- 172.21.15.90:0/1303911234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556c16c3b400 0x556c16c5c2d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 -- 172.21.15.90:0/1303911234 >> 172.21.15.90:0/1303911234 conn(0x556c16c39230 msgr2=0x556c16c534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 -- 172.21.15.90:0/1303911234 shutdown_connections 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 1 -- 172.21.15.90:0/1303911234 wait complete. 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.641+0000 7f04b99c3740 20 client.15387 trim_cache size 0 max 16384 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34703, 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 150.14680198299999, 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 163.15863729200001 2024-09-10T22:03:06.648 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 -- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04940b5460 msgr2=0x7f04940b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04940b5460 0x7f04940b7920 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f04a8004770 tx=0x7f04a80046c0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 -- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b0103b10 msgr2=0x7f04b019b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b0103b10 0x7f04b019b120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f04a000ea80 tx=0x7f04a000ef40 comp rx=0 tx=0).stop 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 -- 172.21.15.90:0/4255435444 shutdown_connections 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f04940b5460 0x7f04940b7920 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f04b010cf70 0x7f04b01a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f04b0104430 0x7f04b019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 --2- 172.21.15.90:0/4255435444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04b0103b10 0x7f04b019b120 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:06.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 -- 172.21.15.90:0/4255435444 >> 172.21.15.90:0/4255435444 conn(0x7f04b00fd5b0 msgr2=0x7f04b00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:06.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 -- 172.21.15.90:0/4255435444 shutdown_connections 2024-09-10T22:03:06.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:06.642+0000 7f04b5800640 1 -- 172.21.15.90:0/4255435444 wait complete. 2024-09-10T22:03:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:06 smithi090 ceph-mon[30350]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T22:03:06.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:06 smithi171 ceph-mon[34138]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T22:03:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:06 smithi154 ceph-mon[34495]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-10T22:03:07.599 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T22:03:07.599 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T22:03:07.611 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-09-10T22:03:07.611 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T22:03:07.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:07 smithi090 ceph-mon[30350]: mgrmap e22: x(active, since 10m), standbys: z, y 2024-09-10T22:03:07.986 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:07.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:07 smithi171 ceph-mon[34138]: mgrmap e22: x(active, since 10m), standbys: z, y 2024-09-10T22:03:08.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:07 smithi154 ceph-mon[34495]: mgrmap e22: x(active, since 10m), standbys: z, y 2024-09-10T22:03:08.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.781+0000 7f9bd8a00640 1 -- 172.21.15.90:0/44742141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd410b330 msgr2=0x7f9bd4111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.781+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/44742141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd410b330 0x7f9bd4111b30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc009ca0 tx=0x7f9bcc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.781+0000 7f9bd8a00640 1 -- 172.21.15.90:0/44742141 shutdown_connections 2024-09-10T22:03:08.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.781+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/44742141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd410b330 0x7f9bd4111b30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.781+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/44742141 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bd410a9b0 0x7f9bd410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.781+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/44742141 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bd4104fa0 0x7f9bd4105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.781+0000 7f9bd8a00640 1 -- 172.21.15.90:0/44742141 >> 172.21.15.90:0/44742141 conn(0x7f9bd4100a50 msgr2=0x7f9bd4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:08.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.781+0000 7f9bd8a00640 1 -- 172.21.15.90:0/44742141 shutdown_connections 2024-09-10T22:03:08.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.781+0000 7f9bd8a00640 1 -- 172.21.15.90:0/44742141 wait complete. 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.782+0000 7f9bd8a00640 1 Processor -- start 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.782+0000 7f9bd8a00640 1 -- start start 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd4104fa0 0x7f9bd419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd8a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bd410a9b0 0x7f9bd419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd8a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bd410b330 0x7f9bd41a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd8a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4114e60 con 0x7f9bd4104fa0 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd8a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4114ce0 con 0x7f9bd410a9b0 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd8a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4114fe0 con 0x7f9bd410b330 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd4104fa0 0x7f9bd419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd4104fa0 0x7f9bd419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54130/0 (socket says 172.21.15.90:54130) 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1e00640 1 -- 172.21.15.90:0/2288257019 learned_addr learned my addr 172.21.15.90:0/2288257019 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1400640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bd410a9b0 0x7f9bd419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:08.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd2800640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bd410b330 0x7f9bd41a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1e00640 1 -- 172.21.15.90:0/2288257019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bd410b330 msgr2=0x7f9bd41a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1e00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bd410b330 0x7f9bd41a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1e00640 1 -- 172.21.15.90:0/2288257019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bd410a9b0 msgr2=0x7f9bd419b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1e00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bd410a9b0 0x7f9bd419b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1e00640 1 -- 172.21.15.90:0/2288257019 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd41a14e0 con 0x7f9bd4104fa0 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.783+0000 7f9bd1e00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd4104fa0 0x7f9bd419b430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400d770 tx=0x7f9bc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.784+0000 7f9bcaa00640 1 -- 172.21.15.90:0/2288257019 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4017070 con 0x7f9bd4104fa0 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.784+0000 7f9bcaa00640 1 -- 172.21.15.90:0/2288257019 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9bc40054b0 con 0x7f9bd4104fa0 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.784+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd41a17d0 con 0x7f9bd4104fa0 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.784+0000 7f9bcaa00640 1 -- 172.21.15.90:0/2288257019 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4004c60 con 0x7f9bd4104fa0 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.784+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd41a7e90 con 0x7f9bd4104fa0 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.785+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd4105380 con 0x7f9bd4104fa0 2024-09-10T22:03:08.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.787+0000 7f9bcaa00640 1 -- 172.21.15.90:0/2288257019 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f9bc4004e00 con 0x7f9bd4104fa0 2024-09-10T22:03:08.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.788+0000 7f9bcaa00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9bb80b53b0 0x7f9bb80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:08.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.789+0000 7f9bcaa00640 1 -- 172.21.15.90:0/2288257019 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9bc40e5be0 con 0x7f9bd4104fa0 2024-09-10T22:03:08.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.789+0000 7f9bd1400640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9bb80b53b0 0x7f9bb80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:08.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.789+0000 7f9bd1400640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9bb80b53b0 0x7f9bb80b7870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4107080 tx=0x7f9bc0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:08.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.792+0000 7f9bcaa00640 1 -- 172.21.15.90:0/2288257019 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9bc40ac5b0 con 0x7f9bd4104fa0 2024-09-10T22:03:08.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.992+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9bd4063590 con 0x7f9bb80b53b0 2024-09-10T22:03:08.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:08 smithi090 ceph-mon[30350]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T22:03:08.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.993+0000 7f9bcaa00640 1 -- 172.21.15.90:0/2288257019 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9bd4063590 con 0x7f9bb80b53b0 2024-09-10T22:03:08.995 INFO:teuthology.orchestra.run.smithi090.stdout:Backend: cephadm 2024-09-10T22:03:08.995 INFO:teuthology.orchestra.run.smithi090.stdout:Available: Yes 2024-09-10T22:03:08.995 INFO:teuthology.orchestra.run.smithi090.stdout:Paused: No 2024-09-10T22:03:08.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.995+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9bb80b53b0 msgr2=0x7f9bb80b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.995+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9bb80b53b0 0x7f9bb80b7870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4107080 tx=0x7f9bc0005f90 comp rx=0 tx=0).stop 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd4104fa0 msgr2=0x7f9bd419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd4104fa0 0x7f9bd419b430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400d770 tx=0x7f9bc400dc30 comp rx=0 tx=0).stop 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 shutdown_connections 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9bb80b53b0 0x7f9bb80b7870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9bd410b330 0x7f9bd41a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9bd410a9b0 0x7f9bd419b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 --2- 172.21.15.90:0/2288257019 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bd4104fa0 0x7f9bd419b430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 >> 172.21.15.90:0/2288257019 conn(0x7f9bd4100a50 msgr2=0x7f9bd4101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 shutdown_connections 2024-09-10T22:03:08.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:08.996+0000 7f9bd8a00640 1 -- 172.21.15.90:0/2288257019 wait complete. 2024-09-10T22:03:08.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:08 smithi171 ceph-mon[34138]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T22:03:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:08 smithi154 ceph-mon[34495]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T22:03:10.138 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T22:03:10.508 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:10.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:10 smithi090 ceph-mon[30350]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:10.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:10 smithi090 ceph-mon[30350]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T22:03:10.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:10 smithi171 ceph-mon[34138]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:10.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:10 smithi171 ceph-mon[34138]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T22:03:11.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.081+0000 7f584f400640 1 -- 172.21.15.90:0/4121945762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105ef0 msgr2=0x7f584810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.081+0000 7f584f400640 1 --2- 172.21.15.90:0/4121945762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105ef0 0x7f584810f930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5840009ca0 tx=0x7f584002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.082+0000 7f584f400640 1 -- 172.21.15.90:0/4121945762 shutdown_connections 2024-09-10T22:03:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.082+0000 7f584f400640 1 --2- 172.21.15.90:0/4121945762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105ef0 0x7f584810f930 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.082+0000 7f584f400640 1 --2- 172.21.15.90:0/4121945762 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5848105570 0x7f58481059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.082+0000 7f584f400640 1 --2- 172.21.15.90:0/4121945762 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5848101220 0x7f5848101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.082+0000 7f584f400640 1 -- 172.21.15.90:0/4121945762 >> 172.21.15.90:0/4121945762 conn(0x7f5848078100 msgr2=0x7f58480ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.082+0000 7f584f400640 1 -- 172.21.15.90:0/4121945762 shutdown_connections 2024-09-10T22:03:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.082+0000 7f584f400640 1 -- 172.21.15.90:0/4121945762 wait complete. 2024-09-10T22:03:11.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.083+0000 7f584f400640 1 Processor -- start 2024-09-10T22:03:11.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.083+0000 7f584f400640 1 -- start start 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f584f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5848101220 0x7f58481021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f584f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105570 0x7f5848102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f584f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5848105ef0 0x7f5848102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f584f400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5848112c90 con 0x7f5848105570 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f584f400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5848112b10 con 0x7f5848101220 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f584f400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5848112e10 con 0x7f5848105ef0 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f5847e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105570 0x7f5848102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f5847e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105570 0x7f5848102730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54142/0 (socket says 172.21.15.90:54142) 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f5847e00640 1 -- 172.21.15.90:0/2694669363 learned_addr learned my addr 172.21.15.90:0/2694669363 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f584ca00640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5848101220 0x7f58481021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:11.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f584d400640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5848105ef0 0x7f5848102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:11.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f5847e00640 1 -- 172.21.15.90:0/2694669363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5848105ef0 msgr2=0x7f5848102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f5847e00640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5848105ef0 0x7f5848102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f5847e00640 1 -- 172.21.15.90:0/2694669363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5848101220 msgr2=0x7f58481021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f5847e00640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5848101220 0x7f58481021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f5847e00640 1 -- 172.21.15.90:0/2694669363 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58481a7500 con 0x7f5848105570 2024-09-10T22:03:11.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.084+0000 7f584d400640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5848105ef0 0x7f5848102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:11.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.085+0000 7f5847e00640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105570 0x7f5848102730 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f583400d950 tx=0x7f583400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:11.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.085+0000 7f5845600640 1 -- 172.21.15.90:0/2694669363 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5834017070 con 0x7f5848105570 2024-09-10T22:03:11.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.085+0000 7f5845600640 1 -- 172.21.15.90:0/2694669363 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f583400de80 con 0x7f5848105570 2024-09-10T22:03:11.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.085+0000 7f5845600640 1 -- 172.21.15.90:0/2694669363 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583400f620 con 0x7f5848105570 2024-09-10T22:03:11.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.085+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58481a77f0 con 0x7f5848105570 2024-09-10T22:03:11.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.085+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58481a7ce0 con 0x7f5848105570 2024-09-10T22:03:11.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.087+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5848101600 con 0x7f5848105570 2024-09-10T22:03:11.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.090+0000 7f5845600640 1 -- 172.21.15.90:0/2694669363 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f5834005030 con 0x7f5848105570 2024-09-10T22:03:11.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.091+0000 7f5845600640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f582c0b5330 0x7f582c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:11.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.091+0000 7f5845600640 1 -- 172.21.15.90:0/2694669363 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f58340e5d40 con 0x7f5848105570 2024-09-10T22:03:11.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.091+0000 7f584ca00640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f582c0b5330 0x7f582c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:11.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.092+0000 7f584ca00640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f582c0b5330 0x7f582c0b77f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f583800abe0 tx=0x7f583800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:11.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.092+0000 7f5845600640 1 -- 172.21.15.90:0/2694669363 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f58340ac710 con 0x7f5848105570 2024-09-10T22:03:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:10 smithi154 ceph-mon[34495]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:10 smithi154 ceph-mon[34495]: pgmap v396: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T22:03:11.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.283+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f5848063590 con 0x7f582c0b5330 2024-09-10T22:03:11.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.295+0000 7f5845600640 1 -- 172.21.15.90:0/2694669363 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f5848063590 con 0x7f582c0b5330 2024-09-10T22:03:11.297 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.a smithi090 running (2m) 26s ago 2m 16.9M - 19.3.0-4854-g8f219c47 589dd18248e4 922cccf8c017 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.b smithi154 running (2m) 30s ago 2m 25.5M - 19.3.0-4854-g8f219c47 589dd18248e4 60d5877916da 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.c smithi171 running (3m) 30s ago 3m 16.9M - 19.3.0-4854-g8f219c47 589dd18248e4 9a12e81784ae 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.d smithi090 running (3m) 26s ago 3m 13.4M - 19.3.0-4854-g8f219c47 589dd18248e4 1e770cf4225f 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.e smithi154 running (3m) 30s ago 3m 18.0M - 19.3.0-4854-g8f219c47 589dd18248e4 a78c6981b572 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.f smithi171 running (3m) 30s ago 3m 18.1M - 19.3.0-4854-g8f219c47 589dd18248e4 917ee409f917 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.g smithi090 running (2m) 26s ago 2m 16.9M - 19.3.0-4854-g8f219c47 589dd18248e4 98f264980d68 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.h smithi154 running (3m) 30s ago 3m 18.1M - 19.3.0-4854-g8f219c47 589dd18248e4 54d88274eab6 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.i smithi171 running (3m) 30s ago 3m 18.1M - 19.3.0-4854-g8f219c47 589dd18248e4 6a13ab59feeb 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.j smithi090 running (3m) 26s ago 3m 13.5M - 19.3.0-4854-g8f219c47 589dd18248e4 ee10b76dd6fd 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.k smithi154 running (3m) 30s ago 3m 17.8M - 19.3.0-4854-g8f219c47 589dd18248e4 5f9d0d7e4467 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mds.l smithi171 running (3m) 30s ago 3m 17.1M - 19.3.0-4854-g8f219c47 589dd18248e4 a4454f43cfa3 2024-09-10T22:03:11.298 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi090 *:9283,8765 running (12m) 26s ago 12m 520M - 19.3.0-4854-g8f219c47 589dd18248e4 bd728928651d 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi154 *:8443,8765 running (10m) 30s ago 10m 456M - 19.3.0-4854-g8f219c47 589dd18248e4 bd31d1cd99cb 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.z smithi171 *:8443,8765 running (10m) 30s ago 10m 458M - 19.3.0-4854-g8f219c47 589dd18248e4 105c9e4c7c76 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (12m) 26s ago 13m 52.5M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 ebe243d48873 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi154 running (10m) 30s ago 10m 40.7M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 670bf7b33c22 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi171 running (11m) 30s ago 11m 42.6M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 6336244ad0d5 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (9m) 26s ago 9m 46.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 5b2057dacd21 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi154 running (9m) 30s ago 9m 57.2M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 e222fd539ceb 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi171 running (8m) 30s ago 8m 46.5M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 91e94588bee0 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (8m) 26s ago 7m 42.9M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 0f21e66333cc 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi154 running (7m) 30s ago 7m 43.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 8e947c405888 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi171 running (6m) 30s ago 6m 59.1M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 729d8cdf342e 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi090 running (6m) 26s ago 6m 53.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 52d593b1ac10 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi154 running (5m) 30s ago 5m 44.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 e20038b802b6 2024-09-10T22:03:11.299 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 smithi171 running (5m) 30s ago 5m 42.6M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 ea05810875ce 2024-09-10T22:03:11.300 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 smithi090 running (4m) 26s ago 4m 42.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 33aa1ce867d5 2024-09-10T22:03:11.300 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 smithi154 running (4m) 30s ago 4m 45.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 e5190e3d7378 2024-09-10T22:03:11.300 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 smithi171 running (3m) 30s ago 3m 42.5M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 beb107cbc71f 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.297+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f582c0b5330 msgr2=0x7f582c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.297+0000 7f584f400640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f582c0b5330 0x7f582c0b77f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f583800abe0 tx=0x7f583800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.297+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105570 msgr2=0x7f5848102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.297+0000 7f584f400640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105570 0x7f5848102730 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f583400d950 tx=0x7f583400dd50 comp rx=0 tx=0).stop 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.298+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 shutdown_connections 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.298+0000 7f584f400640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f582c0b5330 0x7f582c0b77f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.298+0000 7f584f400640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5848105ef0 0x7f5848102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.298+0000 7f584f400640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5848105570 0x7f5848102730 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.298+0000 7f584f400640 1 --2- 172.21.15.90:0/2694669363 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5848101220 0x7f58481021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.298+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 >> 172.21.15.90:0/2694669363 conn(0x7f5848078100 msgr2=0x7f5848107130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:11.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.298+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 shutdown_connections 2024-09-10T22:03:11.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:11.298+0000 7f584f400640 1 -- 172.21.15.90:0/2694669363 wait complete. 2024-09-10T22:03:12.534 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T22:03:12.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:12 smithi090 ceph-mon[30350]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:12.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:12 smithi090 ceph-mon[30350]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T22:03:12.904 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:12 smithi171 ceph-mon[34138]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:12.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:12 smithi171 ceph-mon[34138]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T22:03:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:12 smithi154 ceph-mon[34495]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:12 smithi154 ceph-mon[34495]: pgmap v397: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T22:03:13.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.663+0000 7f446a000640 1 -- 172.21.15.90:0/4279660306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 msgr2=0x7f4464075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:13.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.663+0000 7f446a000640 1 --2- 172.21.15.90:0/4279660306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 0x7f4464075920 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4454009900 tx=0x7f445402a410 comp rx=0 tx=0).stop 2024-09-10T22:03:13.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.663+0000 7f446a000640 1 -- 172.21.15.90:0/4279660306 shutdown_connections 2024-09-10T22:03:13.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.663+0000 7f446a000640 1 --2- 172.21.15.90:0/4279660306 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f446410f3e0 0x7f44641117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.663+0000 7f446a000640 1 --2- 172.21.15.90:0/4279660306 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4464075f40 0x7f446410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.663+0000 7f446a000640 1 --2- 172.21.15.90:0/4279660306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 0x7f4464075920 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.663+0000 7f446a000640 1 -- 172.21.15.90:0/4279660306 >> 172.21.15.90:0/4279660306 conn(0x7f44640fe830 msgr2=0x7f4464100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:13.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.663+0000 7f446a000640 1 -- 172.21.15.90:0/4279660306 shutdown_connections 2024-09-10T22:03:13.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.663+0000 7f446a000640 1 -- 172.21.15.90:0/4279660306 wait complete. 2024-09-10T22:03:13.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.664+0000 7f446a000640 1 Processor -- start 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f446a000640 1 -- start start 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f446a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 0x7f446419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f446a000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4464075f40 0x7f446419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f446a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f446410f3e0 0x7f44641a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f446a000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4464114ea0 con 0x7f4464075540 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f446a000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4464114d20 con 0x7f4464075f40 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f446a000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4464115020 con 0x7f446410f3e0 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f4463400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 0x7f446419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f4463400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 0x7f446419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54158/0 (socket says 172.21.15.90:54158) 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f4463400640 1 -- 172.21.15.90:0/2959239358 learned_addr learned my addr 172.21.15.90:0/2959239358 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.665+0000 7f4463e00640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f446410f3e0 0x7f44641a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f4462a00640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4464075f40 0x7f446419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:13.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f4463400640 1 -- 172.21.15.90:0/2959239358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f446410f3e0 msgr2=0x7f44641a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:13.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f4463400640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f446410f3e0 0x7f44641a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f4463400640 1 -- 172.21.15.90:0/2959239358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4464075f40 msgr2=0x7f446419b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:13.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f4463400640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4464075f40 0x7f446419b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f4463400640 1 -- 172.21.15.90:0/2959239358 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44641a1480 con 0x7f4464075540 2024-09-10T22:03:13.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f4463400640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 0x7f446419b3d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f445402a930 tx=0x7f4454003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:13.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f445be00640 1 -- 172.21.15.90:0/2959239358 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f445403f070 con 0x7f4464075540 2024-09-10T22:03:13.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f445be00640 1 -- 172.21.15.90:0/2959239358 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4454003e40 con 0x7f4464075540 2024-09-10T22:03:13.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f445be00640 1 -- 172.21.15.90:0/2959239358 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454035030 con 0x7f4464075540 2024-09-10T22:03:13.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.666+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44641a1710 con 0x7f4464075540 2024-09-10T22:03:13.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.667+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44641a7e30 con 0x7f4464075540 2024-09-10T22:03:13.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.668+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4464075920 con 0x7f4464075540 2024-09-10T22:03:13.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.670+0000 7f445be00640 1 -- 172.21.15.90:0/2959239358 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f4454031400 con 0x7f4464075540 2024-09-10T22:03:13.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.671+0000 7f445be00640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44480b53b0 0x7f44480b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:13.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.671+0000 7f445be00640 1 -- 172.21.15.90:0/2959239358 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4454106e90 con 0x7f4464075540 2024-09-10T22:03:13.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.671+0000 7f4462a00640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44480b53b0 0x7f44480b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:13.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.672+0000 7f4462a00640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44480b53b0 0x7f44480b7870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4464101040 tx=0x7f445000b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:13.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.673+0000 7f445be00640 1 -- 172.21.15.90:0/2959239358 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f44540cd210 con 0x7f4464075540 2024-09-10T22:03:13.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.871+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4464100440 con 0x7f44480b53b0 2024-09-10T22:03:13.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.877+0000 7f445be00640 1 -- 172.21.15.90:0/2959239358 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4464100440 con 0x7f44480b53b0 2024-09-10T22:03:13.879 INFO:teuthology.orchestra.run.smithi090.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T22:03:13.879 INFO:teuthology.orchestra.run.smithi090.stdout:mds.all 12/12 33s ago 3m smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi154=b;smithi154=e;smithi154=h;smithi154=k;smithi171=c;smithi171=f;smithi171=i;smithi171=l;count:12 2024-09-10T22:03:13.879 INFO:teuthology.orchestra.run.smithi090.stdout:mgr 3/3 33s ago 10m smithi090=x;smithi154=y;smithi171=z;count:3 2024-09-10T22:03:13.879 INFO:teuthology.orchestra.run.smithi090.stdout:mon 3/3 33s ago 11m smithi090:172.21.15.90=a;smithi154:172.21.15.154=b;smithi171:172.21.15.171=c;count:3 2024-09-10T22:03:13.879 INFO:teuthology.orchestra.run.smithi090.stdout:osd 12 33s ago - 2024-09-10T22:03:13.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44480b53b0 msgr2=0x7f44480b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44480b53b0 0x7f44480b7870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4464101040 tx=0x7f445000b390 comp rx=0 tx=0).stop 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 msgr2=0x7f446419b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 0x7f446419b3d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f445402a930 tx=0x7f4454003c20 comp rx=0 tx=0).stop 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 shutdown_connections 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f44480b53b0 0x7f44480b7870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f446410f3e0 0x7f44641a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4464075f40 0x7f446419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 --2- 172.21.15.90:0/2959239358 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4464075540 0x7f446419b3d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 >> 172.21.15.90:0/2959239358 conn(0x7f44640fe830 msgr2=0x7f44641115e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 shutdown_connections 2024-09-10T22:03:13.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:13.880+0000 7f446a000640 1 -- 172.21.15.90:0/2959239358 wait complete. 2024-09-10T22:03:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:14 smithi171 ceph-mon[34138]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T22:03:15.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:14 smithi090 ceph-mon[30350]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T22:03:15.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T22:03:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:14 smithi154 ceph-mon[34495]: pgmap v398: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T22:03:15.511 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:15.903 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:15 smithi090 ceph-mon[30350]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:15.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:15 smithi171 ceph-mon[34138]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:16.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:15 smithi154 ceph-mon[34495]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:16.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.127+0000 7f496d600640 1 -- 172.21.15.90:0/3497181975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4968104fa0 msgr2=0x7f4968105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.127+0000 7f496d600640 1 --2- 172.21.15.90:0/3497181975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4968104fa0 0x7f4968105380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4958009900 tx=0x7f495802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.127+0000 7f496d600640 1 -- 172.21.15.90:0/3497181975 shutdown_connections 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.127+0000 7f496d600640 1 --2- 172.21.15.90:0/3497181975 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f496810b330 0x7f4968111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.127+0000 7f496d600640 1 --2- 172.21.15.90:0/3497181975 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f496810a9b0 0x7f496810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.127+0000 7f496d600640 1 --2- 172.21.15.90:0/3497181975 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4968104fa0 0x7f4968105380 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.127+0000 7f496d600640 1 -- 172.21.15.90:0/3497181975 >> 172.21.15.90:0/3497181975 conn(0x7f49681009f0 msgr2=0x7f4968102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.127+0000 7f496d600640 1 -- 172.21.15.90:0/3497181975 shutdown_connections 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.127+0000 7f496d600640 1 -- 172.21.15.90:0/3497181975 wait complete. 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.128+0000 7f496d600640 1 Processor -- start 2024-09-10T22:03:16.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.128+0000 7f496d600640 1 -- start start 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.128+0000 7f496d600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4968104fa0 0x7f496819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f496d600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f496810a9b0 0x7f496819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f496d600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f496810b330 0x7f49681a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f496d600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4968114e60 con 0x7f496810a9b0 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f496d600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4968114ce0 con 0x7f496810b330 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f496d600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4968114fe0 con 0x7f4968104fa0 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4966000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f496810a9b0 0x7f496819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4966000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f496810a9b0 0x7f496819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54172/0 (socket says 172.21.15.90:54172) 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4966000640 1 -- 172.21.15.90:0/2040285855 learned_addr learned my addr 172.21.15.90:0/2040285855 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4966a00640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4968104fa0 0x7f496819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4967400640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f496810b330 0x7f49681a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4966000640 1 -- 172.21.15.90:0/2040285855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4968104fa0 msgr2=0x7f496819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4966000640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4968104fa0 0x7f496819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4966000640 1 -- 172.21.15.90:0/2040285855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f496810b330 msgr2=0x7f49681a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4966000640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f496810b330 0x7f49681a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.129+0000 7f4966000640 1 -- 172.21.15.90:0/2040285855 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49681a1500 con 0x7f496810a9b0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.130+0000 7f4966000640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f496810a9b0 0x7f496819b990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f495400d770 tx=0x7f495400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.130+0000 7f495f400640 1 -- 172.21.15.90:0/2040285855 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954017070 con 0x7f496810a9b0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.130+0000 7f495f400640 1 -- 172.21.15.90:0/2040285855 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f49540054b0 con 0x7f496810a9b0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.130+0000 7f495f400640 1 -- 172.21.15.90:0/2040285855 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954004c60 con 0x7f496810a9b0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.130+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49681a17f0 con 0x7f496810a9b0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.130+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49681a7eb0 con 0x7f496810a9b0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.131+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4968105380 con 0x7f496810a9b0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.133+0000 7f495f400640 1 -- 172.21.15.90:0/2040285855 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f4954004e00 con 0x7f496810a9b0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.135+0000 7f495f400640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f494c0b53b0 0x7f494c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.135+0000 7f495f400640 1 -- 172.21.15.90:0/2040285855 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f49540e5a80 con 0x7f496810a9b0 2024-09-10T22:03:16.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.135+0000 7f4966a00640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f494c0b53b0 0x7f494c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.136+0000 7f4966a00640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f494c0b53b0 0x7f494c0b7870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4958006010 tx=0x7f4958005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:16.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.138+0000 7f495f400640 1 -- 172.21.15.90:0/2040285855 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f49540e7e20 con 0x7f496810a9b0 2024-09-10T22:03:16.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.330+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4968063590 con 0x7f494c0b53b0 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.331+0000 7f495f400640 1 -- 172.21.15.90:0/2040285855 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4968063590 con 0x7f494c0b53b0 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi090.stdout:HOST ADDR LABELS STATUS 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 172.21.15.90 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi090.stdout:smithi154 172.21.15.154 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi090.stdout:smithi171 172.21.15.171 2024-09-10T22:03:16.333 INFO:teuthology.orchestra.run.smithi090.stdout:3 hosts in cluster 2024-09-10T22:03:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.333+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f494c0b53b0 msgr2=0x7f494c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.333+0000 7f496d600640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f494c0b53b0 0x7f494c0b7870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4958006010 tx=0x7f4958005e80 comp rx=0 tx=0).stop 2024-09-10T22:03:16.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.333+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f496810a9b0 msgr2=0x7f496819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.333+0000 7f496d600640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f496810a9b0 0x7f496819b990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f495400d770 tx=0x7f495400dc30 comp rx=0 tx=0).stop 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.334+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 shutdown_connections 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.334+0000 7f496d600640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f494c0b53b0 0x7f494c0b7870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.334+0000 7f496d600640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f496810b330 0x7f49681a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.334+0000 7f496d600640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f496810a9b0 0x7f496819b990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.334+0000 7f496d600640 1 --2- 172.21.15.90:0/2040285855 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4968104fa0 0x7f496819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.334+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 >> 172.21.15.90:0/2040285855 conn(0x7f49681009f0 msgr2=0x7f4968102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.334+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 shutdown_connections 2024-09-10T22:03:16.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:16.334+0000 7f496d600640 1 -- 172.21.15.90:0/2040285855 wait complete. 2024-09-10T22:03:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:16 smithi090 ceph-mon[30350]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T22:03:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:16 smithi154 ceph-mon[34495]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T22:03:17.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:16 smithi171 ceph-mon[34138]: pgmap v399: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T22:03:17.573 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T22:03:17.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:17 smithi090 ceph-mon[30350]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:17.945 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:17 smithi154 ceph-mon[34495]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:17 smithi171 ceph-mon[34138]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:18.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.615+0000 7fb831e00640 1 -- 172.21.15.90:0/816319355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 msgr2=0x7fb82c1119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.615+0000 7fb831e00640 1 --2- 172.21.15.90:0/816319355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 0x7fb82c1119a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb824009900 tx=0x7fb82402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.615+0000 7fb831e00640 1 -- 172.21.15.90:0/816319355 shutdown_connections 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.615+0000 7fb831e00640 1 --2- 172.21.15.90:0/816319355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 0x7fb82c1119a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.615+0000 7fb831e00640 1 --2- 172.21.15.90:0/816319355 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb82c10a820 0x7fb82c10ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.615+0000 7fb831e00640 1 --2- 172.21.15.90:0/816319355 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb82c104e10 0x7fb82c1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.615+0000 7fb831e00640 1 -- 172.21.15.90:0/816319355 >> 172.21.15.90:0/816319355 conn(0x7fb82c100840 msgr2=0x7fb82c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.615+0000 7fb831e00640 1 -- 172.21.15.90:0/816319355 shutdown_connections 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.615+0000 7fb831e00640 1 -- 172.21.15.90:0/816319355 wait complete. 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.616+0000 7fb831e00640 1 Processor -- start 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.616+0000 7fb831e00640 1 -- start start 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.616+0000 7fb831e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb82c104e10 0x7fb82c19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.616+0000 7fb831e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb82c10a820 0x7fb82c19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.616+0000 7fb831e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 0x7fb82c1a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:18.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.616+0000 7fb831e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c114d60 con 0x7fb82c10b1a0 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.616+0000 7fb831e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c114be0 con 0x7fb82c104e10 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.616+0000 7fb831e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb82c114ee0 con 0x7fb82c10a820 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb831400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 0x7fb82c1a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb831400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 0x7fb82c1a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52412/0 (socket says 172.21.15.90:52412) 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb831400640 1 -- 172.21.15.90:0/3034030236 learned_addr learned my addr 172.21.15.90:0/3034030236 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb830a00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb82c104e10 0x7fb82c19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb82be00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb82c10a820 0x7fb82c19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb831400640 1 -- 172.21.15.90:0/3034030236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb82c10a820 msgr2=0x7fb82c19b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb831400640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb82c10a820 0x7fb82c19b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb831400640 1 -- 172.21.15.90:0/3034030236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb82c104e10 msgr2=0x7fb82c19b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb831400640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb82c104e10 0x7fb82c19b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb831400640 1 -- 172.21.15.90:0/3034030236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82c1a11a0 con 0x7fb82c10b1a0 2024-09-10T22:03:18.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb82be00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb82c10a820 0x7fb82c19b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.617+0000 7fb831400640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 0x7fb82c1a09d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb824004770 tx=0x7fb824003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.618+0000 7fb829600640 1 -- 172.21.15.90:0/3034030236 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82403f070 con 0x7fb82c10b1a0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.618+0000 7fb829600640 1 -- 172.21.15.90:0/3034030236 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb82403bd00 con 0x7fb82c10b1a0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.618+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb82c1a1430 con 0x7fb82c10b1a0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.618+0000 7fb829600640 1 -- 172.21.15.90:0/3034030236 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824035030 con 0x7fb82c10b1a0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.618+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb82c1a7b50 con 0x7fb82c10b1a0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.619+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb82c1051f0 con 0x7fb82c10b1a0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.621+0000 7fb829600640 1 -- 172.21.15.90:0/3034030236 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fb824031bd0 con 0x7fb82c10b1a0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.622+0000 7fb829600640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb8100b53e0 0x7fb8100b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.623+0000 7fb829600640 1 -- 172.21.15.90:0/3034030236 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb824106bd0 con 0x7fb82c10b1a0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.623+0000 7fb830a00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb8100b53e0 0x7fb8100b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.623+0000 7fb830a00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb8100b53e0 0x7fb8100b78a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb81c005fd0 tx=0x7fb81c009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:18.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.626+0000 7fb829600640 1 -- 172.21.15.90:0/3034030236 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb8240cd5a0 con 0x7fb82c10b1a0 2024-09-10T22:03:18.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.823+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb82c107de0 con 0x7fb8100b53e0 2024-09-10T22:03:18.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.825+0000 7fb829600640 1 -- 172.21.15.90:0/3034030236 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb82c107de0 con 0x7fb8100b53e0 2024-09-10T22:03:18.827 INFO:teuthology.orchestra.run.smithi090.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T22:03:18.827 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-10T22:03:18.827 INFO:teuthology.orchestra.run.smithi090.stdout:smithi154 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233003B400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-10T22:03:18.827 INFO:teuthology.orchestra.run.smithi090.stdout:smithi171 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233001S400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T22:03:18.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.827+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb8100b53e0 msgr2=0x7fb8100b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.827+0000 7fb831e00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb8100b53e0 0x7fb8100b78a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb81c005fd0 tx=0x7fb81c009680 comp rx=0 tx=0).stop 2024-09-10T22:03:18.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.827+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 msgr2=0x7fb82c1a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:18.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.827+0000 7fb831e00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 0x7fb82c1a09d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb824004770 tx=0x7fb824003cd0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.828+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 shutdown_connections 2024-09-10T22:03:18.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.828+0000 7fb831e00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb8100b53e0 0x7fb8100b78a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.828+0000 7fb831e00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb82c10b1a0 0x7fb82c1a09d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.828+0000 7fb831e00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb82c10a820 0x7fb82c19b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.828+0000 7fb831e00640 1 --2- 172.21.15.90:0/3034030236 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb82c104e10 0x7fb82c19b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:18.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.828+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 >> 172.21.15.90:0/3034030236 conn(0x7fb82c100840 msgr2=0x7fb82c103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:18.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.828+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 shutdown_connections 2024-09-10T22:03:18.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:18.828+0000 7fb831e00640 1 -- 172.21.15.90:0/3034030236 wait complete. 2024-09-10T22:03:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:18 smithi090 ceph-mon[30350]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-10T22:03:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:18 smithi154 ceph-mon[34495]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-10T22:03:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:18 smithi171 ceph-mon[34138]: pgmap v400: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-10T22:03:20.242 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T22:03:20.264 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-09-10T22:03:20.264 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T22:03:20.637 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:21.042 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:20 smithi090 ceph-mon[30350]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:21.043 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:20 smithi090 ceph-mon[30350]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T22:03:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:20 smithi154 ceph-mon[34495]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:20 smithi154 ceph-mon[34495]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T22:03:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:20 smithi171 ceph-mon[34138]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:21.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:20 smithi171 ceph-mon[34138]: pgmap v401: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T22:03:21.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.337+0000 7f996fa00640 1 -- 172.21.15.90:0/1411692225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075e60 msgr2=0x7f99681135e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:21.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.337+0000 7f996fa00640 1 --2- 172.21.15.90:0/1411692225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075e60 0x7f99681135e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9950009900 tx=0x7f995002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.338+0000 7f996fa00640 1 -- 172.21.15.90:0/1411692225 shutdown_connections 2024-09-10T22:03:21.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.338+0000 7f996fa00640 1 --2- 172.21.15.90:0/1411692225 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9968113b20 0x7f9968115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.338+0000 7f996fa00640 1 --2- 172.21.15.90:0/1411692225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075e60 0x7f99681135e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.338+0000 7f996fa00640 1 --2- 172.21.15.90:0/1411692225 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968075540 0x7f9968075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.338+0000 7f996fa00640 1 -- 172.21.15.90:0/1411692225 >> 172.21.15.90:0/1411692225 conn(0x7f99680fe830 msgr2=0x7f9968100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.338+0000 7f996fa00640 1 -- 172.21.15.90:0/1411692225 shutdown_connections 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.338+0000 7f996fa00640 1 -- 172.21.15.90:0/1411692225 wait complete. 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996fa00640 1 Processor -- start 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996fa00640 1 -- start start 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996fa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075540 0x7f996819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996fa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9968075e60 0x7f996819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996fa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968113b20 0x7f99681a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996fa00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99681192b0 con 0x7f9968075540 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996fa00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9968119130 con 0x7f9968113b20 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996fa00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9968119430 con 0x7f9968075e60 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075540 0x7f996819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:21.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075540 0x7f996819f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52426/0 (socket says 172.21.15.90:52426) 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.339+0000 7f996d000640 1 -- 172.21.15.90:0/4247113659 learned_addr learned my addr 172.21.15.90:0/4247113659 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f996da00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968113b20 0x7f99681a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f995fe00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9968075e60 0x7f996819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f996d000640 1 -- 172.21.15.90:0/4247113659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9968075e60 msgr2=0x7f996819fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f996d000640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9968075e60 0x7f996819fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f996d000640 1 -- 172.21.15.90:0/4247113659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968113b20 msgr2=0x7f99681a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f996d000640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968113b20 0x7f99681a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f996d000640 1 -- 172.21.15.90:0/4247113659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99681a58f0 con 0x7f9968075540 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f995fe00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9968075e60 0x7f996819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f996d000640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075540 0x7f996819f840 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f995800e9f0 tx=0x7f995800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f9966000640 1 -- 172.21.15.90:0/4247113659 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995800cd80 con 0x7f9968075540 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f9966000640 1 -- 172.21.15.90:0/4247113659 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99580108c0 con 0x7f9968075540 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99681a5be0 con 0x7f9968075540 2024-09-10T22:03:21.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.340+0000 7f9966000640 1 -- 172.21.15.90:0/4247113659 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99580187c0 con 0x7f9968075540 2024-09-10T22:03:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.341+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99681ac2a0 con 0x7f9968075540 2024-09-10T22:03:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.342+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9968075920 con 0x7f9968075540 2024-09-10T22:03:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.344+0000 7f9966000640 1 -- 172.21.15.90:0/4247113659 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f9958010430 con 0x7f9968075540 2024-09-10T22:03:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.345+0000 7f9966000640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f994c0b5330 0x7f994c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.345+0000 7f9966000640 1 -- 172.21.15.90:0/4247113659 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f99580e6600 con 0x7f9968075540 2024-09-10T22:03:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.346+0000 7f995fe00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f994c0b5330 0x7f994c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.346+0000 7f995fe00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f994c0b5330 0x7f994c0b77f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f995002a910 tx=0x7f9950005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.349+0000 7f9966000640 1 -- 172.21.15.90:0/4247113659 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f99580acfd0 con 0x7f9968075540 2024-09-10T22:03:21.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.566+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f99681ac660 con 0x7f9968075540 2024-09-10T22:03:21.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.566+0000 7f9966000640 1 -- 172.21.15.90:0/4247113659 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3138 (secure 0 0 0) 0x7f99580ac8d0 con 0x7f9968075540 2024-09-10T22:03:21.568 INFO:teuthology.orchestra.run.smithi090.stdout:e29 2024-09-10T22:03:21.568 INFO:teuthology.orchestra.run.smithi090.stdout:btime 2024-09-10T22:01:34:736152+0000 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 29 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-09-10T22:00:35.595646+0000 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-09-10T22:01:34.015563+0000 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-09-10T22:03:21.569 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 300 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 5 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1,2,3,4 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=34703,1=34670,2=34646,3=34640,4=34607} 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3,4] 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-09-10T22:03:21.570 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 0 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 34703 members: 34607,34646,34640,34670,34703 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b{0:34703} state up:active seq 5 addr [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.i{0:24676} state up:standby-replay seq 1 addr [v2:172.21.15.171:6838/3372214298,v1:172.21.15.171:6839/3372214298] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.h{1:34670} state up:active seq 22 addr [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c{1:24655} state up:standby-replay seq 1 addr [v2:172.21.15.171:6836/2889350064,v1:172.21.15.171:6837/2889350064] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.f{2:34646} state up:active seq 24 addr [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.l{2:24598} state up:standby-replay seq 1 addr [v2:172.21.15.171:6832/3906453887,v1:172.21.15.171:6833/3906453887] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.e{3:34640} state up:active seq 25 addr [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a{3:14853} state up:standby-replay seq 1 addr [v2:172.21.15.90:6840/3169114114,v1:172.21.15.90:6841/3169114114] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.k{4:34607} state up:active seq 29 addr [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.571 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.g{4:14841} state up:standby-replay seq 1 addr [v2:172.21.15.90:6838/3596805819,v1:172.21.15.90:6839/3596805819] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.572 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:21.572 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:21.572 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-09-10T22:03:21.572 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:21.572 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.j{-1:14772} state up:standby seq 1 addr [v2:172.21.15.90:6834/1514545418,v1:172.21.15.90:6835/1514545418] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.572 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d{-1:14784} state up:standby seq 1 addr [v2:172.21.15.90:6836/4068804869,v1:172.21.15.90:6837/4068804869] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f994c0b5330 msgr2=0x7f994c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f994c0b5330 0x7f994c0b77f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f995002a910 tx=0x7f9950005fd0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075540 msgr2=0x7f996819f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075540 0x7f996819f840 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f995800e9f0 tx=0x7f995800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 shutdown_connections 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f994c0b5330 0x7f994c0b77f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9968113b20 0x7f99681a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9968075e60 0x7f996819fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 --2- 172.21.15.90:0/4247113659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9968075540 0x7f996819f840 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 >> 172.21.15.90:0/4247113659 conn(0x7f99680fe830 msgr2=0x7f99680ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 shutdown_connections 2024-09-10T22:03:21.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:21.570+0000 7f996fa00640 1 -- 172.21.15.90:0/4247113659 wait complete. 2024-09-10T22:03:22.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:21 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4247113659' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:03:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:21 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4247113659' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:03:22.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:21 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4247113659' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T22:03:22.635 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T22:03:22.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:22 smithi090 ceph-mon[30350]: pgmap v402: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:23.009 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:22 smithi154 ceph-mon[34495]: pgmap v402: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:22 smithi171 ceph-mon[34138]: pgmap v402: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.567+0000 7f747f200640 1 -- 172.21.15.90:0/1561621156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74800776d0 msgr2=0x7f7480077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.567+0000 7f747f200640 1 --2- 172.21.15.90:0/1561621156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74800776d0 0x7f7480077ab0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7470009ca0 tx=0x7f747002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.567+0000 7f747f200640 1 -- 172.21.15.90:0/1561621156 shutdown_connections 2024-09-10T22:03:23.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.567+0000 7f747f200640 1 --2- 172.21.15.90:0/1561621156 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74801137e0 0x7f7480115bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.567+0000 7f747f200640 1 --2- 172.21.15.90:0/1561621156 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7480077ff0 0x7f74801132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.567+0000 7f747f200640 1 --2- 172.21.15.90:0/1561621156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74800776d0 0x7f7480077ab0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.567+0000 7f747f200640 1 -- 172.21.15.90:0/1561621156 >> 172.21.15.90:0/1561621156 conn(0x7f7480100840 msgr2=0x7f7480102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.567+0000 7f747f200640 1 -- 172.21.15.90:0/1561621156 shutdown_connections 2024-09-10T22:03:23.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.567+0000 7f747f200640 1 -- 172.21.15.90:0/1561621156 wait complete. 2024-09-10T22:03:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.568+0000 7f747f200640 1 Processor -- start 2024-09-10T22:03:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.568+0000 7f747f200640 1 -- start start 2024-09-10T22:03:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74800776d0 0x7f748019f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7480077ff0 0x7f748019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:23.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74801137e0 0x7f74801a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f74801190e0 con 0x7f7480077ff0 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747f200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7480118f60 con 0x7f74800776d0 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7480119260 con 0x7f74801137e0 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74800776d0 0x7f748019f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74800776d0 0x7f748019f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:37920/0 (socket says 172.21.15.90:37920) 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747de00640 1 -- 172.21.15.90:0/375830229 learned_addr learned my addr 172.21.15.90:0/375830229 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7480077ff0 0x7f748019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.569+0000 7f747e800640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74801137e0 0x7f74801a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f747d400640 1 -- 172.21.15.90:0/375830229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74801137e0 msgr2=0x7f74801a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f747d400640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74801137e0 0x7f74801a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f747d400640 1 -- 172.21.15.90:0/375830229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74800776d0 msgr2=0x7f748019f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:23.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f747d400640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74800776d0 0x7f748019f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f747d400640 1 -- 172.21.15.90:0/375830229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74801a55c0 con 0x7f7480077ff0 2024-09-10T22:03:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f747de00640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74800776d0 0x7f748019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:03:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f747d400640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7480077ff0 0x7f748019fa50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f746c00d6d0 tx=0x7f746c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f7476a00640 1 -- 172.21.15.90:0/375830229 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c017070 con 0x7f7480077ff0 2024-09-10T22:03:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f7476a00640 1 -- 172.21.15.90:0/375830229 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f746c009820 con 0x7f7480077ff0 2024-09-10T22:03:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74801a58b0 con 0x7f7480077ff0 2024-09-10T22:03:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f7476a00640 1 -- 172.21.15.90:0/375830229 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c004bc0 con 0x7f7480077ff0 2024-09-10T22:03:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.570+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74801abf70 con 0x7f7480077ff0 2024-09-10T22:03:23.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.571+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7480077ab0 con 0x7f7480077ff0 2024-09-10T22:03:23.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.574+0000 7f7476a00640 1 -- 172.21.15.90:0/375830229 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f746c005220 con 0x7f7480077ff0 2024-09-10T22:03:23.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.575+0000 7f7476a00640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f74640b53e0 0x7f74640b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:23.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.576+0000 7f7476a00640 1 -- 172.21.15.90:0/375830229 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f746c0e5c40 con 0x7f7480077ff0 2024-09-10T22:03:23.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.576+0000 7f747de00640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f74640b53e0 0x7f74640b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:23.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.576+0000 7f747de00640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f74640b53e0 0x7f74640b78a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f74700098b0 tx=0x7f7470009800 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:23.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.579+0000 7f7476a00640 1 -- 172.21.15.90:0/375830229 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f746c005540 con 0x7f7480077ff0 2024-09-10T22:03:23.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.773+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f74801ac300 con 0x7f7480077ff0 2024-09-10T22:03:23.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.774+0000 7f7476a00640 1 -- 172.21.15.90:0/375830229 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+6071 (secure 0 0 0) 0x7f746c0ac610 con 0x7f7480077ff0 2024-09-10T22:03:23.776 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 90 2024-09-10T22:03:23.776 INFO:teuthology.orchestra.run.smithi090.stdout:fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T22:03:23.776 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-09-10T21:50:11.271441+0000 2024-09-10T22:03:23.776 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-09-10T22:02:35.713572+0000 2024-09-10T22:03:23.776 INFO:teuthology.orchestra.run.smithi090.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T22:03:23.776 INFO:teuthology.orchestra.run.smithi090.stdout:crush_version 27 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:full_ratio 0.95 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:backfillfull_ratio 0.9 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:nearfull_ratio 0.85 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:require_min_compat_client luminous 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:min_compat_client luminous 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:require_osd_release squid 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:stretch_mode_enabled false 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T22:03:23.777 INFO:teuthology.orchestra.run.smithi090.stdout:max_osd 12 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 up in weight 1 up_from 9 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6802/405981593,v1:172.21.15.90:6803/405981593] [v2:172.21.15.90:6804/405981593,v1:172.21.15.90:6805/405981593] exists,up e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 up in weight 1 up_from 14 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6800/2687850784,v1:172.21.15.154:6801/2687850784] [v2:172.21.15.154:6802/2687850784,v1:172.21.15.154:6803/2687850784] exists,up 417251ca-442c-4c88-881d-c28668067397 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 up in weight 1 up_from 19 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6800/862625037,v1:172.21.15.171:6801/862625037] [v2:172.21.15.171:6802/862625037,v1:172.21.15.171:6803/862625037] exists,up 529eeb1a-fe6a-4077-b8a5-97246ab90ec5 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6810/995208383,v1:172.21.15.90:6811/995208383] [v2:172.21.15.90:6812/995208383,v1:172.21.15.90:6813/995208383] exists,up 5d6629c1-3c41-462a-84ce-21a906d40bb0 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 up in weight 1 up_from 32 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6808/1188268169,v1:172.21.15.154:6809/1188268169] [v2:172.21.15.154:6810/1188268169,v1:172.21.15.154:6811/1188268169] exists,up 99cff575-9f60-486c-b99a-48d5c3cf531c 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 up in weight 1 up_from 37 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6808/2109988699,v1:172.21.15.171:6809/2109988699] [v2:172.21.15.171:6810/2109988699,v1:172.21.15.171:6811/2109988699] exists,up f7931a53-09a3-408d-91c4-b85dd2fec784 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 up in weight 1 up_from 43 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6818/3973504003,v1:172.21.15.90:6819/3973504003] [v2:172.21.15.90:6820/3973504003,v1:172.21.15.90:6821/3973504003] exists,up 2bdf9899-a84f-42fd-a62f-6c19c39f8ba4 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 up in weight 1 up_from 48 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6816/1617520219,v1:172.21.15.154:6817/1617520219] [v2:172.21.15.154:6818/1617520219,v1:172.21.15.154:6819/1617520219] exists,up 627062fc-f4ab-4e64-a68a-33beb4664f3e 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 up in weight 1 up_from 54 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6816/4137047501,v1:172.21.15.171:6817/4137047501] [v2:172.21.15.171:6818/4137047501,v1:172.21.15.171:6819/4137047501] exists,up 6ec3145c-deaf-467c-800c-257e339ebde5 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 up in weight 1 up_from 60 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6826/1146146212,v1:172.21.15.90:6827/1146146212] [v2:172.21.15.90:6828/1146146212,v1:172.21.15.90:6829/1146146212] exists,up 7e58c2f4-87df-4021-98ff-fd65c0a4fbcf 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 up in weight 1 up_from 65 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.154:6824/1803626214,v1:172.21.15.154:6825/1803626214] [v2:172.21.15.154:6826/1803626214,v1:172.21.15.154:6827/1803626214] exists,up e57aa10e-348d-4de3-be31-927241f583e0 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 up in weight 1 up_from 71 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.171:6824/1226533587,v1:172.21.15.171:6825/1226533587] [v2:172.21.15.171:6826/1226533587,v1:172.21.15.171:6827/1226533587] exists,up 599b64e5-d2f1-42c7-9532-67156d42d99a 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.b [2,4] 2024-09-10T22:03:23.778 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.10 [2,1,8,11] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.1d [8,11] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.0 [5,9] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.6 [4,6] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.11 [4,7] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.1d [5,9] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.3a [4,8] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.3b [3,9] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.e [9,11] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.12 [9,10] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.1b [9,8] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 4.1c [5,8] 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2204308252 expires 2024-09-11T21:50:49.799954+0000 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/1572343768 expires 2024-09-11T21:52:33.477408+0000 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/1613780567 expires 2024-09-11T21:50:49.799954+0000 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/1385951279 expires 2024-09-11T21:50:49.799954+0000 2024-09-10T22:03:23.779 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/323032163 expires 2024-09-11T21:51:17.626867+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/3233554768 expires 2024-09-11T21:52:33.477408+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/1613780567 expires 2024-09-11T21:50:49.799954+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/3858975894 expires 2024-09-11T21:50:49.799954+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2255342982 expires 2024-09-11T21:51:17.626867+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/3643795744 expires 2024-09-11T21:51:17.626867+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2773946294 expires 2024-09-11T21:52:33.477408+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2694078227 expires 2024-09-11T21:51:17.626867+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/323032163 expires 2024-09-11T21:51:17.626867+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/3233554768 expires 2024-09-11T21:52:33.477408+0000 2024-09-10T22:03:23.780 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/3503416190 expires 2024-09-11T21:52:33.477408+0000 2024-09-10T22:03:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f74640b53e0 msgr2=0x7f74640b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f74640b53e0 0x7f74640b78a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f74700098b0 tx=0x7f7470009800 comp rx=0 tx=0).stop 2024-09-10T22:03:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7480077ff0 msgr2=0x7f748019fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7480077ff0 0x7f748019fa50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f746c00d6d0 tx=0x7f746c00db90 comp rx=0 tx=0).stop 2024-09-10T22:03:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 shutdown_connections 2024-09-10T22:03:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f74640b53e0 0x7f74640b78a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f74801137e0 0x7f74801a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7480077ff0 0x7f748019fa50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 --2- 172.21.15.90:0/375830229 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f74800776d0 0x7f748019f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.776+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 >> 172.21.15.90:0/375830229 conn(0x7f7480100840 msgr2=0x7f7480102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.777+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 shutdown_connections 2024-09-10T22:03:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:23.777+0000 7f747f200640 1 -- 172.21.15.90:0/375830229 wait complete. 2024-09-10T22:03:24.638 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T22:03:24.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:03:24.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:24 smithi090 ceph-mon[30350]: pgmap v403: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:24.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:24 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/375830229' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:03:25.021 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:24 smithi154 ceph-mon[34495]: pgmap v403: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:24 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/375830229' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:03:25.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:24 smithi171 ceph-mon[34138]: pgmap v403: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:25.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:24 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/375830229' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T22:03:25.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.632+0000 7fa0c9a00640 1 -- 172.21.15.90:0/1710100915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 msgr2=0x7fa0c4111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.632+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/1710100915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 0x7fa0c4111780 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009ca0 tx=0x7fa0bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.632+0000 7fa0c9a00640 1 -- 172.21.15.90:0/1710100915 shutdown_connections 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.632+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/1710100915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 0x7fa0c4111780 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.632+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/1710100915 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c4075e00 0x7fa0c410ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.632+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/1710100915 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0c40754e0 0x7fa0c40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.632+0000 7fa0c9a00640 1 -- 172.21.15.90:0/1710100915 >> 172.21.15.90:0/1710100915 conn(0x7fa0c40fe810 msgr2=0x7fa0c4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.632+0000 7fa0c9a00640 1 -- 172.21.15.90:0/1710100915 shutdown_connections 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.632+0000 7fa0c9a00640 1 -- 172.21.15.90:0/1710100915 wait complete. 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.633+0000 7fa0c9a00640 1 Processor -- start 2024-09-10T22:03:25.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.633+0000 7fa0c9a00640 1 -- start start 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.633+0000 7fa0c9a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c40754e0 0x7fa0c419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.633+0000 7fa0c9a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0c4075e00 0x7fa0c419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.633+0000 7fa0c9a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 0x7fa0c41a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.633+0000 7fa0c9a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4114e40 con 0x7fa0c410f390 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.633+0000 7fa0c9a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4114cc0 con 0x7fa0c40754e0 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.633+0000 7fa0c9a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4114fc0 con 0x7fa0c4075e00 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0c3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 0x7fa0c41a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0c3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 0x7fa0c41a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52462/0 (socket says 172.21.15.90:52462) 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0c3400640 1 -- 172.21.15.90:0/3383057429 learned_addr learned my addr 172.21.15.90:0/3383057429 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0c2a00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c40754e0 0x7fa0c419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0bbe00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0c4075e00 0x7fa0c419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:25.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0c3400640 1 -- 172.21.15.90:0/3383057429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0c4075e00 msgr2=0x7fa0c419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0c3400640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0c4075e00 0x7fa0c419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0c3400640 1 -- 172.21.15.90:0/3383057429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c40754e0 msgr2=0x7fa0c419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0c3400640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c40754e0 0x7fa0c419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0c3400640 1 -- 172.21.15.90:0/3383057429 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c41a1420 con 0x7fa0c410f390 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.634+0000 7fa0bbe00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0c4075e00 0x7fa0c419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.635+0000 7fa0c3400640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 0x7fa0c41a0c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc031040 tx=0x7fa0bc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.635+0000 7fa0bb400640 1 -- 172.21.15.90:0/3383057429 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc032c90 con 0x7fa0c410f390 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.635+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c41a16b0 con 0x7fa0c410f390 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.635+0000 7fa0bb400640 1 -- 172.21.15.90:0/3383057429 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa0bc032e30 con 0x7fa0c410f390 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.635+0000 7fa0bb400640 1 -- 172.21.15.90:0/3383057429 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc030ae0 con 0x7fa0c410f390 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.635+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c41a7dd0 con 0x7fa0c410f390 2024-09-10T22:03:25.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.636+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c40758c0 con 0x7fa0c410f390 2024-09-10T22:03:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.638+0000 7fa0bb400640 1 -- 172.21.15.90:0/3383057429 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fa0bc0046b0 con 0x7fa0c410f390 2024-09-10T22:03:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.640+0000 7fa0bb400640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0a80b5330 0x7fa0a80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.640+0000 7fa0bb400640 1 -- 172.21.15.90:0/3383057429 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa0bc1073e0 con 0x7fa0c410f390 2024-09-10T22:03:25.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.640+0000 7fa0c2a00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0a80b5330 0x7fa0a80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:25.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.641+0000 7fa0c2a00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0a80b5330 0x7fa0a80b77f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400bbe0 tx=0x7fa0b400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:25.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.642+0000 7fa0bb400640 1 -- 172.21.15.90:0/3383057429 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa0bc0cddb0 con 0x7fa0c410f390 2024-09-10T22:03:25.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.871+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0c41a8090 con 0x7fa0c410f390 2024-09-10T22:03:25.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.872+0000 7fa0bb400640 1 -- 172.21.15.90:0/3383057429 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7fa0bc0cd6b0 con 0x7fa0c410f390 2024-09-10T22:03:25.874 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:25.874 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"btime":"2024-09-10T22:01:34:736152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:01:34.015563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:03:25.875 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-09-10T22:03:25.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0a80b5330 msgr2=0x7fa0a80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:25.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0a80b5330 0x7fa0a80b77f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400bbe0 tx=0x7fa0b400b390 comp rx=0 tx=0).stop 2024-09-10T22:03:25.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 msgr2=0x7fa0c41a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:25.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 0x7fa0c41a0c50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc031040 tx=0x7fa0bc003f10 comp rx=0 tx=0).stop 2024-09-10T22:03:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 shutdown_connections 2024-09-10T22:03:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa0a80b5330 0x7fa0a80b77f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c410f390 0x7fa0c41a0c50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa0c4075e00 0x7fa0c419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 --2- 172.21.15.90:0/3383057429 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa0c40754e0 0x7fa0c419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 >> 172.21.15.90:0/3383057429 conn(0x7fa0c40fe810 msgr2=0x7fa0c4111540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:25.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 shutdown_connections 2024-09-10T22:03:25.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:25.875+0000 7fa0c9a00640 1 -- 172.21.15.90:0/3383057429 wait complete. 2024-09-10T22:03:26.601 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:03:26.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:26 smithi090 ceph-mon[30350]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:03:26.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:26 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3383057429' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:03:26.992 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:26 smithi154 ceph-mon[34495]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:03:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:26 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3383057429' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:03:27.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:26 smithi171 ceph-mon[34138]: pgmap v404: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:03:27.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:26 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3383057429' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:03:27.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.663+0000 7f06df000640 1 -- 172.21.15.90:0/3524567597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d8103cc0 msgr2=0x7f06d81040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:27.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.663+0000 7f06df000640 1 --2- 172.21.15.90:0/3524567597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d8103cc0 0x7f06d81040a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009900 tx=0x7f06c802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.664+0000 7f06df000640 1 -- 172.21.15.90:0/3524567597 shutdown_connections 2024-09-10T22:03:27.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.664+0000 7f06df000640 1 --2- 172.21.15.90:0/3524567597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06d810d120 0x7f06d810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.664+0000 7f06df000640 1 --2- 172.21.15.90:0/3524567597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d81045e0 0x7f06d810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.664+0000 7f06df000640 1 --2- 172.21.15.90:0/3524567597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d8103cc0 0x7f06d81040a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.664+0000 7f06df000640 1 -- 172.21.15.90:0/3524567597 >> 172.21.15.90:0/3524567597 conn(0x7f06d80fd700 msgr2=0x7f06d80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.664+0000 7f06df000640 1 -- 172.21.15.90:0/3524567597 shutdown_connections 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.664+0000 7f06df000640 1 -- 172.21.15.90:0/3524567597 wait complete. 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.664+0000 7f06df000640 1 Processor -- start 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06df000640 1 -- start start 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06df000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8103cc0 0x7f06d819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06df000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06d81045e0 0x7f06d819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06df000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d810d120 0x7f06d81a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06df000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8112c90 con 0x7f06d810d120 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06df000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8112b10 con 0x7f06d8103cc0 2024-09-10T22:03:27.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06df000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8112e10 con 0x7f06d81045e0 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06dd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d810d120 0x7f06d81a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06d7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8103cc0 0x7f06d819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06d7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8103cc0 0x7f06d819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:37958/0 (socket says 172.21.15.90:37958) 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.665+0000 7f06d7e00640 1 -- 172.21.15.90:0/3140787311 learned_addr learned my addr 172.21.15.90:0/3140787311 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.666+0000 7f06cfe00640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06d81045e0 0x7f06d819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.666+0000 7f06dd000640 1 -- 172.21.15.90:0/3140787311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06d81045e0 msgr2=0x7f06d819b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.666+0000 7f06dd000640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06d81045e0 0x7f06d819b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.666+0000 7f06dd000640 1 -- 172.21.15.90:0/3140787311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8103cc0 msgr2=0x7f06d819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.666+0000 7f06dd000640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8103cc0 0x7f06d819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.666+0000 7f06dd000640 1 -- 172.21.15.90:0/3140787311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d81a1470 con 0x7f06d810d120 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.666+0000 7f06dd000640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d810d120 0x7f06d81a0ca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009fd0 tx=0x7f06d000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.667+0000 7f06d5600640 1 -- 172.21.15.90:0/3140787311 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d000ce20 con 0x7f06d810d120 2024-09-10T22:03:27.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.667+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d81a1760 con 0x7f06d810d120 2024-09-10T22:03:27.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.667+0000 7f06d5600640 1 -- 172.21.15.90:0/3140787311 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f06d0005020 con 0x7f06d810d120 2024-09-10T22:03:27.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.667+0000 7f06d5600640 1 -- 172.21.15.90:0/3140787311 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0010690 con 0x7f06d810d120 2024-09-10T22:03:27.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.667+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d81a7e20 con 0x7f06d810d120 2024-09-10T22:03:27.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.668+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d81040a0 con 0x7f06d810d120 2024-09-10T22:03:27.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.670+0000 7f06d5600640 1 -- 172.21.15.90:0/3140787311 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f06d0004a40 con 0x7f06d810d120 2024-09-10T22:03:27.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.671+0000 7f06d5600640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f06bc0b5330 0x7f06bc0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:27.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.672+0000 7f06d5600640 1 -- 172.21.15.90:0/3140787311 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f06d00e68d0 con 0x7f06d810d120 2024-09-10T22:03:27.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.672+0000 7f06d7e00640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f06bc0b5330 0x7f06bc0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:27.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.672+0000 7f06d7e00640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f06bc0b5330 0x7f06bc0b77f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06c80096c0 tx=0x7f06c8005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:27.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.675+0000 7f06d5600640 1 -- 172.21.15.90:0/3140787311 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f06d00ad2a0 con 0x7f06d810d120 2024-09-10T22:03:27.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.899+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06d81a8210 con 0x7f06d810d120 2024-09-10T22:03:27.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.899+0000 7f06d5600640 1 -- 172.21.15.90:0/3140787311 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11986 (secure 0 0 0) 0x7f06d00acba0 con 0x7f06d810d120 2024-09-10T22:03:27.901 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:27.902 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"btime":"2024-09-10T22:01:34:736152+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:01:34.015563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:03:27.902 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-09-10T22:03:27.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f06bc0b5330 msgr2=0x7f06bc0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:27.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f06bc0b5330 0x7f06bc0b77f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f06c80096c0 tx=0x7f06c8005e80 comp rx=0 tx=0).stop 2024-09-10T22:03:27.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d810d120 msgr2=0x7f06d81a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:27.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d810d120 0x7f06d81a0ca0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009fd0 tx=0x7f06d000ef10 comp rx=0 tx=0).stop 2024-09-10T22:03:27.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 shutdown_connections 2024-09-10T22:03:27.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f06bc0b5330 0x7f06bc0b77f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06d810d120 0x7f06d81a0ca0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f06d81045e0 0x7f06d819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 --2- 172.21.15.90:0/3140787311 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f06d8103cc0 0x7f06d819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:27.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 >> 172.21.15.90:0/3140787311 conn(0x7f06d80fd700 msgr2=0x7f06d80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:27.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.902+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 shutdown_connections 2024-09-10T22:03:27.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:27.903+0000 7f06df000640 1 -- 172.21.15.90:0/3140787311 wait complete. 2024-09-10T22:03:28.864 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:03:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:28 smithi090 ceph-mon[30350]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:28 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3140787311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:03:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:28 smithi154 ceph-mon[34495]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:28 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3140787311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:03:29.235 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:29.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:28 smithi171 ceph-mon[34138]: pgmap v405: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:29.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:28 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3140787311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:03:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.799+0000 7efea0000640 1 -- 172.21.15.90:0/2378326668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810f390 msgr2=0x7efe98111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.799+0000 7efea0000640 1 --2- 172.21.15.90:0/2378326668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810f390 0x7efe98111780 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efe90009900 tx=0x7efe9002a410 comp rx=0 tx=0).stop 2024-09-10T22:03:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.800+0000 7efea0000640 1 -- 172.21.15.90:0/2378326668 shutdown_connections 2024-09-10T22:03:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.800+0000 7efea0000640 1 --2- 172.21.15.90:0/2378326668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe9810f390 0x7efe98111780 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.800+0000 7efea0000640 1 --2- 172.21.15.90:0/2378326668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe98075e00 0x7efe9810ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.800+0000 7efea0000640 1 --2- 172.21.15.90:0/2378326668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe980754e0 0x7efe980758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.800+0000 7efea0000640 1 -- 172.21.15.90:0/2378326668 >> 172.21.15.90:0/2378326668 conn(0x7efe980fe810 msgr2=0x7efe98100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.800+0000 7efea0000640 1 -- 172.21.15.90:0/2378326668 shutdown_connections 2024-09-10T22:03:29.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.800+0000 7efea0000640 1 -- 172.21.15.90:0/2378326668 wait complete. 2024-09-10T22:03:29.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.801+0000 7efea0000640 1 Processor -- start 2024-09-10T22:03:29.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.801+0000 7efea0000640 1 -- start start 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efea0000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe980754e0 0x7efe9819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efea0000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe98075e00 0x7efe9819b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efea0000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe9810f390 0x7efe981a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efea0000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe98114e60 con 0x7efe980754e0 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efea0000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe98114ce0 con 0x7efe98075e00 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efea0000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efe98114fe0 con 0x7efe9810f390 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9d600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe980754e0 0x7efe9819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9d600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe980754e0 0x7efe9819b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45170/0 (socket says 172.21.15.90:45170) 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9d600640 1 -- 172.21.15.90:0/3398480500 learned_addr learned my addr 172.21.15.90:0/3398480500 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9e000640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe9810f390 0x7efe981a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:29.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9cc00640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe98075e00 0x7efe9819b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9d600640 1 -- 172.21.15.90:0/3398480500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe9810f390 msgr2=0x7efe981a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9d600640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe9810f390 0x7efe981a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9d600640 1 -- 172.21.15.90:0/3398480500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe98075e00 msgr2=0x7efe9819b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9d600640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe98075e00 0x7efe9819b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:29.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.802+0000 7efe9d600640 1 -- 172.21.15.90:0/3398480500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe981a1520 con 0x7efe980754e0 2024-09-10T22:03:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.803+0000 7efe9d600640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe980754e0 0x7efe9819b470 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00ebc0 tx=0x7efe8c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.803+0000 7efe96000640 1 -- 172.21.15.90:0/3398480500 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c010630 con 0x7efe980754e0 2024-09-10T22:03:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.803+0000 7efe96000640 1 -- 172.21.15.90:0/3398480500 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe8c010c60 con 0x7efe980754e0 2024-09-10T22:03:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.803+0000 7efe96000640 1 -- 172.21.15.90:0/3398480500 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c018b00 con 0x7efe980754e0 2024-09-10T22:03:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.803+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe981a1810 con 0x7efe980754e0 2024-09-10T22:03:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.803+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe981a80a0 con 0x7efe980754e0 2024-09-10T22:03:29.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.804+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe980769f0 con 0x7efe980754e0 2024-09-10T22:03:29.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.806+0000 7efe96000640 1 -- 172.21.15.90:0/3398480500 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7efe8c018ca0 con 0x7efe980754e0 2024-09-10T22:03:29.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.808+0000 7efe96000640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe800b53b0 0x7efe800b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:29.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.808+0000 7efe96000640 1 -- 172.21.15.90:0/3398480500 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7efe8c0e6650 con 0x7efe980754e0 2024-09-10T22:03:29.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.808+0000 7efe9cc00640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe800b53b0 0x7efe800b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:29.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.809+0000 7efe9cc00640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe800b53b0 0x7efe800b7870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe98100f10 tx=0x7efe8400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:29.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.811+0000 7efe96000640 1 -- 172.21.15.90:0/3398480500 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efe8c0ac9d0 con 0x7efe980754e0 2024-09-10T22:03:29.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.996+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe981a82a0 con 0x7efe980754e0 2024-09-10T22:03:29.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:29.998+0000 7efe96000640 1 -- 172.21.15.90:0/3398480500 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25227 (secure 0 0 0) 0x7efe8c0b1400 con 0x7efe980754e0 2024-09-10T22:03:30.000 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:30.000 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:02:35.713572+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:03:30.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.000+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe800b53b0 msgr2=0x7efe800b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:30.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.000+0000 7efea0000640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe800b53b0 0x7efe800b7870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efe98100f10 tx=0x7efe8400b390 comp rx=0 tx=0).stop 2024-09-10T22:03:30.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.000+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe980754e0 msgr2=0x7efe9819b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:30.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.000+0000 7efea0000640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe980754e0 0x7efe9819b470 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00ebc0 tx=0x7efe8c00c6a0 comp rx=0 tx=0).stop 2024-09-10T22:03:30.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.001+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 shutdown_connections 2024-09-10T22:03:30.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.001+0000 7efea0000640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe800b53b0 0x7efe800b7870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:30.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.001+0000 7efea0000640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe9810f390 0x7efe981a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:30.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.001+0000 7efea0000640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe98075e00 0x7efe9819b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:30.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.001+0000 7efea0000640 1 --2- 172.21.15.90:0/3398480500 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe980754e0 0x7efe9819b470 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:30.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.001+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 >> 172.21.15.90:0/3398480500 conn(0x7efe980fe810 msgr2=0x7efe98111590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:30.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.001+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 shutdown_connections 2024-09-10T22:03:30.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:30.001+0000 7efea0000640 1 -- 172.21.15.90:0/3398480500 wait complete. 2024-09-10T22:03:30.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:00:35.595646+0000', 'modified': '2024-09-10T22:01:34.015563+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34703, 'mds_1': 34670, 'mds_2': 34646, 'mds_3': 34640, 'mds_4': 34607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14841': {'gid': 14841, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6839/3596805819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 3596805819}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 3596805819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14853': {'gid': 14853, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6841/3169114114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6840', 'nonce': 3169114114}, {'type': 'v1', 'addr': '172.21.15.90:6841', 'nonce': 3169114114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24598': {'gid': 24598, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6833/3906453887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6832', 'nonce': 3906453887}, {'type': 'v1', 'addr': '172.21.15.171:6833', 'nonce': 3906453887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24655': {'gid': 24655, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/2889350064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 2889350064}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 2889350064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24676': {'gid': 24676, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6839/3372214298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 3372214298}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 3372214298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34607': {'gid': 34607, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.154:6833/2781539957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6832', 'nonce': 2781539957}, {'type': 'v1', 'addr': '172.21.15.154:6833', 'nonce': 2781539957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34640': {'gid': 34640, 'name': 'e', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.154:6835/622088003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 622088003}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 622088003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34646': {'gid': 34646, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.171:6835/2969925217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 2969925217}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 2969925217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34670': {'gid': 34670, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.154:6837/3907253955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 3907253955}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 3907253955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34703': {'gid': 34703, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.154:6839/703390433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 703390433}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 703390433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34703, 'qdb_cluster': [34607, 34646, 34640, 34670, 34703]} 2024-09-10T22:03:30.760 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T22:03:30.760 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T22:03:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:30 smithi090 ceph-mon[30350]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:03:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:30 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3398480500' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:03:31.130 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:30 smithi154 ceph-mon[34495]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:03:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:30 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3398480500' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:03:31.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:30 smithi171 ceph-mon[34138]: pgmap v406: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:03:31.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:30 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3398480500' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:03:31.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.771+0000 7feacae00640 1 -- 172.21.15.90:0/1546080629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac410b350 msgr2=0x7feac4111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:31.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.771+0000 7feacae00640 1 --2- 172.21.15.90:0/1546080629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac410b350 0x7feac4111b50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7feabc009ca0 tx=0x7feabc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.771+0000 7feacae00640 1 -- 172.21.15.90:0/1546080629 shutdown_connections 2024-09-10T22:03:31.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.771+0000 7feacae00640 1 --2- 172.21.15.90:0/1546080629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac410b350 0x7feac4111b50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.771+0000 7feacae00640 1 --2- 172.21.15.90:0/1546080629 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac410a9d0 0x7feac410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.771+0000 7feacae00640 1 --2- 172.21.15.90:0/1546080629 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac4104fc0 0x7feac41053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.771+0000 7feacae00640 1 -- 172.21.15.90:0/1546080629 >> 172.21.15.90:0/1546080629 conn(0x7feac41009f0 msgr2=0x7feac4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:31.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.771+0000 7feacae00640 1 -- 172.21.15.90:0/1546080629 shutdown_connections 2024-09-10T22:03:31.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.771+0000 7feacae00640 1 -- 172.21.15.90:0/1546080629 wait complete. 2024-09-10T22:03:31.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feacae00640 1 Processor -- start 2024-09-10T22:03:31.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feacae00640 1 -- start start 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feacae00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac4104fc0 0x7feac419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feacae00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac410a9d0 0x7feac419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feacae00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac410b350 0x7feac41a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feacae00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feac4114e80 con 0x7feac4104fc0 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feacae00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7feac4114d00 con 0x7feac410a9d0 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feacae00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7feac4115000 con 0x7feac410b350 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feac3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac4104fc0 0x7feac419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feac3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac4104fc0 0x7feac419b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45188/0 (socket says 172.21.15.90:45188) 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.772+0000 7feac3e00640 1 -- 172.21.15.90:0/508030482 learned_addr learned my addr 172.21.15.90:0/508030482 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:31.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac8e00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac410b350 0x7feac41a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feabbe00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac410a9d0 0x7feac419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac3e00640 1 -- 172.21.15.90:0/508030482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac410b350 msgr2=0x7feac41a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac3e00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac410b350 0x7feac41a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac3e00640 1 -- 172.21.15.90:0/508030482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac410a9d0 msgr2=0x7feac419b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac3e00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac410a9d0 0x7feac419b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac3e00640 1 -- 172.21.15.90:0/508030482 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac41a1360 con 0x7feac4104fc0 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac3e00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac4104fc0 0x7feac419b2b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feab400d950 tx=0x7feab400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac1600640 1 -- 172.21.15.90:0/508030482 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4017070 con 0x7feac4104fc0 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac1600640 1 -- 172.21.15.90:0/508030482 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feab4009c30 con 0x7feac4104fc0 2024-09-10T22:03:31.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feac1600640 1 -- 172.21.15.90:0/508030482 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab400f620 con 0x7feac4104fc0 2024-09-10T22:03:31.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.773+0000 7feacae00640 1 -- 172.21.15.90:0/508030482 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feac41a1650 con 0x7feac4104fc0 2024-09-10T22:03:31.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.774+0000 7feacae00640 1 -- 172.21.15.90:0/508030482 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feac41a7d10 con 0x7feac4104fc0 2024-09-10T22:03:31.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feacd834740 1 Processor -- start 2024-09-10T22:03:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feacd834740 1 -- start start 2024-09-10T22:03:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feacd834740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6240 0x557c89dc6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feacd834740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 0x557c89dd25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feacd834740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89dd4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feacd834740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c89d3dd50 con 0x557c89dd2af0 2024-09-10T22:03:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feac1600640 1 -- 172.21.15.90:0/508030482 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7feab4010460 con 0x7feac4104fc0 2024-09-10T22:03:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feacd834740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c89d2d180 con 0x557c89dc6240 2024-09-10T22:03:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feacd834740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557c89d30f90 con 0x557c89dc6b60 2024-09-10T22:03:31.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.777+0000 7feabbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89dd4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feac8e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6240 0x557c89dc6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feabbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89dd4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45198/0 (socket says 172.21.15.90:45198) 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feabbe00640 1 -- 172.21.15.90:0/3612240015 learned_addr learned my addr 172.21.15.90:0/3612240015 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feac3e00640 1 --2- 172.21.15.90:0/3612240015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 0x557c89dd25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feabbe00640 1 -- 172.21.15.90:0/3612240015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 msgr2=0x557c89dd25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feabbe00640 1 --2- 172.21.15.90:0/3612240015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 0x557c89dd25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feabbe00640 1 -- 172.21.15.90:0/3612240015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6240 msgr2=0x557c89dc6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feabbe00640 1 --2- 172.21.15.90:0/3612240015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6240 0x557c89dc6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feabbe00640 1 -- 172.21.15.90:0/3612240015 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c89cf7a70 con 0x557c89dd2af0 2024-09-10T22:03:31.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.778+0000 7feabbe00640 1 --2- 172.21.15.90:0/3612240015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89dd4ee0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feaac00cc70 tx=0x7feaac00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbd130acf66e3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.779+0000 7feab9600640 1 -- 172.21.15.90:0/3612240015 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac014070 con 0x557c89dd2af0 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.779+0000 7feac1600640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7feaa80b53b0 0x7feaa80b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.779+0000 7feab9600640 1 -- 172.21.15.90:0/3612240015 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feaac0047a0 con 0x557c89dd2af0 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.779+0000 7feab9600640 1 -- 172.21.15.90:0/3612240015 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac010690 con 0x557c89dd2af0 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.779+0000 7feac1600640 1 -- 172.21.15.90:0/508030482 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7feab40e5d90 con 0x7feac4104fc0 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.779+0000 7feac8e00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7feaa80b53b0 0x7feaa80b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feacd834740 1 -- 172.21.15.90:0/3612240015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 msgr2=0x557c89dd4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feacd834740 1 --2- 172.21.15.90:0/3612240015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89dd4ee0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7feaac00cc70 tx=0x7feaac00e720 comp rx=0 tx=0).stop 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feac8e00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7feaa80b53b0 0x7feaa80b7870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fe0 tx=0x7feabc0093c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:31.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feacd834740 1 -- 172.21.15.90:0/3612240015 shutdown_connections 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feacd834740 1 --2- 172.21.15.90:0/3612240015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89dd4ee0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feacd834740 1 --2- 172.21.15.90:0/3612240015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 0x557c89dd25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feacd834740 1 --2- 172.21.15.90:0/3612240015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6240 0x557c89dc6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feacd834740 1 -- 172.21.15.90:0/3612240015 >> 172.21.15.90:0/3612240015 conn(0x557c89dc4020 msgr2=0x557c89dc5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feacd834740 1 -- 172.21.15.90:0/3612240015 shutdown_connections 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.780+0000 7feacd834740 1 -- 172.21.15.90:0/3612240015 wait complete. 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.781+0000 7feacd834740 1 -- start start 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.781+0000 7feacd834740 1 Processor -- start 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feacd834740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 0x557c89dcfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feacd834740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6b60 0x557c89dcfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feacd834740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89dd0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feacd834740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c89b8e5f0 con 0x557c89dc6b60 2024-09-10T22:03:31.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feacd834740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c89cc8460 con 0x557c89dd2af0 2024-09-10T22:03:31.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feacd834740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557c89dd5840 con 0x557c89dc6240 2024-09-10T22:03:31.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feacd834740 10 client.0 _ll_register_callbacks cb 0x557c89cd8210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:31.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feabbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6b60 0x557c89dcfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feac8e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89dd0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feabbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6b60 0x557c89dcfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45200/0 (socket says 172.21.15.90:45200) 2024-09-10T22:03:31.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feabbe00640 1 -- 172.21.15.90:0/2342108813 learned_addr learned my addr 172.21.15.90:0/2342108813 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:31.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.782+0000 7feac3e00640 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 0x557c89dcfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.783+0000 7feac8e00640 1 -- 172.21.15.90:0/2342108813 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 msgr2=0x557c89dcfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:31.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.783+0000 7feac8e00640 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 0x557c89dcfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.783+0000 7feac8e00640 1 -- 172.21.15.90:0/2342108813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6b60 msgr2=0x557c89dcfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:31.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.783+0000 7feac8e00640 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6b60 0x557c89dcfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.783+0000 7feac8e00640 1 -- 172.21.15.90:0/2342108813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c89a8b540 con 0x557c89dd2af0 2024-09-10T22:03:31.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.783+0000 7feabbe00640 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6b60 0x557c89dcfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.783+0000 7feac8e00640 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89dd0530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feabc006010 tx=0x7feabc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4be4ec30150a2a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.783+0000 7feab2000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.784+0000 7feab2000640 1 -- 172.21.15.90:0/2342108813 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc004000 con 0x557c89dd2af0 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.784+0000 7feacd834740 1 -- 172.21.15.90:0/2342108813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557c89ccaa70 con 0x557c89dd2af0 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.784+0000 7feab2000640 1 -- 172.21.15.90:0/2342108813 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7feabc0041a0 con 0x557c89dd2af0 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.784+0000 7feab2000640 1 -- 172.21.15.90:0/2342108813 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feabc0043a0 con 0x557c89dd2af0 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.784+0000 7feacd834740 10 client.35204 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.784+0000 7feacd834740 1 -- 172.21.15.90:0/2342108813 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c89ce56f0 con 0x557c89dd2af0 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.785+0000 7feab2000640 1 -- 172.21.15.90:0/2342108813 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x557c89ccaa70 con 0x557c89dd2af0 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.785+0000 7feab2000640 1 -- 172.21.15.90:0/2342108813 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7feabc051070 con 0x557c89dd2af0 2024-09-10T22:03:31.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.785+0000 7feacd834740 10 client.35204 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.785+0000 7feacd834740 10 client.35204 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.785+0000 7feacd834740 20 client.35204 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.785+0000 7feacd834740 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x557c89de3870 0x557c89de4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.785+0000 7feacd834740 4 client.35204 mds_command: new command op to 34607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.785+0000 7feacd834740 1 -- 172.21.15.90:0/2342108813 --> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c89d3fca0 con 0x557c89de3870 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.785+0000 7feac3e00640 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x557c89de3870 0x557c89de4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.786+0000 7feab2000640 10 client.35204 ms_handle_connect on v2:172.21.15.154:6832/2781539957 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.786+0000 7feac3e00640 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x557c89de3870 0x557c89de4ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fad4042a7f6ec735 server_cookie=972ce4532bd32140 in_seq=0 out_seq=0 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.787+0000 7feab2000640 1 -- 172.21.15.90:0/2342108813 <== mds.4 v2:172.21.15.154:6832/2781539957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c89d3fca0 con 0x557c89de3870 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.787+0000 7feab2000640 10 client.35204 handle_command_reply: tid=0 2024-09-10T22:03:31.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.787+0000 7feacd834740 1 client.35204 shutdown 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 -- 172.21.15.90:0/2342108813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 msgr2=0x557c89dd0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89dd0530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feabc006010 tx=0x7feabc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 -- 172.21.15.90:0/2342108813 shutdown_connections 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x557c89de3870 0x557c89de4ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89dd0530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6b60 0x557c89dcfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feab2000640 0 client.35204 ms_handle_reset on v2:172.21.15.154:6832/2781539957 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 --2- 172.21.15.90:0/2342108813 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 0x557c89dcfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 -- 172.21.15.90:0/2342108813 >> 172.21.15.90:0/2342108813 conn(0x557c89dc4020 msgr2=0x557c89dc9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 -- 172.21.15.90:0/2342108813 shutdown_connections 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 1 -- 172.21.15.90:0/2342108813 wait complete. 2024-09-10T22:03:31.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.788+0000 7feacd834740 20 client.35204 trim_cache size 0 max 16384 2024-09-10T22:03:31.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.806+0000 7feacd834740 1 Processor -- start 2024-09-10T22:03:31.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.806+0000 7feacd834740 1 -- start start 2024-09-10T22:03:31.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feacd834740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 0x557c89de4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feacd834740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6b60 0x557c89dca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feacd834740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89de2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feacd834740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c89de3020 con 0x557c89dd2af0 2024-09-10T22:03:31.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feacd834740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c89de36f0 con 0x557c89dc6b60 2024-09-10T22:03:31.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feacd834740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557c89de3dc0 con 0x557c89dc6240 2024-09-10T22:03:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feac8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89de2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feac8e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89de2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45204/0 (socket says 172.21.15.90:45204) 2024-09-10T22:03:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feac8e00640 1 -- 172.21.15.90:0/2090477778 learned_addr learned my addr 172.21.15.90:0/2090477778 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feabbe00640 1 --2- 172.21.15.90:0/2090477778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6b60 0x557c89dca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feac3e00640 1 --2- 172.21.15.90:0/2090477778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 0x557c89de4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feac8e00640 1 -- 172.21.15.90:0/2090477778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 msgr2=0x557c89de4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:31.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feac8e00640 1 --2- 172.21.15.90:0/2090477778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 0x557c89de4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feac8e00640 1 -- 172.21.15.90:0/2090477778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6b60 msgr2=0x557c89dca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:31.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feac8e00640 1 --2- 172.21.15.90:0/2090477778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6b60 0x557c89dca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.807+0000 7feac8e00640 1 -- 172.21.15.90:0/2090477778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c89ddf720 con 0x557c89dd2af0 2024-09-10T22:03:31.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.808+0000 7feac8e00640 1 --2- 172.21.15.90:0/2090477778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89de2580 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7feabc043040 tx=0x7feabc053640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc53f2bea591d17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:31.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.808+0000 7feab8c00640 1 -- 172.21.15.90:0/2090477778 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc004200 con 0x557c89dd2af0 2024-09-10T22:03:31.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.808+0000 7feab8c00640 1 -- 172.21.15.90:0/2090477778 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feabc036040 con 0x557c89dd2af0 2024-09-10T22:03:31.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.808+0000 7feab8c00640 1 -- 172.21.15.90:0/2090477778 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feabc0537e0 con 0x557c89dd2af0 2024-09-10T22:03:31.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 -- 172.21.15.90:0/2090477778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 msgr2=0x557c89de2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:31.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 --2- 172.21.15.90:0/2090477778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89de2580 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7feabc043040 tx=0x7feabc053640 comp rx=0 tx=0).stop 2024-09-10T22:03:31.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 -- 172.21.15.90:0/2090477778 shutdown_connections 2024-09-10T22:03:31.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 --2- 172.21.15.90:0/2090477778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dd2af0 0x557c89de2580 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 --2- 172.21.15.90:0/2090477778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dc6b60 0x557c89dca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 --2- 172.21.15.90:0/2090477778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6240 0x557c89de4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 -- 172.21.15.90:0/2090477778 >> 172.21.15.90:0/2090477778 conn(0x557c89dc4020 msgr2=0x557c89dce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:31.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 -- 172.21.15.90:0/2090477778 shutdown_connections 2024-09-10T22:03:31.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 -- 172.21.15.90:0/2090477778 wait complete. 2024-09-10T22:03:31.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.809+0000 7feacd834740 1 -- start start 2024-09-10T22:03:31.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feacd834740 1 Processor -- start 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feacd834740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6240 0x557c89de76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feacd834740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 0x557c89de7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feacd834740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89decca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feacd834740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c89de4b60 con 0x557c89dc6240 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feacd834740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c89de4490 con 0x557c89dd2af0 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feacd834740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557c89de5230 con 0x557c89dc6b60 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feacd834740 10 client.0 _ll_register_callbacks cb 0x7feabc003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feac3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6240 0x557c89de76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feac3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6240 0x557c89de76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45206/0 (socket says 172.21.15.90:45206) 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feac3e00640 1 -- 172.21.15.90:0/974131255 learned_addr learned my addr 172.21.15.90:0/974131255 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:31.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.810+0000 7feabbe00640 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 0x557c89de7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feac8e00640 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89decca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feac3e00640 1 -- 172.21.15.90:0/974131255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 msgr2=0x557c89de7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feac3e00640 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 0x557c89de7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feac3e00640 1 -- 172.21.15.90:0/974131255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 msgr2=0x557c89decca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feac3e00640 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89decca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feac3e00640 1 -- 172.21.15.90:0/974131255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c89ded340 con 0x557c89dc6240 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feac8e00640 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89decca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feac3e00640 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6240 0x557c89de76f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feab40a87a0 tx=0x7feab40b5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9634e679fd3d72e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feab2000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:31.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feab2000640 1 -- 172.21.15.90:0/974131255 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feab4020040 con 0x557c89dc6240 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feab2000640 1 -- 172.21.15.90:0/974131255 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7feab4010780 con 0x557c89dc6240 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.811+0000 7feacd834740 1 -- 172.21.15.90:0/974131255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feabc036040 con 0x557c89dc6240 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feab2000640 1 -- 172.21.15.90:0/974131255 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feab40ac760 con 0x557c89dc6240 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feacd834740 10 client.15465 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feacd834740 1 -- 172.21.15.90:0/974131255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557c89e50b30 con 0x557c89dc6240 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feab2000640 1 -- 172.21.15.90:0/974131255 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feabc036040 con 0x557c89dc6240 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feab2000640 1 -- 172.21.15.90:0/974131255 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7feab40971b0 con 0x557c89dc6240 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feacd834740 10 client.15465 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feacd834740 10 client.15465 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feacd834740 20 client.15465 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feacd834740 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x557c89e556e0 0x557c89e543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:31.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feacd834740 4 client.15465 mds_command: new command op to 34607 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.812+0000 7feacd834740 1 -- 172.21.15.90:0/974131255 --> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] -- command(tid 0: {"prefix": "status"}) -- 0x557c89d3b750 con 0x557c89e556e0 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.813+0000 7feabbe00640 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x557c89e556e0 0x557c89e543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.813+0000 7feabbe00640 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x557c89e556e0 0x557c89e543f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c5035547a7b6faf server_cookie=9ae39b4f3e46fde7 in_seq=0 out_seq=0 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.813+0000 7feab2000640 10 client.15465 ms_handle_connect on v2:172.21.15.154:6832/2781539957 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feab2000640 1 -- 172.21.15.90:0/974131255 <== mds.4 v2:172.21.15.154:6832/2781539957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557c89d3b750 con 0x557c89e556e0 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feab2000640 10 client.15465 handle_command_reply: tid=0 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 client.15465 shutdown 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 -- 172.21.15.90:0/974131255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6240 msgr2=0x557c89de76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6240 0x557c89de76f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7feab40a87a0 tx=0x7feab40b5040 comp rx=0 tx=0).stop 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 -- 172.21.15.90:0/974131255 shutdown_connections 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x557c89e556e0 0x557c89e543f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c89dd2af0 0x557c89decca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c89dc6b60 0x557c89de7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 --2- 172.21.15.90:0/974131255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c89dc6240 0x557c89de76f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feab2000640 0 client.15465 ms_handle_reset on v2:172.21.15.154:6832/2781539957 2024-09-10T22:03:31.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 -- 172.21.15.90:0/974131255 >> 172.21.15.90:0/974131255 conn(0x557c89dc4020 msgr2=0x557c89dc2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:31.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 -- 172.21.15.90:0/974131255 shutdown_connections 2024-09-10T22:03:31.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 1 -- 172.21.15.90:0/974131255 wait complete. 2024-09-10T22:03:31.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.814+0000 7feacd834740 20 client.15465 trim_cache size 0 max 16384 2024-09-10T22:03:31.821 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:31.821 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 4, 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34607, 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 120.081193258, 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 228.319119457 2024-09-10T22:03:31.822 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:31.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 -- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7feaa80b53b0 msgr2=0x7feaa80b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:31.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7feaa80b53b0 0x7feaa80b7870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7feabc005fe0 tx=0x7feabc0093c0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 -- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac4104fc0 msgr2=0x7feac419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:31.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac4104fc0 0x7feac419b2b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7feab400d950 tx=0x7feab400de10 comp rx=0 tx=0).stop 2024-09-10T22:03:31.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 -- 172.21.15.90:0/508030482 shutdown_connections 2024-09-10T22:03:31.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7feaa80b53b0 0x7feaa80b7870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7feac410b350 0x7feac41a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7feac410a9d0 0x7feac419b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 --2- 172.21.15.90:0/508030482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feac4104fc0 0x7feac419b2b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 -- 172.21.15.90:0/508030482 >> 172.21.15.90:0/508030482 conn(0x7feac41009f0 msgr2=0x7feac4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 -- 172.21.15.90:0/508030482 shutdown_connections 2024-09-10T22:03:31.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:31.815+0000 7feacae00640 1 -- 172.21.15.90:0/508030482 wait complete. 2024-09-10T22:03:32.757 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-10T22:03:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:32 smithi090 ceph-mon[30350]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:32 smithi154 ceph-mon[34495]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:32 smithi171 ceph-mon[34138]: pgmap v407: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.733+0000 7f022b600640 1 -- 172.21.15.90:0/2251325073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224113b20 msgr2=0x7f0224115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.733+0000 7f022b600640 1 --2- 172.21.15.90:0/2251325073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224113b20 0x7f0224115f10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f021c009900 tx=0x7f021c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.733+0000 7f022b600640 1 -- 172.21.15.90:0/2251325073 shutdown_connections 2024-09-10T22:03:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.733+0000 7f022b600640 1 --2- 172.21.15.90:0/2251325073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224113b20 0x7f0224115f10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.733+0000 7f022b600640 1 --2- 172.21.15.90:0/2251325073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0224075e60 0x7f02241135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.733+0000 7f022b600640 1 --2- 172.21.15.90:0/2251325073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0224075540 0x7f0224075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.733+0000 7f022b600640 1 -- 172.21.15.90:0/2251325073 >> 172.21.15.90:0/2251325073 conn(0x7f02240fe830 msgr2=0x7f0224100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.733+0000 7f022b600640 1 -- 172.21.15.90:0/2251325073 shutdown_connections 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.733+0000 7f022b600640 1 -- 172.21.15.90:0/2251325073 wait complete. 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.734+0000 7f022b600640 1 Processor -- start 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.734+0000 7f022b600640 1 -- start start 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.734+0000 7f022b600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224075540 0x7f022419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.734+0000 7f022b600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0224075e60 0x7f022419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.734+0000 7f022b600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0224113b20 0x7f02241a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.734+0000 7f022b600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0224119280 con 0x7f0224075540 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.734+0000 7f022b600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0224119100 con 0x7f0224113b20 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.734+0000 7f022b600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0224119400 con 0x7f0224075e60 2024-09-10T22:03:33.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0228c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224075540 0x7f022419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0228c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224075540 0x7f022419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45220/0 (socket says 172.21.15.90:45220) 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0228c00640 1 -- 172.21.15.90:0/4043938777 learned_addr learned my addr 172.21.15.90:0/4043938777 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0229600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0224113b20 0x7f02241a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0223e00640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0224075e60 0x7f022419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0228c00640 1 -- 172.21.15.90:0/4043938777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0224075e60 msgr2=0x7f022419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0228c00640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0224075e60 0x7f022419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0228c00640 1 -- 172.21.15.90:0/4043938777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0224113b20 msgr2=0x7f02241a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0228c00640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0224113b20 0x7f02241a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0228c00640 1 -- 172.21.15.90:0/4043938777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02241a58c0 con 0x7f0224075540 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0223e00640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0224075e60 0x7f022419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0229600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0224113b20 0x7f02241a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0228c00640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224075540 0x7f022419f810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f021400db80 tx=0x7f021400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:33.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.735+0000 7f0221600640 1 -- 172.21.15.90:0/4043938777 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214017070 con 0x7f0224075540 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.736+0000 7f0221600640 1 -- 172.21.15.90:0/4043938777 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0214004020 con 0x7f0224075540 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.736+0000 7f022b600640 1 -- 172.21.15.90:0/4043938777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02241a5bb0 con 0x7f0224075540 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.736+0000 7f0221600640 1 -- 172.21.15.90:0/4043938777 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021400be60 con 0x7f0224075540 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.736+0000 7f022b600640 1 -- 172.21.15.90:0/4043938777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02241ac270 con 0x7f0224075540 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.739+0000 7f022e01a740 1 Processor -- start 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.739+0000 7f0221600640 1 -- 172.21.15.90:0/4043938777 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f0214023060 con 0x7f0224075540 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.739+0000 7f022e01a740 1 -- start start 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f022e01a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 0x559fc4bea620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f022e01a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4beab60 0x559fc4bf65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f022e01a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bf6af0 0x559fc4bf8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f022e01a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559fc4b61d50 con 0x559fc4bf6af0 2024-09-10T22:03:33.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f022e01a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559fc4b51180 con 0x559fc4beab60 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f022e01a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559fc4b54f90 con 0x559fc4bea240 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0223e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bf6af0 0x559fc4bf8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0223e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bf6af0 0x559fc4bf8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45226/0 (socket says 172.21.15.90:45226) 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0223e00640 1 -- 172.21.15.90:0/4238403816 learned_addr learned my addr 172.21.15.90:0/4238403816 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0228c00640 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4beab60 0x559fc4bf65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0229600640 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 0x559fc4bea620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0223e00640 1 -- 172.21.15.90:0/4238403816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 msgr2=0x559fc4bea620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0223e00640 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 0x559fc4bea620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0223e00640 1 -- 172.21.15.90:0/4238403816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4beab60 msgr2=0x559fc4bf65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0223e00640 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4beab60 0x559fc4bf65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.740+0000 7f0223e00640 1 -- 172.21.15.90:0/4238403816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc4b1ba70 con 0x559fc4bf6af0 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.741+0000 7f0229600640 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 0x559fc4bea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:33.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.741+0000 7f0223e00640 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bf6af0 0x559fc4bf8ee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f021000bce0 tx=0x7f021000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5dbe8149992e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.741+0000 7f0221600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02080b5330 0x7f02080b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.741+0000 7f0219c00640 1 -- 172.21.15.90:0/4238403816 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210015070 con 0x559fc4bf6af0 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.741+0000 7f0219c00640 1 -- 172.21.15.90:0/4238403816 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02100118d0 con 0x559fc4bf6af0 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.741+0000 7f0219c00640 1 -- 172.21.15.90:0/4238403816 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02100107d0 con 0x559fc4bf6af0 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.741+0000 7f0221600640 1 -- 172.21.15.90:0/4043938777 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f02140e5fe0 con 0x7f0224075540 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.741+0000 7f0229600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02080b5330 0x7f02080b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f022e01a740 1 -- 172.21.15.90:0/4238403816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bf6af0 msgr2=0x559fc4bf8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f022e01a740 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bf6af0 0x559fc4bf8ee0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f021000bce0 tx=0x7f021000f730 comp rx=0 tx=0).stop 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f022e01a740 1 -- 172.21.15.90:0/4238403816 shutdown_connections 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f022e01a740 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bf6af0 0x559fc4bf8ee0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f022e01a740 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4beab60 0x559fc4bf65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f022e01a740 1 --2- 172.21.15.90:0/4238403816 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 0x559fc4bea620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f022e01a740 1 -- 172.21.15.90:0/4238403816 >> 172.21.15.90:0/4238403816 conn(0x559fc4be8020 msgr2=0x559fc4be9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f022e01a740 1 -- 172.21.15.90:0/4238403816 shutdown_connections 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f0229600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02080b5330 0x7f02080b77f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f021c009230 tx=0x7f021c005bb0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.742+0000 7f022e01a740 1 -- 172.21.15.90:0/4238403816 wait complete. 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.743+0000 7f022e01a740 1 -- start start 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.743+0000 7f022e01a740 1 Processor -- start 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.743+0000 7f022e01a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 0x559fc4bf3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.743+0000 7f022e01a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4bf3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.743+0000 7f022e01a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 0x559fc4bf44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.743+0000 7f022e01a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559fc49b25f0 con 0x559fc4beab60 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.743+0000 7f022e01a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559fc4aec460 con 0x559fc4bf6af0 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.743+0000 7f022e01a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559fc4bf9890 con 0x559fc4bea240 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f022e01a740 10 client.0 _ll_register_callbacks cb 0x559fc4afc210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0223e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4bf3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0223e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4bf3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45242/0 (socket says 172.21.15.90:45242) 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0223e00640 1 -- 172.21.15.90:0/4206742308 learned_addr learned my addr 172.21.15.90:0/4206742308 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0229600640 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 0x559fc4bf44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0228c00640 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 0x559fc4bf3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0223e00640 1 -- 172.21.15.90:0/4206742308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 msgr2=0x559fc4bf3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0223e00640 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 0x559fc4bf3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0223e00640 1 -- 172.21.15.90:0/4206742308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 msgr2=0x559fc4bf44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0223e00640 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 0x559fc4bf44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.744+0000 7f0223e00640 1 -- 172.21.15.90:0/4206742308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc48af540 con 0x559fc4beab60 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.745+0000 7f0223e00640 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4bf3f80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f021000b9f0 tx=0x7f021000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d4b69756eebecf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.745+0000 7f020e000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.745+0000 7f020e000640 1 -- 172.21.15.90:0/4206742308 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210028060 con 0x559fc4beab60 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.745+0000 7f020e000640 1 -- 172.21.15.90:0/4206742308 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f02100053d0 con 0x559fc4beab60 2024-09-10T22:03:33.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.745+0000 7f022e01a740 1 -- 172.21.15.90:0/4206742308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559fc4aeea70 con 0x559fc4beab60 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.745+0000 7f020e000640 1 -- 172.21.15.90:0/4206742308 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f021002f050 con 0x559fc4beab60 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.745+0000 7f022e01a740 10 client.15477 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.745+0000 7f022e01a740 1 -- 172.21.15.90:0/4206742308 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559fc4b096f0 con 0x559fc4beab60 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.745+0000 7f020e000640 1 -- 172.21.15.90:0/4206742308 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559fc4aeea70 con 0x559fc4beab60 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.746+0000 7f020e000640 1 -- 172.21.15.90:0/4206742308 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f02100106e0 con 0x559fc4beab60 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.746+0000 7f022e01a740 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.746+0000 7f022e01a740 10 client.15477 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.746+0000 7f022e01a740 20 client.15477 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.746+0000 7f022e01a740 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559fc4c07460 0x559fc4c088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.746+0000 7f022e01a740 4 client.15477 mds_command: new command op to 34640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.746+0000 7f022e01a740 1 -- 172.21.15.90:0/4206742308 --> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fc4b63ca0 con 0x559fc4c07460 2024-09-10T22:03:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.746+0000 7f0228c00640 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559fc4c07460 0x559fc4c088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.747+0000 7f0228c00640 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559fc4c07460 0x559fc4c088d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eea3717d9e55cdb2 server_cookie=6898e67ffa27d489 in_seq=0 out_seq=0 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.747+0000 7f020e000640 10 client.15477 ms_handle_connect on v2:172.21.15.154:6834/622088003 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f020e000640 1 -- 172.21.15.90:0/4206742308 <== mds.3 v2:172.21.15.154:6834/622088003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559fc4b63ca0 con 0x559fc4c07460 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f020e000640 10 client.15477 handle_command_reply: tid=0 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 client.15477 shutdown 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 -- 172.21.15.90:0/4206742308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 msgr2=0x559fc4bf3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4bf3f80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f021000b9f0 tx=0x7f021000bf10 comp rx=0 tx=0).stop 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 -- 172.21.15.90:0/4206742308 shutdown_connections 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 0x559fc4bf44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4bf3f80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559fc4c07460 0x559fc4c088d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 --2- 172.21.15.90:0/4206742308 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bea240 0x559fc4bf3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 -- 172.21.15.90:0/4206742308 >> 172.21.15.90:0/4206742308 conn(0x559fc4be8020 msgr2=0x559fc4bedc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f020e000640 0 client.15477 ms_handle_reset on v2:172.21.15.154:6834/622088003 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 -- 172.21.15.90:0/4206742308 shutdown_connections 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 1 -- 172.21.15.90:0/4206742308 wait complete. 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.748+0000 7f022e01a740 20 client.15477 trim_cache size 0 max 16384 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.767+0000 7f022e01a740 1 Processor -- start 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.767+0000 7f022e01a740 1 -- start start 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f022e01a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bea240 0x559fc4bee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f022e01a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4beab60 0x559fc4be9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f022e01a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 0x559fc4bf4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f022e01a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559fc4c06c10 con 0x559fc4bea240 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f022e01a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559fc4c072e0 con 0x559fc4bf6af0 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f022e01a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559fc4c079b0 con 0x559fc4beab60 2024-09-10T22:03:33.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0228c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bea240 0x559fc4bee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0228c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bea240 0x559fc4bee400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45250/0 (socket says 172.21.15.90:45250) 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0228c00640 1 -- 172.21.15.90:0/11588727 learned_addr learned my addr 172.21.15.90:0/11588727 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0229600640 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 0x559fc4bf4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0223e00640 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4beab60 0x559fc4be9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0228c00640 1 -- 172.21.15.90:0/11588727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4beab60 msgr2=0x559fc4be9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0228c00640 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4beab60 0x559fc4be9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0228c00640 1 -- 172.21.15.90:0/11588727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 msgr2=0x559fc4bf4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0228c00640 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 0x559fc4bf4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0228c00640 1 -- 172.21.15.90:0/11588727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc4bf65f0 con 0x559fc4bea240 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0229600640 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 0x559fc4bf4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.768+0000 7f0223e00640 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4beab60 0x559fc4be9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.769+0000 7f0228c00640 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bea240 0x559fc4bee400 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f02140ac240 tx=0x7f02140e9a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facc4b8f09bda35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:33.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.769+0000 7f0219200640 1 -- 172.21.15.90:0/11588727 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02140aba20 con 0x559fc4bea240 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.769+0000 7f0219200640 1 -- 172.21.15.90:0/11588727 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f02140b5180 con 0x559fc4bea240 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.769+0000 7f0219200640 1 -- 172.21.15.90:0/11588727 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02140b0d90 con 0x559fc4bea240 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.769+0000 7f022e01a740 1 -- 172.21.15.90:0/11588727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bea240 msgr2=0x559fc4bee400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.769+0000 7f022e01a740 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bea240 0x559fc4bee400 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f02140ac240 tx=0x7f02140e9a20 comp rx=0 tx=0).stop 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.770+0000 7f022e01a740 1 -- 172.21.15.90:0/11588727 shutdown_connections 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.770+0000 7f022e01a740 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bf6af0 0x559fc4bf4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.770+0000 7f022e01a740 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4beab60 0x559fc4be9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.770+0000 7f022e01a740 1 --2- 172.21.15.90:0/11588727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4bea240 0x559fc4bee400 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.770+0000 7f022e01a740 1 -- 172.21.15.90:0/11588727 >> 172.21.15.90:0/11588727 conn(0x559fc4be8020 msgr2=0x559fc4c01b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.770+0000 7f022e01a740 1 -- 172.21.15.90:0/11588727 shutdown_connections 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.770+0000 7f022e01a740 1 -- 172.21.15.90:0/11588727 wait complete. 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.770+0000 7f022e01a740 1 -- start start 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.770+0000 7f022e01a740 1 Processor -- start 2024-09-10T22:03:33.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f022e01a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bea240 0x559fc4c0bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f022e01a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4c0c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f022e01a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bf6af0 0x559fc4c109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f022e01a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559fc4c08750 con 0x559fc4beab60 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f022e01a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x559fc4c08080 con 0x559fc4bea240 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f022e01a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x559fc4c08e20 con 0x559fc4bf6af0 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f022e01a740 10 client.0 _ll_register_callbacks cb 0x7f021000bdd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0223e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4c0c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0223e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4c0c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45262/0 (socket says 172.21.15.90:45262) 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0223e00640 1 -- 172.21.15.90:0/2095506154 learned_addr learned my addr 172.21.15.90:0/2095506154 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0228c00640 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bea240 0x559fc4c0bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0229600640 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bf6af0 0x559fc4c109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0223e00640 1 -- 172.21.15.90:0/2095506154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bf6af0 msgr2=0x559fc4c109e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:33.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0223e00640 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bf6af0 0x559fc4c109e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0223e00640 1 -- 172.21.15.90:0/2095506154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bea240 msgr2=0x559fc4c0bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:33.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0223e00640 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bea240 0x559fc4c0bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0223e00640 1 -- 172.21.15.90:0/2095506154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc4c11080 con 0x559fc4beab60 2024-09-10T22:03:33.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0228c00640 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bea240 0x559fc4c0bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:33.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0229600640 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bf6af0 0x559fc4c109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:33.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f0223e00640 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4c0c0b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f021000ba40 tx=0x7f021000fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e887cbe13498d2b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:33.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.771+0000 7f020e000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:33.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.772+0000 7f020e000640 1 -- 172.21.15.90:0/2095506154 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0210028040 con 0x559fc4beab60 2024-09-10T22:03:33.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.772+0000 7f020e000640 1 -- 172.21.15.90:0/2095506154 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f0210010440 con 0x559fc4beab60 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.772+0000 7f022e01a740 1 -- 172.21.15.90:0/2095506154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559fc4b5f750 con 0x559fc4beab60 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.772+0000 7f020e000640 1 -- 172.21.15.90:0/2095506154 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0210015070 con 0x559fc4beab60 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.772+0000 7f022e01a740 10 client.15483 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.772+0000 7f022e01a740 1 -- 172.21.15.90:0/2095506154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559fc4c746c0 con 0x559fc4beab60 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.772+0000 7f020e000640 1 -- 172.21.15.90:0/2095506154 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x559fc4b5f750 con 0x559fc4beab60 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.772+0000 7f020e000640 1 -- 172.21.15.90:0/2095506154 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f021001a810 con 0x559fc4beab60 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.773+0000 7f022e01a740 10 client.15483 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.773+0000 7f022e01a740 10 client.15483 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.773+0000 7f022e01a740 20 client.15483 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.773+0000 7f022e01a740 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559fc4c79310 0x559fc4c78020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:33.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.773+0000 7f022e01a740 4 client.15483 mds_command: new command op to 34640 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.773+0000 7f022e01a740 1 -- 172.21.15.90:0/2095506154 --> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] -- command(tid 0: {"prefix": "status"}) -- 0x559fc4b5cb80 con 0x559fc4c79310 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.773+0000 7f0228c00640 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559fc4c79310 0x559fc4c78020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f0228c00640 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559fc4c79310 0x559fc4c78020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c37dfce5faf1471 server_cookie=2acd5757f30df6a4 in_seq=0 out_seq=0 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f020e000640 10 client.15483 ms_handle_connect on v2:172.21.15.154:6834/622088003 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f020e000640 1 -- 172.21.15.90:0/2095506154 <== mds.3 v2:172.21.15.154:6834/622088003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559fc4b5cb80 con 0x559fc4c79310 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f020e000640 10 client.15483 handle_command_reply: tid=0 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 client.15483 shutdown 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 -- 172.21.15.90:0/2095506154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 msgr2=0x559fc4c0c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4c0c0b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f021000ba40 tx=0x7f021000fd60 comp rx=0 tx=0).stop 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 -- 172.21.15.90:0/2095506154 shutdown_connections 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x559fc4c79310 0x559fc4c78020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x559fc4bf6af0 0x559fc4c109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559fc4beab60 0x559fc4c0c0b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f020e000640 0 client.15483 ms_handle_reset on v2:172.21.15.154:6834/622088003 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 --2- 172.21.15.90:0/2095506154 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x559fc4bea240 0x559fc4c0bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 -- 172.21.15.90:0/2095506154 >> 172.21.15.90:0/2095506154 conn(0x559fc4be8020 msgr2=0x559fc4bf5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 -- 172.21.15.90:0/2095506154 shutdown_connections 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.774+0000 7f022e01a740 1 -- 172.21.15.90:0/2095506154 wait complete. 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022e01a740 20 client.15483 trim_cache size 0 max 16384 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 3, 2024-09-10T22:03:33.845 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34640, 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 123.05657994000001, 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 216.24049395200001 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:33.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 -- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02080b5330 msgr2=0x7f02080b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02080b5330 0x7f02080b77f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f021c009230 tx=0x7f021c005bb0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 -- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224075540 msgr2=0x7f022419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224075540 0x7f022419f810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f021400db80 tx=0x7f021400b630 comp rx=0 tx=0).stop 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 -- 172.21.15.90:0/4043938777 shutdown_connections 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f02080b5330 0x7f02080b77f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0224113b20 0x7f02241a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0224075e60 0x7f022419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 --2- 172.21.15.90:0/4043938777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0224075540 0x7f022419f810 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 -- 172.21.15.90:0/4043938777 >> 172.21.15.90:0/4043938777 conn(0x7f02240fe830 msgr2=0x7f02240ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:33.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 -- 172.21.15.90:0/4043938777 shutdown_connections 2024-09-10T22:03:33.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:33.775+0000 7f022b600640 1 -- 172.21.15.90:0/4043938777 wait complete. 2024-09-10T22:03:34.980 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T22:03:35.050 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:34 smithi171 ceph-mon[34138]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:34 smithi090 ceph-mon[30350]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:34 smithi154 ceph-mon[34495]: pgmap v408: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:35.349 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:36.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:35.998+0000 7f109e800640 1 -- 172.21.15.90:0/3391098943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f109810b320 msgr2=0x7f1098111b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:36.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:35.998+0000 7f109e800640 1 --2- 172.21.15.90:0/3391098943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f109810b320 0x7f1098111b20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1090009900 tx=0x7f109002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:35.999+0000 7f109e800640 1 -- 172.21.15.90:0/3391098943 shutdown_connections 2024-09-10T22:03:36.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:35.999+0000 7f109e800640 1 --2- 172.21.15.90:0/3391098943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f109810b320 0x7f1098111b20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:35.999+0000 7f109e800640 1 --2- 172.21.15.90:0/3391098943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109810a9a0 0x7f109810ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:35.999+0000 7f109e800640 1 --2- 172.21.15.90:0/3391098943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1098104f90 0x7f1098105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:35.999+0000 7f109e800640 1 -- 172.21.15.90:0/3391098943 >> 172.21.15.90:0/3391098943 conn(0x7f1098100a00 msgr2=0x7f1098102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:36.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:35.999+0000 7f109e800640 1 -- 172.21.15.90:0/3391098943 shutdown_connections 2024-09-10T22:03:36.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:35.999+0000 7f109e800640 1 -- 172.21.15.90:0/3391098943 wait complete. 2024-09-10T22:03:36.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.000+0000 7f109e800640 1 Processor -- start 2024-09-10T22:03:36.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.000+0000 7f109e800640 1 -- start start 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.000+0000 7f109e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1098104f90 0x7f109819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.000+0000 7f109e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109810a9a0 0x7f109819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.000+0000 7f109e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f109810b320 0x7f10981a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.000+0000 7f109e800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1098114e00 con 0x7f1098104f90 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.000+0000 7f109e800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1098114c80 con 0x7f109810a9a0 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.000+0000 7f109e800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1098114f80 con 0x7f109810b320 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1098104f90 0x7f109819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1098104f90 0x7f109819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45280/0 (socket says 172.21.15.90:45280) 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097400640 1 -- 172.21.15.90:0/234275590 learned_addr learned my addr 172.21.15.90:0/234275590 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f108ea00640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109810a9a0 0x7f109819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097e00640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f109810b320 0x7f10981a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097400640 1 -- 172.21.15.90:0/234275590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f109810b320 msgr2=0x7f10981a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097400640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f109810b320 0x7f10981a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097400640 1 -- 172.21.15.90:0/234275590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109810a9a0 msgr2=0x7f109819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097400640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109810a9a0 0x7f109819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097400640 1 -- 172.21.15.90:0/234275590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981a13e0 con 0x7f1098104f90 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.001+0000 7f1097400640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1098104f90 0x7f109819b330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f108800db80 tx=0x7f108800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.002+0000 7f1094c00640 1 -- 172.21.15.90:0/234275590 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1088017070 con 0x7f1098104f90 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.002+0000 7f1094c00640 1 -- 172.21.15.90:0/234275590 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1088004020 con 0x7f1098104f90 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.002+0000 7f109e800640 1 -- 172.21.15.90:0/234275590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981a16d0 con 0x7f1098104f90 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.002+0000 7f1094c00640 1 -- 172.21.15.90:0/234275590 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108800be60 con 0x7f1098104f90 2024-09-10T22:03:36.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.002+0000 7f109e800640 1 -- 172.21.15.90:0/234275590 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981a7d90 con 0x7f1098104f90 2024-09-10T22:03:36.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.005+0000 7f10a140d740 1 Processor -- start 2024-09-10T22:03:36.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.005+0000 7f10a140d740 1 -- start start 2024-09-10T22:03:36.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.005+0000 7f1094c00640 1 -- 172.21.15.90:0/234275590 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f1088023060 con 0x7f1098104f90 2024-09-10T22:03:36.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.005+0000 7f10a140d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 0x5653cfc25620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.005+0000 7f10a140d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 0x5653cfc315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.005+0000 7f10a140d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc31af0 0x5653cfc33ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.005+0000 7f10a140d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5653cfb9cd50 con 0x5653cfc31af0 2024-09-10T22:03:36.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.005+0000 7f10a140d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5653cfb8c180 con 0x5653cfc25b60 2024-09-10T22:03:36.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.005+0000 7f10a140d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5653cfb8ff90 con 0x5653cfc25240 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f108ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc31af0 0x5653cfc33ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f108ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc31af0 0x5653cfc33ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45282/0 (socket says 172.21.15.90:45282) 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f108ea00640 1 -- 172.21.15.90:0/2274915639 learned_addr learned my addr 172.21.15.90:0/2274915639 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f1097400640 1 --2- 172.21.15.90:0/2274915639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 0x5653cfc315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f1097e00640 1 --2- 172.21.15.90:0/2274915639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 0x5653cfc25620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f108ea00640 1 -- 172.21.15.90:0/2274915639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 msgr2=0x5653cfc25620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f108ea00640 1 --2- 172.21.15.90:0/2274915639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 0x5653cfc25620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f108ea00640 1 -- 172.21.15.90:0/2274915639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 msgr2=0x5653cfc315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f108ea00640 1 --2- 172.21.15.90:0/2274915639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 0x5653cfc315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.006+0000 7f108ea00640 1 -- 172.21.15.90:0/2274915639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cfb56a70 con 0x5653cfc31af0 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.007+0000 7f108ea00640 1 --2- 172.21.15.90:0/2274915639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc31af0 0x5653cfc33ee0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f108000bc60 tx=0x7f10800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4247df09f02352c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:36.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.007+0000 7f108cc00640 1 -- 172.21.15.90:0/2274915639 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108001c480 con 0x5653cfc31af0 2024-09-10T22:03:36.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.007+0000 7f108cc00640 1 -- 172.21.15.90:0/2274915639 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f108001cab0 con 0x5653cfc31af0 2024-09-10T22:03:36.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.007+0000 7f108cc00640 1 -- 172.21.15.90:0/2274915639 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10800119f0 con 0x5653cfc31af0 2024-09-10T22:03:36.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f1094c00640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f107c0b5330 0x7f107c0b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f10a140d740 1 -- 172.21.15.90:0/2274915639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc31af0 msgr2=0x5653cfc33ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:36.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f10a140d740 1 --2- 172.21.15.90:0/2274915639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc31af0 0x5653cfc33ee0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f108000bc60 tx=0x7f10800107a0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f1094c00640 1 -- 172.21.15.90:0/234275590 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f10880e5f50 con 0x7f1098104f90 2024-09-10T22:03:36.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f1097e00640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f107c0b5330 0x7f107c0b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f10a140d740 1 -- 172.21.15.90:0/2274915639 shutdown_connections 2024-09-10T22:03:36.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f10a140d740 1 --2- 172.21.15.90:0/2274915639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc31af0 0x5653cfc33ee0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f10a140d740 1 --2- 172.21.15.90:0/2274915639 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 0x5653cfc315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f10a140d740 1 --2- 172.21.15.90:0/2274915639 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 0x5653cfc25620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f10a140d740 1 -- 172.21.15.90:0/2274915639 >> 172.21.15.90:0/2274915639 conn(0x5653cfc23020 msgr2=0x5653cfc24210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f10a140d740 1 -- 172.21.15.90:0/2274915639 shutdown_connections 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.008+0000 7f10a140d740 1 -- 172.21.15.90:0/2274915639 wait complete. 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.009+0000 7f10a140d740 1 -- start start 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.009+0000 7f1097e00640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f107c0b5330 0x7f107c0b77f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1090005ab0 tx=0x7f1090005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.009+0000 7f10a140d740 1 Processor -- start 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.009+0000 7f10a140d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc2ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.009+0000 7f10a140d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25b60 0x5653cfc2efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.009+0000 7f10a140d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 0x5653cfc2f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.009+0000 7f10a140d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5653cf9ed5f0 con 0x5653cfc25240 2024-09-10T22:03:36.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.009+0000 7f10a140d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5653cfb27460 con 0x5653cfc31af0 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.009+0000 7f10a140d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5653cfc34840 con 0x5653cfc25b60 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f10a140d740 10 client.0 _ll_register_callbacks cb 0x5653cfb37210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f1097400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc2ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f1097400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc2ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45288/0 (socket says 172.21.15.90:45288) 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f1097400640 1 -- 172.21.15.90:0/3902110239 learned_addr learned my addr 172.21.15.90:0/3902110239 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f1097e00640 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 0x5653cfc2f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f108ea00640 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25b60 0x5653cfc2efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f1097400640 1 -- 172.21.15.90:0/3902110239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25b60 msgr2=0x5653cfc2efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f1097400640 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25b60 0x5653cfc2efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f1097400640 1 -- 172.21.15.90:0/3902110239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 msgr2=0x5653cfc2f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f1097400640 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 0x5653cfc2f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f1097400640 1 -- 172.21.15.90:0/3902110239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cf8ea540 con 0x5653cfc25240 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.010+0000 7f108ea00640 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25b60 0x5653cfc2efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:36.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f1097400640 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc2ea80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f10880a87f0 tx=0x7f1088004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee2798611a9c55f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:36.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f1085600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:36.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f1085600640 1 -- 172.21.15.90:0/3902110239 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10880b50f0 con 0x5653cfc25240 2024-09-10T22:03:36.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f1085600640 1 -- 172.21.15.90:0/3902110239 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f10880cf110 con 0x5653cfc25240 2024-09-10T22:03:36.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f10a140d740 1 -- 172.21.15.90:0/3902110239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653cfb29a70 con 0x5653cfc25240 2024-09-10T22:03:36.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f1085600640 1 -- 172.21.15.90:0/3902110239 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10880abd30 con 0x5653cfc25240 2024-09-10T22:03:36.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f10a140d740 10 client.15495 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f10a140d740 1 -- 172.21.15.90:0/3902110239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653cfb446f0 con 0x5653cfc25240 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f1085600640 1 -- 172.21.15.90:0/3902110239 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5653cfb29a70 con 0x5653cfc25240 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.011+0000 7f1085600640 1 -- 172.21.15.90:0/3902110239 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f10880e9050 con 0x5653cfc25240 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.012+0000 7f10a140d740 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.012+0000 7f10a140d740 10 client.15495 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.012+0000 7f10a140d740 20 client.15495 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.012+0000 7f10a140d740 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x5653cfc42800 0x5653cfc43c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.012+0000 7f10a140d740 4 client.15495 mds_command: new command op to 34646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.012+0000 7f10a140d740 1 -- 172.21.15.90:0/3902110239 --> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653cfb9eca0 con 0x5653cfc42800 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.013+0000 7f108ea00640 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x5653cfc42800 0x5653cfc43c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.013+0000 7f108ea00640 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x5653cfc42800 0x5653cfc43c70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ecccc3cb3b7a879 server_cookie=b2e4b3a5be0ca4ad in_seq=0 out_seq=0 2024-09-10T22:03:36.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.013+0000 7f1085600640 10 client.15495 ms_handle_connect on v2:172.21.15.171:6834/2969925217 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.014+0000 7f1085600640 1 -- 172.21.15.90:0/3902110239 <== mds.2 v2:172.21.15.171:6834/2969925217 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5653cfb9eca0 con 0x5653cfc42800 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.014+0000 7f1085600640 10 client.15495 handle_command_reply: tid=0 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.014+0000 7f10a140d740 1 client.15495 shutdown 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 -- 172.21.15.90:0/3902110239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 msgr2=0x5653cfc2ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc2ea80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f10880a87f0 tx=0x7f1088004700 comp rx=0 tx=0).stop 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 -- 172.21.15.90:0/3902110239 shutdown_connections 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x5653cfc42800 0x5653cfc43c70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 0x5653cfc2f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25b60 0x5653cfc2efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 --2- 172.21.15.90:0/3902110239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc2ea80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 -- 172.21.15.90:0/3902110239 >> 172.21.15.90:0/3902110239 conn(0x5653cfc23020 msgr2=0x5653cfc28c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:36.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f1085600640 0 client.15495 ms_handle_reset on v2:172.21.15.171:6834/2969925217 2024-09-10T22:03:36.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 -- 172.21.15.90:0/3902110239 shutdown_connections 2024-09-10T22:03:36.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 1 -- 172.21.15.90:0/3902110239 wait complete. 2024-09-10T22:03:36.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.015+0000 7f10a140d740 20 client.15495 trim_cache size 0 max 16384 2024-09-10T22:03:36.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f10a140d740 1 Processor -- start 2024-09-10T22:03:36.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f10a140d740 1 -- start start 2024-09-10T22:03:36.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f10a140d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 0x5653cfc43c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f10a140d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25b60 0x5653cfc416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f10a140d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 0x5653cfc292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f10a140d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5653cfc41fb0 con 0x5653cfc25b60 2024-09-10T22:03:36.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f10a140d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5653cfc42680 con 0x5653cfc31af0 2024-09-10T22:03:36.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f10a140d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5653cfc42d50 con 0x5653cfc25240 2024-09-10T22:03:36.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f108ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25b60 0x5653cfc416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f108ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25b60 0x5653cfc416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45304/0 (socket says 172.21.15.90:45304) 2024-09-10T22:03:36.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.034+0000 7f108ea00640 1 -- 172.21.15.90:0/195197364 learned_addr learned my addr 172.21.15.90:0/195197364 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:36.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f108ea00640 1 -- 172.21.15.90:0/195197364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 msgr2=0x5653cfc43c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:36.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f1097400640 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 0x5653cfc43c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f1097e00640 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 0x5653cfc292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f108ea00640 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 0x5653cfc43c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f108ea00640 1 -- 172.21.15.90:0/195197364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 msgr2=0x5653cfc292b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f108ea00640 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 0x5653cfc292b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f108ea00640 1 -- 172.21.15.90:0/195197364 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cfc25990 con 0x5653cfc25b60 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f1097e00640 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 0x5653cfc292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f108ea00640 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25b60 0x5653cfc416a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x5653cfc251f0 tx=0x7f1080005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78291784c2eb9ef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f1087e00640 1 -- 172.21.15.90:0/195197364 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f108000bd50 con 0x5653cfc25b60 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f1087e00640 1 -- 172.21.15.90:0/195197364 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10800054d0 con 0x5653cfc25b60 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.035+0000 7f1087e00640 1 -- 172.21.15.90:0/195197364 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1080004d60 con 0x5653cfc25b60 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.036+0000 7f10a140d740 1 -- 172.21.15.90:0/195197364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25b60 msgr2=0x5653cfc416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.036+0000 7f10a140d740 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25b60 0x5653cfc416a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x5653cfc251f0 tx=0x7f1080005850 comp rx=0 tx=0).stop 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.036+0000 7f10a140d740 1 -- 172.21.15.90:0/195197364 shutdown_connections 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.036+0000 7f10a140d740 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc31af0 0x5653cfc292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.036+0000 7f10a140d740 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25b60 0x5653cfc416a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.036+0000 7f10a140d740 1 --2- 172.21.15.90:0/195197364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc25240 0x5653cfc43c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.036+0000 7f10a140d740 1 -- 172.21.15.90:0/195197364 >> 172.21.15.90:0/195197364 conn(0x5653cfc23020 msgr2=0x5653cfc2d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:36.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.036+0000 7f10a140d740 1 -- 172.21.15.90:0/195197364 shutdown_connections 2024-09-10T22:03:36.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.036+0000 7f10a140d740 1 -- 172.21.15.90:0/195197364 wait complete. 2024-09-10T22:03:36.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.037+0000 7f10a140d740 1 -- start start 2024-09-10T22:03:36.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.037+0000 7f10a140d740 1 Processor -- start 2024-09-10T22:03:36.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.037+0000 7f10a140d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc45eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.037+0000 7f10a140d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 0x5653cfc463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.037+0000 7f10a140d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc31af0 0x5653cfc46930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.037+0000 7f10a140d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5653cfc43af0 con 0x5653cfc25240 2024-09-10T22:03:36.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.037+0000 7f10a140d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5653cfc43420 con 0x5653cfc25b60 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.037+0000 7f10a140d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5653cfc441c0 con 0x5653cfc31af0 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.037+0000 7f10a140d740 10 client.0 _ll_register_callbacks cb 0x7f10880a80c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc45eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc45eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45314/0 (socket says 172.21.15.90:45314) 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097400640 1 -- 172.21.15.90:0/775748099 learned_addr learned my addr 172.21.15.90:0/775748099 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f108ea00640 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 0x5653cfc463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097e00640 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc31af0 0x5653cfc46930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097400640 1 -- 172.21.15.90:0/775748099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc31af0 msgr2=0x5653cfc46930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097400640 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc31af0 0x5653cfc46930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097400640 1 -- 172.21.15.90:0/775748099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 msgr2=0x5653cfc463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097400640 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 0x5653cfc463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097400640 1 -- 172.21.15.90:0/775748099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cfc4b6c0 con 0x5653cfc25240 2024-09-10T22:03:36.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1097400640 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc45eb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1088000f80 tx=0x7f10880b0d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2579a8ab167aec2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1085600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1085600640 1 -- 172.21.15.90:0/775748099 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10880abb70 con 0x5653cfc25240 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f1085600640 1 -- 172.21.15.90:0/775748099 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f10880ab750 con 0x5653cfc25240 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.038+0000 7f10a140d740 1 -- 172.21.15.90:0/775748099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10800054d0 con 0x5653cfc25240 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f1085600640 1 -- 172.21.15.90:0/775748099 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10880b50f0 con 0x5653cfc25240 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f10a140d740 10 client.15501 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f10a140d740 1 -- 172.21.15.90:0/775748099 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653cfcaf430 con 0x5653cfc25240 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f1085600640 1 -- 172.21.15.90:0/775748099 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10800054d0 con 0x5653cfc25240 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f1085600640 1 -- 172.21.15.90:0/775748099 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f10880a7d00 con 0x5653cfc25240 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f10a140d740 10 client.15501 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f10a140d740 10 client.15501 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f10a140d740 20 client.15501 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:36.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f10a140d740 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x5653cfcb3e60 0x5653cfcb2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f10a140d740 4 client.15501 mds_command: new command op to 34646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.039+0000 7f10a140d740 1 -- 172.21.15.90:0/775748099 --> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] -- command(tid 0: {"prefix": "status"}) -- 0x5653cfb9a750 con 0x5653cfcb3e60 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.040+0000 7f108ea00640 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x5653cfcb3e60 0x5653cfcb2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.040+0000 7f108ea00640 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x5653cfcb3e60 0x5653cfcb2b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97a0314bcd8a8b0b server_cookie=8244be61d54d3dd9 in_seq=0 out_seq=0 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.040+0000 7f1085600640 10 client.15501 ms_handle_connect on v2:172.21.15.171:6834/2969925217 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f1085600640 1 -- 172.21.15.90:0/775748099 <== mds.2 v2:172.21.15.171:6834/2969925217 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5653cfb9a750 con 0x5653cfcb3e60 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f1085600640 10 client.15501 handle_command_reply: tid=0 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 client.15501 shutdown 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 -- 172.21.15.90:0/775748099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 msgr2=0x5653cfc45eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:36.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc45eb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1088000f80 tx=0x7f10880b0d00 comp rx=0 tx=0).stop 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 -- 172.21.15.90:0/775748099 shutdown_connections 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x5653cfcb3e60 0x5653cfcb2b70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5653cfc31af0 0x5653cfc46930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5653cfc25b60 0x5653cfc463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 --2- 172.21.15.90:0/775748099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5653cfc25240 0x5653cfc45eb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 -- 172.21.15.90:0/775748099 >> 172.21.15.90:0/775748099 conn(0x5653cfc23020 msgr2=0x5653cfc240c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f1085600640 0 client.15501 ms_handle_reset on v2:172.21.15.171:6834/2969925217 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 -- 172.21.15.90:0/775748099 shutdown_connections 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 1 -- 172.21.15.90:0/775748099 wait complete. 2024-09-10T22:03:36.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.041+0000 7f10a140d740 20 client.15501 trim_cache size 0 max 16384 2024-09-10T22:03:36.048 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:36.048 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:03:36.048 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-09-10T22:03:36.048 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34646, 2024-09-10T22:03:36.048 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:03:36.048 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:03:36.048 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:03:36.048 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 126.32701316000001, 2024-09-10T22:03:36.049 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:03:36.049 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:03:36.049 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:03:36.049 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 215.12644581399999 2024-09-10T22:03:36.049 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 -- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f107c0b5330 msgr2=0x7f107c0b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:36.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f107c0b5330 0x7f107c0b77f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1090005ab0 tx=0x7f1090005a20 comp rx=0 tx=0).stop 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 -- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1098104f90 msgr2=0x7f109819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1098104f90 0x7f109819b330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f108800db80 tx=0x7f108800b630 comp rx=0 tx=0).stop 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 -- 172.21.15.90:0/234275590 shutdown_connections 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f107c0b5330 0x7f107c0b77f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f109810b320 0x7f10981a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f109810a9a0 0x7f109819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 --2- 172.21.15.90:0/234275590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1098104f90 0x7f109819b330 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 -- 172.21.15.90:0/234275590 >> 172.21.15.90:0/234275590 conn(0x7f1098100a00 msgr2=0x7f1098102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 -- 172.21.15.90:0/234275590 shutdown_connections 2024-09-10T22:03:36.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:36.042+0000 7f109e800640 1 -- 172.21.15.90:0/234275590 wait complete. 2024-09-10T22:03:36.940 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T22:03:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:36 smithi090 ceph-mon[30350]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:03:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:36 smithi154 ceph-mon[34495]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:03:37.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:36 smithi171 ceph-mon[34138]: pgmap v409: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:03:37.311 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:37.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.917+0000 7f0600200640 1 -- 172.21.15.90:0/2514387858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8102dc0 msgr2=0x7f05f81031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:37.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.917+0000 7f0600200640 1 --2- 172.21.15.90:0/2514387858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8102dc0 0x7f05f81031a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009900 tx=0x7f05ec02a410 comp rx=0 tx=0).stop 2024-09-10T22:03:37.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.918+0000 7f0600200640 1 -- 172.21.15.90:0/2514387858 shutdown_connections 2024-09-10T22:03:37.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.918+0000 7f0600200640 1 --2- 172.21.15.90:0/2514387858 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f8109150 0x7f05f810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.918+0000 7f0600200640 1 --2- 172.21.15.90:0/2514387858 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05f81087d0 0x7f05f8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.918+0000 7f0600200640 1 --2- 172.21.15.90:0/2514387858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8102dc0 0x7f05f81031a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.918+0000 7f0600200640 1 -- 172.21.15.90:0/2514387858 >> 172.21.15.90:0/2514387858 conn(0x7f05f80fe830 msgr2=0x7f05f8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:37.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.918+0000 7f0600200640 1 -- 172.21.15.90:0/2514387858 shutdown_connections 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.918+0000 7f0600200640 1 -- 172.21.15.90:0/2514387858 wait complete. 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.919+0000 7f0600200640 1 Processor -- start 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.919+0000 7f0600200640 1 -- start start 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f0600200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05f8102dc0 0x7f05f81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f0600200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f81087d0 0x7f05f8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f0600200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8109150 0x7f05f819eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f0600200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8112d10 con 0x7f05f8109150 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f0600200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8112b90 con 0x7f05f81087d0 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f0600200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8112e90 con 0x7f05f8102dc0 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f05fe200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8109150 0x7f05f819eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f05fe200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8109150 0x7f05f819eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51462/0 (socket says 172.21.15.90:51462) 2024-09-10T22:03:37.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f05fe200640 1 -- 172.21.15.90:0/3650474401 learned_addr learned my addr 172.21.15.90:0/3650474401 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f05fd800640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05f8102dc0 0x7f05f81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f05fce00640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f81087d0 0x7f05f8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f05fe200640 1 -- 172.21.15.90:0/3650474401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05f8102dc0 msgr2=0x7f05f81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.920+0000 7f05fe200640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05f8102dc0 0x7f05f81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.921+0000 7f05fe200640 1 -- 172.21.15.90:0/3650474401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f81087d0 msgr2=0x7f05f8199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.921+0000 7f05fe200640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f81087d0 0x7f05f8199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.921+0000 7f05fe200640 1 -- 172.21.15.90:0/3650474401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f819f290 con 0x7f05f8109150 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.921+0000 7f05fe200640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8109150 0x7f05f819eac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f05f000eb10 tx=0x7f05f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.921+0000 7f05f6000640 1 -- 172.21.15.90:0/3650474401 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0004b20 con 0x7f05f8109150 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.921+0000 7f05f6000640 1 -- 172.21.15.90:0/3650474401 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05f0004cc0 con 0x7f05f8109150 2024-09-10T22:03:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.921+0000 7f0600200640 1 -- 172.21.15.90:0/3650474401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f819f580 con 0x7f05f8109150 2024-09-10T22:03:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.921+0000 7f05f6000640 1 -- 172.21.15.90:0/3650474401 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0010690 con 0x7f05f8109150 2024-09-10T22:03:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.921+0000 7f0600200640 1 -- 172.21.15.90:0/3650474401 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f81a5c40 con 0x7f05f8109150 2024-09-10T22:03:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.925+0000 7f0602bcc740 1 Processor -- start 2024-09-10T22:03:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f05f6000640 1 -- 172.21.15.90:0/3650474401 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f05f0005020 con 0x7f05f8109150 2024-09-10T22:03:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f0602bcc740 1 -- start start 2024-09-10T22:03:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418085620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085b60 0x5654180959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 0x5654180982f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f0602bcc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x565417ffcd50 con 0x565418085240 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f0602bcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565417fec180 con 0x565418095f00 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f0602bcc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565417feff90 con 0x565418085b60 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f05fce00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418085620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f05fce00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418085620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51466/0 (socket says 172.21.15.90:51466) 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.926+0000 7f05fce00640 1 -- 172.21.15.90:0/3493520581 learned_addr learned my addr 172.21.15.90:0/3493520581 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05fce00640 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 0x5654180982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05fd800640 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085b60 0x5654180959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05fce00640 1 -- 172.21.15.90:0/3493520581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085b60 msgr2=0x5654180959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05fce00640 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085b60 0x5654180959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05fce00640 1 -- 172.21.15.90:0/3493520581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 msgr2=0x5654180982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05fce00640 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 0x5654180982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05fce00640 1 -- 172.21.15.90:0/3493520581 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565417fb6a70 con 0x565418085240 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05fd800640 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085b60 0x5654180959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05fce00640 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418085620 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f05e4013650 tx=0x7f05e4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7e93bae030cd9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05eaa00640 1 -- 172.21.15.90:0/3493520581 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4004050 con 0x565418085240 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05eaa00640 1 -- 172.21.15.90:0/3493520581 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05e4005020 con 0x565418085240 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.927+0000 7f05eaa00640 1 -- 172.21.15.90:0/3493520581 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4004a00 con 0x565418085240 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f05f6000640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f05e00b53b0 0x7f05e00b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f0602bcc740 1 -- 172.21.15.90:0/3493520581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 msgr2=0x565418085620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f0602bcc740 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418085620 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f05e4013650 tx=0x7f05e4009830 comp rx=0 tx=0).stop 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f05f6000640 1 -- 172.21.15.90:0/3650474401 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f05f00e6660 con 0x7f05f8109150 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f05fd800640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f05e00b53b0 0x7f05e00b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f0602bcc740 1 -- 172.21.15.90:0/3493520581 shutdown_connections 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f0602bcc740 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 0x5654180982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f0602bcc740 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085b60 0x5654180959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f0602bcc740 1 --2- 172.21.15.90:0/3493520581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418085620 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f0602bcc740 1 -- 172.21.15.90:0/3493520581 >> 172.21.15.90:0/3493520581 conn(0x565418083020 msgr2=0x565418084210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.928+0000 7f0602bcc740 1 -- 172.21.15.90:0/3493520581 shutdown_connections 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.929+0000 7f0602bcc740 1 -- 172.21.15.90:0/3493520581 wait complete. 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.929+0000 7f0602bcc740 1 -- start start 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.929+0000 7f05fd800640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f05e00b53b0 0x7f05e00b7870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f05ec006010 tx=0x7f05ec005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.929+0000 7f0602bcc740 1 Processor -- start 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418092f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418093440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 0x565418093980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f0602bcc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x565417e4d5f0 con 0x565418085240 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f0602bcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x565417f87460 con 0x565418085b60 2024-09-10T22:03:37.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f0602bcc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x565418098c50 con 0x565418095f00 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f0602bcc740 10 client.0 _ll_register_callbacks cb 0x565417f97210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fd800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418092f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fd800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418092f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51474/0 (socket says 172.21.15.90:51474) 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fd800640 1 -- 172.21.15.90:0/2007185773 learned_addr learned my addr 172.21.15.90:0/2007185773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fce00640 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418093440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fe200640 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 0x565418093980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fd800640 1 -- 172.21.15.90:0/2007185773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 msgr2=0x565418093980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fd800640 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 0x565418093980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fd800640 1 -- 172.21.15.90:0/2007185773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 msgr2=0x565418093440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fd800640 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418093440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fd800640 1 -- 172.21.15.90:0/2007185773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565417d4a540 con 0x565418085240 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.930+0000 7f05fce00640 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418093440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.931+0000 7f05fd800640 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418092f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f05ec033000 tx=0x7f05ec02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37b19a83fb91771 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.931+0000 7f05df400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.931+0000 7f05df400640 1 -- 172.21.15.90:0/2007185773 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec04d020 con 0x565418085240 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.931+0000 7f05df400640 1 -- 172.21.15.90:0/2007185773 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f05ec003c00 con 0x565418085240 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.931+0000 7f0602bcc740 1 -- 172.21.15.90:0/2007185773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565417f89a70 con 0x565418085240 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.931+0000 7f05df400640 1 -- 172.21.15.90:0/2007185773 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05ec058050 con 0x565418085240 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.931+0000 7f0602bcc740 10 client.15513 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.931+0000 7f0602bcc740 1 -- 172.21.15.90:0/2007185773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565417fa46f0 con 0x565418085240 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.931+0000 7f05df400640 1 -- 172.21.15.90:0/2007185773 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x565417f89a70 con 0x565418085240 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.932+0000 7f05df400640 1 -- 172.21.15.90:0/2007185773 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f05ec03c440 con 0x565418085240 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.932+0000 7f0602bcc740 10 client.15513 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.932+0000 7f0602bcc740 10 client.15513 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.932+0000 7f0602bcc740 20 client.15513 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:37.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.932+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x5654180a6cc0 0x5654180a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.932+0000 7f0602bcc740 4 client.15513 mds_command: new command op to 34670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.932+0000 7f0602bcc740 1 -- 172.21.15.90:0/2007185773 --> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565417ffeca0 con 0x5654180a6cc0 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.932+0000 7f05fce00640 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x5654180a6cc0 0x5654180a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.933+0000 7f05fce00640 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x5654180a6cc0 0x5654180a8130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1d12de810f8545c server_cookie=8ce5cc1ad1e12ce0 in_seq=0 out_seq=0 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.933+0000 7f05df400640 10 client.15513 ms_handle_connect on v2:172.21.15.154:6836/3907253955 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.934+0000 7f05df400640 1 -- 172.21.15.90:0/2007185773 <== mds.1 v2:172.21.15.154:6836/3907253955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565417ffeca0 con 0x5654180a6cc0 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.934+0000 7f05df400640 10 client.15513 handle_command_reply: tid=0 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.934+0000 7f0602bcc740 1 client.15513 shutdown 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 -- 172.21.15.90:0/2007185773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 msgr2=0x565418092f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418092f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f05ec033000 tx=0x7f05ec02a410 comp rx=0 tx=0).stop 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 -- 172.21.15.90:0/2007185773 shutdown_connections 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x5654180a6cc0 0x5654180a8130 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 0x565418093980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418093440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2007185773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418092f00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f05df400640 0 client.15513 ms_handle_reset on v2:172.21.15.154:6836/3907253955 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 -- 172.21.15.90:0/2007185773 >> 172.21.15.90:0/2007185773 conn(0x565418083020 msgr2=0x565418082140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 -- 172.21.15.90:0/2007185773 shutdown_connections 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 1 -- 172.21.15.90:0/2007185773 wait complete. 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.935+0000 7f0602bcc740 20 client.15513 trim_cache size 0 max 16384 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f0602bcc740 1 Processor -- start 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f0602bcc740 1 -- start start 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418096ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418097530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 0x5654180917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f0602bcc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5654180a6470 con 0x565418085240 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f0602bcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5654180a6b40 con 0x565418085b60 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f0602bcc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5654180a7210 con 0x565418095f00 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f05fd800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418096ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f05fd800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418096ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51484/0 (socket says 172.21.15.90:51484) 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f05fd800640 1 -- 172.21.15.90:0/385086549 learned_addr learned my addr 172.21.15.90:0/385086549 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.954+0000 7f05fce00640 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418097530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05fe200640 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 0x5654180917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05fd800640 1 -- 172.21.15.90:0/385086549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 msgr2=0x5654180917a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05fd800640 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 0x5654180917a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05fd800640 1 -- 172.21.15.90:0/385086549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 msgr2=0x565418097530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05fd800640 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418097530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05fd800640 1 -- 172.21.15.90:0/385086549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565418091420 con 0x565418085240 2024-09-10T22:03:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05fce00640 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418097530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05fd800640 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418096ff0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0057c0 tx=0x7f05ec0042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e1c0e618f82f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05ea000640 1 -- 172.21.15.90:0/385086549 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec04d070 con 0x565418085240 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05ea000640 1 -- 172.21.15.90:0/385086549 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05ec003e90 con 0x565418085240 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.955+0000 7f05ea000640 1 -- 172.21.15.90:0/385086549 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec035040 con 0x565418085240 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 -- 172.21.15.90:0/385086549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 msgr2=0x565418096ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418096ff0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0057c0 tx=0x7f05ec0042c0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 -- 172.21.15.90:0/385086549 shutdown_connections 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418095f00 0x5654180917a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418085b60 0x565418097530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 --2- 172.21.15.90:0/385086549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085240 0x565418096ff0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 -- 172.21.15.90:0/385086549 >> 172.21.15.90:0/385086549 conn(0x565418083020 msgr2=0x5654180a5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 -- 172.21.15.90:0/385086549 shutdown_connections 2024-09-10T22:03:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 -- 172.21.15.90:0/385086549 wait complete. 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.956+0000 7f0602bcc740 1 -- start start 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f0602bcc740 1 Processor -- start 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085240 0x5654180908e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085b60 0x5654180b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f0602bcc740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 0x5654180b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f0602bcc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5654180a7fb0 con 0x565418085b60 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f0602bcc740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5654180a78e0 con 0x565418095f00 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f0602bcc740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5654180a8680 con 0x565418085240 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f0602bcc740 10 client.0 _ll_register_callbacks cb 0x7f05ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f05fce00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085b60 0x5654180b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f05fce00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085b60 0x5654180b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51496/0 (socket says 172.21.15.90:51496) 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f05fce00640 1 -- 172.21.15.90:0/2902633269 learned_addr learned my addr 172.21.15.90:0/2902633269 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.957+0000 7f05fd800640 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085240 0x5654180908e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05fe200640 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 0x5654180b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05fce00640 1 -- 172.21.15.90:0/2902633269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085240 msgr2=0x5654180908e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05fce00640 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085240 0x5654180908e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05fce00640 1 -- 172.21.15.90:0/2902633269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 msgr2=0x5654180b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05fce00640 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 0x5654180b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05fce00640 1 -- 172.21.15.90:0/2902633269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654180b10f0 con 0x565418085b60 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05fe200640 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 0x5654180b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05fce00640 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085b60 0x5654180b0380 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f05e4013450 tx=0x7f05e4005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b685d643bf578412 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05df400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05df400640 1 -- 172.21.15.90:0/2902633269 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e402d030 con 0x565418085b60 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f0602bcc740 1 -- 172.21.15.90:0/2902633269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05ec003e90 con 0x565418085b60 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.958+0000 7f05df400640 1 -- 172.21.15.90:0/2902633269 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f05e4005430 con 0x565418085b60 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f05df400640 1 -- 172.21.15.90:0/2902633269 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05e4037050 con 0x565418085b60 2024-09-10T22:03:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f0602bcc740 10 client.15519 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f0602bcc740 1 -- 172.21.15.90:0/2902633269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565418117df0 con 0x565418085b60 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f05df400640 1 -- 172.21.15.90:0/2902633269 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f05ec003e90 con 0x565418085b60 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f05df400640 1 -- 172.21.15.90:0/2902633269 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f05e401d020 con 0x565418085b60 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f0602bcc740 10 client.15519 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f0602bcc740 10 client.15519 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f0602bcc740 20 client.15519 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56541811c730 0x56541811b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f0602bcc740 4 client.15519 mds_command: new command op to 34670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f0602bcc740 1 -- 172.21.15.90:0/2902633269 --> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] -- command(tid 0: {"prefix": "status"}) -- 0x565417ffa750 con 0x56541811c730 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.959+0000 7f05fd800640 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56541811c730 0x56541811b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f05fd800640 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56541811c730 0x56541811b440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abd9af5311a6e475 server_cookie=58328bbcdc1f7cb3 in_seq=0 out_seq=0 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f05df400640 10 client.15519 ms_handle_connect on v2:172.21.15.154:6836/3907253955 2024-09-10T22:03:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f05df400640 1 -- 172.21.15.90:0/2902633269 <== mds.1 v2:172.21.15.154:6836/3907253955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x565417ffa750 con 0x56541811c730 2024-09-10T22:03:37.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f05df400640 10 client.15519 handle_command_reply: tid=0 2024-09-10T22:03:37.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f0602bcc740 1 client.15519 shutdown 2024-09-10T22:03:37.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f0602bcc740 1 -- 172.21.15.90:0/2902633269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085b60 msgr2=0x5654180b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:37.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085b60 0x5654180b0380 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f05e4013450 tx=0x7f05e4005b40 comp rx=0 tx=0).stop 2024-09-10T22:03:37.985 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:37.985 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:03:37.985 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34670, 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 129.26197149699999, 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 210.37071809299999 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:37.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f0602bcc740 1 -- 172.21.15.90:0/2902633269 shutdown_connections 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x56541811c730 0x56541811b440 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x565418095f00 0x5654180b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x565418085b60 0x5654180b0380 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f05df400640 0 client.15519 ms_handle_reset on v2:172.21.15.154:6836/3907253955 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f0602bcc740 1 --2- 172.21.15.90:0/2902633269 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x565418085240 0x5654180908e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.960+0000 7f0602bcc740 1 -- 172.21.15.90:0/2902633269 >> 172.21.15.90:0/2902633269 conn(0x565418083020 msgr2=0x565418096ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0602bcc740 1 -- 172.21.15.90:0/2902633269 shutdown_connections 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0602bcc740 1 -- 172.21.15.90:0/2902633269 wait complete. 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0602bcc740 20 client.15519 trim_cache size 0 max 16384 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0600200640 1 -- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f05e00b53b0 msgr2=0x7f05e00b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0600200640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f05e00b53b0 0x7f05e00b7870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f05ec006010 tx=0x7f05ec005e80 comp rx=0 tx=0).stop 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0600200640 1 -- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8109150 msgr2=0x7f05f819eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:37.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0600200640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8109150 0x7f05f819eac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f05f000eb10 tx=0x7f05f000c2d0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0600200640 1 -- 172.21.15.90:0/3650474401 shutdown_connections 2024-09-10T22:03:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0600200640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f05e00b53b0 0x7f05e00b7870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0600200640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05f8109150 0x7f05f819eac0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0600200640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f05f81087d0 0x7f05f8199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.961+0000 7f0600200640 1 --2- 172.21.15.90:0/3650474401 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f05f8102dc0 0x7f05f81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.962+0000 7f0600200640 1 -- 172.21.15.90:0/3650474401 >> 172.21.15.90:0/3650474401 conn(0x7f05f80fe830 msgr2=0x7f05f8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.962+0000 7f0600200640 1 -- 172.21.15.90:0/3650474401 shutdown_connections 2024-09-10T22:03:37.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:37.962+0000 7f0600200640 1 -- 172.21.15.90:0/3650474401 wait complete. 2024-09-10T22:03:38.821 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:03:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:38 smithi090 ceph-mon[30350]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:38 smithi154 ceph-mon[34495]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:39.191 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:39.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:38 smithi171 ceph-mon[34138]: pgmap v410: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:39.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.849+0000 7f9280c00640 1 -- 172.21.15.90:0/4007009961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c0754e0 msgr2=0x7f927c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:39.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.849+0000 7f9280c00640 1 --2- 172.21.15.90:0/4007009961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c0754e0 0x7f927c0758c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f926c009900 tx=0x7f926c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.849+0000 7f9280c00640 1 -- 172.21.15.90:0/4007009961 shutdown_connections 2024-09-10T22:03:39.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.849+0000 7f9280c00640 1 --2- 172.21.15.90:0/4007009961 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f927c10f3d0 0x7f927c1117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.849+0000 7f9280c00640 1 --2- 172.21.15.90:0/4007009961 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f927c075e00 0x7f927c10ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.849+0000 7f9280c00640 1 --2- 172.21.15.90:0/4007009961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c0754e0 0x7f927c0758c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.849+0000 7f9280c00640 1 -- 172.21.15.90:0/4007009961 >> 172.21.15.90:0/4007009961 conn(0x7f927c0fe830 msgr2=0x7f927c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:39.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.849+0000 7f9280c00640 1 -- 172.21.15.90:0/4007009961 shutdown_connections 2024-09-10T22:03:39.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.849+0000 7f9280c00640 1 -- 172.21.15.90:0/4007009961 wait complete. 2024-09-10T22:03:39.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.850+0000 7f9280c00640 1 Processor -- start 2024-09-10T22:03:39.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.850+0000 7f9280c00640 1 -- start start 2024-09-10T22:03:39.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f9280c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f927c0754e0 0x7f927c19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f9280c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f927c075e00 0x7f927c19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f9280c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c10f3d0 0x7f927c1a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f9280c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f927c114f40 con 0x7f927c10f3d0 2024-09-10T22:03:39.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f9280c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f927c114dc0 con 0x7f927c075e00 2024-09-10T22:03:39.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f9280c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1150c0 con 0x7f927c0754e0 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f927a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c10f3d0 0x7f927c1a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f927a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c10f3d0 0x7f927c1a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51516/0 (socket says 172.21.15.90:51516) 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f927a800640 1 -- 172.21.15.90:0/1283532161 learned_addr learned my addr 172.21.15.90:0/1283532161 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f9279400640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f927c075e00 0x7f927c19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.851+0000 7f9279e00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f927c0754e0 0x7f927c19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f927a800640 1 -- 172.21.15.90:0/1283532161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f927c0754e0 msgr2=0x7f927c19b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f927a800640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f927c0754e0 0x7f927c19b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f927a800640 1 -- 172.21.15.90:0/1283532161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f927c075e00 msgr2=0x7f927c19ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f927a800640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f927c075e00 0x7f927c19ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f927a800640 1 -- 172.21.15.90:0/1283532161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c1a1600 con 0x7f927c10f3d0 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f9279400640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f927c075e00 0x7f927c19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:39.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f9279e00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f927c0754e0 0x7f927c19b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:39.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f927a800640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c10f3d0 0x7f927c1a0e30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f927400b9e0 tx=0x7f927400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:39.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f9272a00640 1 -- 172.21.15.90:0/1283532161 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274019070 con 0x7f927c10f3d0 2024-09-10T22:03:39.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f9272a00640 1 -- 172.21.15.90:0/1283532161 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9274014440 con 0x7f927c10f3d0 2024-09-10T22:03:39.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f9272a00640 1 -- 172.21.15.90:0/1283532161 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740125a0 con 0x7f927c10f3d0 2024-09-10T22:03:39.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f9280c00640 1 -- 172.21.15.90:0/1283532161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c1a18f0 con 0x7f927c10f3d0 2024-09-10T22:03:39.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.852+0000 7f9280c00640 1 -- 172.21.15.90:0/1283532161 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c1a8030 con 0x7f927c10f3d0 2024-09-10T22:03:39.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.856+0000 7f92836c7740 1 Processor -- start 2024-09-10T22:03:39.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.857+0000 7f9272a00640 1 -- 172.21.15.90:0/1283532161 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f927400cb20 con 0x7f927c10f3d0 2024-09-10T22:03:39.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.857+0000 7f92836c7740 1 -- start start 2024-09-10T22:03:39.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.857+0000 7f92836c7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7f9c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.857+0000 7f92836c7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 0x562ef7fa85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.857+0000 7f92836c7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7faaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.857+0000 7f92836c7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ef7f13d50 con 0x562ef7f9c240 2024-09-10T22:03:39.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.857+0000 7f92836c7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562ef7f03180 con 0x562ef7fa8af0 2024-09-10T22:03:39.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.857+0000 7f92836c7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562ef7f06f90 con 0x562ef7f9cb60 2024-09-10T22:03:39.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7f9c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7f9c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51524/0 (socket says 172.21.15.90:51524) 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 -- 172.21.15.90:0/4102728525 learned_addr learned my addr 172.21.15.90:0/4102728525 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 --2- 172.21.15.90:0/4102728525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7faaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279e00640 1 --2- 172.21.15.90:0/4102728525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 0x562ef7fa85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 -- 172.21.15.90:0/4102728525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 msgr2=0x562ef7fa85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 --2- 172.21.15.90:0/4102728525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 0x562ef7fa85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 -- 172.21.15.90:0/4102728525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 msgr2=0x562ef7faaee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 --2- 172.21.15.90:0/4102728525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7faaee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 -- 172.21.15.90:0/4102728525 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef7ecda70 con 0x562ef7f9c240 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.858+0000 7f9279400640 1 --2- 172.21.15.90:0/4102728525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7f9c620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9268007f60 tx=0x7f9268007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32b4fb0d45015ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.859+0000 7f9272a00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f92600b5330 0x7f92600b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.859+0000 7f9267000640 1 -- 172.21.15.90:0/4102728525 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926801c070 con 0x562ef7f9c240 2024-09-10T22:03:39.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.859+0000 7f9267000640 1 -- 172.21.15.90:0/4102728525 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9268010040 con 0x562ef7f9c240 2024-09-10T22:03:39.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.859+0000 7f9267000640 1 -- 172.21.15.90:0/4102728525 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268014610 con 0x562ef7f9c240 2024-09-10T22:03:39.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.859+0000 7f9272a00640 1 -- 172.21.15.90:0/1283532161 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f92740e5960 con 0x7f927c10f3d0 2024-09-10T22:03:39.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.859+0000 7f9279e00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f92600b5330 0x7f92600b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 -- 172.21.15.90:0/4102728525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 msgr2=0x562ef7f9c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:39.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 --2- 172.21.15.90:0/4102728525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7f9c620 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9268007f60 tx=0x7f9268007420 comp rx=0 tx=0).stop 2024-09-10T22:03:39.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f9279e00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f92600b5330 0x7f92600b77f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f926c000c00 tx=0x7f926c0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:39.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 -- 172.21.15.90:0/4102728525 shutdown_connections 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 --2- 172.21.15.90:0/4102728525 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7faaee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 --2- 172.21.15.90:0/4102728525 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 0x562ef7fa85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 --2- 172.21.15.90:0/4102728525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7f9c620 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 -- 172.21.15.90:0/4102728525 >> 172.21.15.90:0/4102728525 conn(0x562ef7f9a020 msgr2=0x562ef7f9b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 -- 172.21.15.90:0/4102728525 shutdown_connections 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 -- 172.21.15.90:0/4102728525 wait complete. 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.860+0000 7f92836c7740 1 -- start start 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f92836c7740 1 Processor -- start 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f92836c7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fa5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f92836c7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 0x562ef7fa5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f92836c7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7fa6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f92836c7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ef7d645f0 con 0x562ef7f9c240 2024-09-10T22:03:39.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f92836c7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562ef7e9e460 con 0x562ef7fa8af0 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f92836c7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562ef7fab840 con 0x562ef7f9cb60 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f92836c7740 10 client.0 _ll_register_callbacks cb 0x562ef7eae210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f9279e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fa5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f9279e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fa5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51530/0 (socket says 172.21.15.90:51530) 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.861+0000 7f9279e00640 1 -- 172.21.15.90:0/1550786378 learned_addr learned my addr 172.21.15.90:0/1550786378 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.862+0000 7f9279400640 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 0x562ef7fa5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.862+0000 7f927a800640 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7fa6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.862+0000 7f9279e00640 1 -- 172.21.15.90:0/1550786378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 msgr2=0x562ef7fa5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.862+0000 7f9279e00640 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 0x562ef7fa5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.862+0000 7f9279e00640 1 -- 172.21.15.90:0/1550786378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 msgr2=0x562ef7fa6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.862+0000 7f9279e00640 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7fa6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.862+0000 7f9279e00640 1 -- 172.21.15.90:0/1550786378 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef7c61540 con 0x562ef7f9c240 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.862+0000 7f9279e00640 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fa5ab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f926c032fd0 tx=0x7f926c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f70fb3d673b875e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.862+0000 7f925b400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f925b400640 1 -- 172.21.15.90:0/1550786378 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c04c030 con 0x562ef7f9c240 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f92836c7740 1 -- 172.21.15.90:0/1550786378 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ef7ea0a70 con 0x562ef7f9c240 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f925b400640 1 -- 172.21.15.90:0/1550786378 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f926c003800 con 0x562ef7f9c240 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f925b400640 1 -- 172.21.15.90:0/1550786378 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f926c059050 con 0x562ef7f9c240 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f92836c7740 10 client.15531 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f92836c7740 1 -- 172.21.15.90:0/1550786378 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ef7ebb6f0 con 0x562ef7f9c240 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f925b400640 1 -- 172.21.15.90:0/1550786378 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562ef7ea0a70 con 0x562ef7f9c240 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f925b400640 1 -- 172.21.15.90:0/1550786378 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f926c048a70 con 0x562ef7f9c240 2024-09-10T22:03:39.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f92836c7740 10 client.15531 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:39.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f92836c7740 10 client.15531 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:03:39.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.863+0000 7f92836c7740 20 client.15531 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:39.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.864+0000 7f92836c7740 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562ef7fb9970 0x562ef7fbade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.864+0000 7f92836c7740 4 client.15531 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:03:39.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.864+0000 7f92836c7740 1 -- 172.21.15.90:0/1550786378 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ef7f15ca0 con 0x562ef7fb9970 2024-09-10T22:03:39.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.864+0000 7f9279400640 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562ef7fb9970 0x562ef7fbade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.865+0000 7f9279400640 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562ef7fb9970 0x562ef7fbade0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93b8e5ebd11c9d4a server_cookie=a2e390c188ded90 in_seq=0 out_seq=0 2024-09-10T22:03:39.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.865+0000 7f925b400640 10 client.15531 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:03:39.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f925b400640 1 -- 172.21.15.90:0/1550786378 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ef7f15ca0 con 0x562ef7fb9970 2024-09-10T22:03:39.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f925b400640 10 client.15531 handle_command_reply: tid=0 2024-09-10T22:03:39.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 client.15531 shutdown 2024-09-10T22:03:39.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 -- 172.21.15.90:0/1550786378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 msgr2=0x562ef7fa5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:39.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fa5ab0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f926c032fd0 tx=0x7f926c009260 comp rx=0 tx=0).stop 2024-09-10T22:03:39.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 -- 172.21.15.90:0/1550786378 shutdown_connections 2024-09-10T22:03:39.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562ef7fb9970 0x562ef7fbade0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7fa6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9cb60 0x562ef7fa5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f925b400640 0 client.15531 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:03:39.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 --2- 172.21.15.90:0/1550786378 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fa5ab0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 -- 172.21.15.90:0/1550786378 >> 172.21.15.90:0/1550786378 conn(0x562ef7f9a020 msgr2=0x562ef7f9fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:39.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 -- 172.21.15.90:0/1550786378 shutdown_connections 2024-09-10T22:03:39.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 1 -- 172.21.15.90:0/1550786378 wait complete. 2024-09-10T22:03:39.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.866+0000 7f92836c7740 20 client.15531 trim_cache size 0 max 16384 2024-09-10T22:03:39.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.885+0000 7f92836c7740 1 Processor -- start 2024-09-10T22:03:39.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.885+0000 7f92836c7740 1 -- start start 2024-09-10T22:03:39.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.885+0000 7f92836c7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9c240 0x562ef7fa0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.885+0000 7f92836c7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9cb60 0x562ef7fa0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.885+0000 7f92836c7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7fa1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.885+0000 7f92836c7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ef7fb9120 con 0x562ef7f9cb60 2024-09-10T22:03:39.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.885+0000 7f92836c7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562ef7fb97f0 con 0x562ef7fa8af0 2024-09-10T22:03:39.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.885+0000 7f92836c7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562ef7fb9ec0 con 0x562ef7f9c240 2024-09-10T22:03:39.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9cb60 0x562ef7fa0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9cb60 0x562ef7fa0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51544/0 (socket says 172.21.15.90:51544) 2024-09-10T22:03:39.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279400640 1 -- 172.21.15.90:0/1631331258 learned_addr learned my addr 172.21.15.90:0/1631331258 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:39.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279e00640 1 --2- 172.21.15.90:0/1631331258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9c240 0x562ef7fa0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279400640 1 -- 172.21.15.90:0/1631331258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9c240 msgr2=0x562ef7fa0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:39.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279400640 1 --2- 172.21.15.90:0/1631331258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9c240 0x562ef7fa0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279400640 1 -- 172.21.15.90:0/1631331258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 msgr2=0x562ef7fa1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:39.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279400640 1 --2- 172.21.15.90:0/1631331258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7fa1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279400640 1 -- 172.21.15.90:0/1631331258 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef7fab840 con 0x562ef7f9cb60 2024-09-10T22:03:39.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9279400640 1 --2- 172.21.15.90:0/1631331258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9cb60 0x562ef7fa0d30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9268007f30 tx=0x7f9268007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0f9def865698b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:39.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.886+0000 7f9266600640 1 -- 172.21.15.90:0/1631331258 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926801c040 con 0x562ef7f9cb60 2024-09-10T22:03:39.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f9266600640 1 -- 172.21.15.90:0/1631331258 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f926801d040 con 0x562ef7f9cb60 2024-09-10T22:03:39.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f9266600640 1 -- 172.21.15.90:0/1631331258 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9268006050 con 0x562ef7f9cb60 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f92836c7740 1 -- 172.21.15.90:0/1631331258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9cb60 msgr2=0x562ef7fa0d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f92836c7740 1 --2- 172.21.15.90:0/1631331258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9cb60 0x562ef7fa0d30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9268007f30 tx=0x7f9268007420 comp rx=0 tx=0).stop 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f92836c7740 1 -- 172.21.15.90:0/1631331258 shutdown_connections 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f92836c7740 1 --2- 172.21.15.90:0/1631331258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7fa8af0 0x562ef7fa1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f92836c7740 1 --2- 172.21.15.90:0/1631331258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9cb60 0x562ef7fa0d30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f92836c7740 1 --2- 172.21.15.90:0/1631331258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7f9c240 0x562ef7fa0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f92836c7740 1 -- 172.21.15.90:0/1631331258 >> 172.21.15.90:0/1631331258 conn(0x562ef7f9a020 msgr2=0x562ef7f94ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f92836c7740 1 -- 172.21.15.90:0/1631331258 shutdown_connections 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.887+0000 7f92836c7740 1 -- 172.21.15.90:0/1631331258 wait complete. 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.888+0000 7f92836c7740 1 -- start start 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.888+0000 7f92836c7740 1 Processor -- start 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.888+0000 7f92836c7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fbd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.888+0000 7f92836c7740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7f9cb60 0x562ef7fbd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.888+0000 7f92836c7740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7fa8af0 0x562ef7fbdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.888+0000 7f92836c7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562ef7fbac60 con 0x562ef7f9c240 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.888+0000 7f92836c7740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x562ef7fba590 con 0x562ef7f9cb60 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.888+0000 7f92836c7740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562ef7fbb330 con 0x562ef7fa8af0 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f92836c7740 10 client.0 _ll_register_callbacks cb 0x7f926c0315e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f9279e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fbd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f9279e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fbd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51560/0 (socket says 172.21.15.90:51560) 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f9279e00640 1 -- 172.21.15.90:0/2211676799 learned_addr learned my addr 172.21.15.90:0/2211676799 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f9279400640 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7f9cb60 0x562ef7fbd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f927a800640 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7fa8af0 0x562ef7fbdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f9279e00640 1 -- 172.21.15.90:0/2211676799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7fa8af0 msgr2=0x562ef7fbdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f9279e00640 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7fa8af0 0x562ef7fbdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f9279e00640 1 -- 172.21.15.90:0/2211676799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7f9cb60 msgr2=0x562ef7fbd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f9279e00640 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7f9cb60 0x562ef7fbd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.889+0000 7f9279e00640 1 -- 172.21.15.90:0/2211676799 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef7fc2920 con 0x562ef7f9c240 2024-09-10T22:03:39.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.890+0000 7f9279e00640 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fbd090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f926c005ec0 tx=0x7f926c003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a8e588ae4af3be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:39.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.890+0000 7f925b400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.890+0000 7f925b400640 1 -- 172.21.15.90:0/2211676799 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c062060 con 0x562ef7f9c240 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.890+0000 7f92836c7740 1 -- 172.21.15.90:0/2211676799 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926801d040 con 0x562ef7f9c240 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.890+0000 7f925b400640 1 -- 172.21.15.90:0/2211676799 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f926c048440 con 0x562ef7f9c240 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.890+0000 7f925b400640 1 -- 172.21.15.90:0/2211676799 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f926c069050 con 0x562ef7f9c240 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.890+0000 7f92836c7740 10 client.15537 fetch_fsmap learned FSMap version 29 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.890+0000 7f92836c7740 1 -- 172.21.15.90:0/2211676799 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ef80266a0 con 0x562ef7f9c240 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.890+0000 7f925b400640 1 -- 172.21.15.90:0/2211676799 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f926801d040 con 0x562ef7f9c240 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.891+0000 7f925b400640 1 -- 172.21.15.90:0/2211676799 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f926c03c520 con 0x562ef7f9c240 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.891+0000 7f92836c7740 10 client.15537 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.891+0000 7f92836c7740 10 client.15537 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.891+0000 7f92836c7740 20 client.15537 populate_metadata read hostname 'smithi090' 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.891+0000 7f92836c7740 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562ef802b250 0x562ef8029f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.891+0000 7f92836c7740 4 client.15537 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.891+0000 7f92836c7740 1 -- 172.21.15.90:0/2211676799 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "status"}) -- 0x562ef7f11750 con 0x562ef802b250 2024-09-10T22:03:39.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.891+0000 7f9279400640 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562ef802b250 0x562ef8029f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.892+0000 7f9279400640 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562ef802b250 0x562ef8029f60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf37582db8222ab1 server_cookie=2f2388354ef9739e in_seq=0 out_seq=0 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.892+0000 7f925b400640 10 client.15537 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.892+0000 7f925b400640 1 -- 172.21.15.90:0/2211676799 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562ef7f11750 con 0x562ef802b250 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.892+0000 7f925b400640 10 client.15537 handle_command_reply: tid=0 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.892+0000 7f92836c7740 1 client.15537 shutdown 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 -- 172.21.15.90:0/2211676799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 msgr2=0x562ef7fbd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fbd090 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f926c005ec0 tx=0x7f926c003ab0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 -- 172.21.15.90:0/2211676799 shutdown_connections 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562ef802b250 0x562ef8029f60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562ef7fa8af0 0x562ef7fbdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562ef7f9cb60 0x562ef7fbd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 --2- 172.21.15.90:0/2211676799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562ef7f9c240 0x562ef7fbd090 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f925b400640 0 client.15537 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 -- 172.21.15.90:0/2211676799 >> 172.21.15.90:0/2211676799 conn(0x562ef7f9a020 msgr2=0x562ef7fa0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:39.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 -- 172.21.15.90:0/2211676799 shutdown_connections 2024-09-10T22:03:39.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 1 -- 172.21.15.90:0/2211676799 wait complete. 2024-09-10T22:03:39.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.893+0000 7f92836c7740 20 client.15537 trim_cache size 0 max 16384 2024-09-10T22:03:39.897 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:39.897 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:03:39.897 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-09-10T22:03:39.897 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34703, 2024-09-10T22:03:39.897 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:03:39.897 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:03:39.898 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:03:39.898 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 183.39863967400001, 2024-09-10T22:03:39.898 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 29, 2024-09-10T22:03:39.898 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:03:39.898 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 87, 2024-09-10T22:03:39.898 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 196.41047544700001 2024-09-10T22:03:39.898 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:39.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 -- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f92600b5330 msgr2=0x7f92600b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f92600b5330 0x7f92600b77f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f926c000c00 tx=0x7f926c0058d0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 -- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c10f3d0 msgr2=0x7f927c1a0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c10f3d0 0x7f927c1a0e30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f927400b9e0 tx=0x7f927400bea0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 -- 172.21.15.90:0/1283532161 shutdown_connections 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f92600b5330 0x7f92600b77f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f927c10f3d0 0x7f927c1a0e30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f927c075e00 0x7f927c19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 --2- 172.21.15.90:0/1283532161 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f927c0754e0 0x7f927c19b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 -- 172.21.15.90:0/1283532161 >> 172.21.15.90:0/1283532161 conn(0x7f927c0fe830 msgr2=0x7f927c0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 -- 172.21.15.90:0/1283532161 shutdown_connections 2024-09-10T22:03:39.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:39.894+0000 7f9280c00640 1 -- 172.21.15.90:0/1283532161 wait complete. 2024-09-10T22:03:40.649 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T22:03:40.662 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T22:03:40.663 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-10T22:03:40.663 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-09-10T22:03:40.663 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T22:03:40.663 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:40.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:40.663 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-09-10T22:03:40.699 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T22:03:40.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:40.700 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T22:03:40.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:40.799 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-09-10T22:03:40.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:40 smithi090 ceph-mon[30350]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:03:41.107 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T22:03:41.107 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T22:03:41.108 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T22:03:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:40 smithi154 ceph-mon[34495]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:03:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:40 smithi171 ceph-mon[34138]: pgmap v411: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:03:41.476 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:42.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.102+0000 7f360be00640 1 -- 172.21.15.90:0/4100857949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604113980 msgr2=0x7f3604115d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.102+0000 7f360be00640 1 --2- 172.21.15.90:0/4100857949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604113980 0x7f3604115d70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f35fc009ca0 tx=0x7f35fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.102+0000 7f360be00640 1 -- 172.21.15.90:0/4100857949 shutdown_connections 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.102+0000 7f360be00640 1 --2- 172.21.15.90:0/4100857949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604113980 0x7f3604115d70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.102+0000 7f360be00640 1 --2- 172.21.15.90:0/4100857949 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3604077ff0 0x7f3604113440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.102+0000 7f360be00640 1 --2- 172.21.15.90:0/4100857949 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36040776d0 0x7f3604077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.102+0000 7f360be00640 1 -- 172.21.15.90:0/4100857949 >> 172.21.15.90:0/4100857949 conn(0x7f3604100a00 msgr2=0x7f3604102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.102+0000 7f360be00640 1 -- 172.21.15.90:0/4100857949 shutdown_connections 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.102+0000 7f360be00640 1 -- 172.21.15.90:0/4100857949 wait complete. 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.103+0000 7f360be00640 1 Processor -- start 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.103+0000 7f360be00640 1 -- start start 2024-09-10T22:03:42.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f360be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36040776d0 0x7f360419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f360be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604077ff0 0x7f360419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f360be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3604113980 0x7f36041a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f360be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3604119260 con 0x7f3604077ff0 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f360be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f36041190e0 con 0x7f3604113980 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f360be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f36041193e0 con 0x7f36040776d0 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3608a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604077ff0 0x7f360419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3608a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604077ff0 0x7f360419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51572/0 (socket says 172.21.15.90:51572) 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3608a00640 1 -- 172.21.15.90:0/588743212 learned_addr learned my addr 172.21.15.90:0/588743212 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3609e00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3604113980 0x7f36041a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3609400640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36040776d0 0x7f360419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3608a00640 1 -- 172.21.15.90:0/588743212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36040776d0 msgr2=0x7f360419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:42.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3608a00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36040776d0 0x7f360419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3608a00640 1 -- 172.21.15.90:0/588743212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3604113980 msgr2=0x7f36041a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3608a00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3604113980 0x7f36041a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3608a00640 1 -- 172.21.15.90:0/588743212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36041a5800 con 0x7f3604077ff0 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.104+0000 7f3609e00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3604113980 0x7f36041a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.105+0000 7f3608a00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604077ff0 0x7f360419fc90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f35f000d950 tx=0x7f35f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.105+0000 7f3602000640 1 -- 172.21.15.90:0/588743212 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0017070 con 0x7f3604077ff0 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.105+0000 7f3602000640 1 -- 172.21.15.90:0/588743212 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35f0009c30 con 0x7f3604077ff0 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.105+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36041a5af0 con 0x7f3604077ff0 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.105+0000 7f3602000640 1 -- 172.21.15.90:0/588743212 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f000f620 con 0x7f3604077ff0 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.105+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36041ac380 con 0x7f3604077ff0 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.106+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3604078be0 con 0x7f3604077ff0 2024-09-10T22:03:42.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.109+0000 7f3602000640 1 -- 172.21.15.90:0/588743212 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f35f0010460 con 0x7f3604077ff0 2024-09-10T22:03:42.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.111+0000 7f3602000640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35ec0b53b0 0x7f35ec0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:42.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.111+0000 7f3602000640 1 -- 172.21.15.90:0/588743212 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f35f00e5d30 con 0x7f3604077ff0 2024-09-10T22:03:42.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.111+0000 7f3609400640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35ec0b53b0 0x7f35ec0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:42.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.112+0000 7f3609400640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35ec0b53b0 0x7f35ec0b7870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005fd0 tx=0x7f35f8005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:42.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.112+0000 7f3602000640 1 -- 172.21.15.90:0/588743212 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f35f00ac7e0 con 0x7f3604077ff0 2024-09-10T22:03:42.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.333+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3604077ab0 con 0x7f3604077ff0 2024-09-10T22:03:42.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.333+0000 7f3602000640 1 -- 172.21.15.90:0/588743212 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f3604077ab0 con 0x7f3604077ff0 2024-09-10T22:03:42.335 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T22:03:42.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.336+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35ec0b53b0 msgr2=0x7f35ec0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:42.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.336+0000 7f360be00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35ec0b53b0 0x7f35ec0b7870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f35f8005fd0 tx=0x7f35f8005850 comp rx=0 tx=0).stop 2024-09-10T22:03:42.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.336+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604077ff0 msgr2=0x7f360419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:42.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.336+0000 7f360be00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604077ff0 0x7f360419fc90 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f35f000d950 tx=0x7f35f000de10 comp rx=0 tx=0).stop 2024-09-10T22:03:42.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.337+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 shutdown_connections 2024-09-10T22:03:42.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.337+0000 7f360be00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35ec0b53b0 0x7f35ec0b7870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.337+0000 7f360be00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3604113980 0x7f36041a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.337+0000 7f360be00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3604077ff0 0x7f360419fc90 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.337+0000 7f360be00640 1 --2- 172.21.15.90:0/588743212 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f36040776d0 0x7f360419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:42.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.337+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 >> 172.21.15.90:0/588743212 conn(0x7f3604100a00 msgr2=0x7f3604102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:42.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.337+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 shutdown_connections 2024-09-10T22:03:42.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:42.337+0000 7f360be00640 1 -- 172.21.15.90:0/588743212 wait complete. 2024-09-10T22:03:43.093 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:42 smithi090 ceph-mon[30350]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:43.093 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:42 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/588743212' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:03:43.131 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T22:03:43.131 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T22:03:43.131 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-09-10T22:03:43.131 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T22:03:43.131 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:43.131 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T22:03:43.131 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T22:03:43.131 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T22:03:43.131 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:03:43.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:43.132 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-09-10T22:03:43.168 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T22:03:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T22:03:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T22:03:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:43.168 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T22:03:43.169 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 41886sec preferred_lft 41886sec 2024-09-10T22:03:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link noprefixroute 2024-09-10T22:03:43.170 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-09-10T22:03:43.170 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T22:03:43.170 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:03:43.170 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:03:43.170 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-09-10T22:03:43.170 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-09-10T22:03:43.170 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-09-10T22:03:43.170 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T22:03:43.170 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:03:43.171 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:42 smithi154 ceph-mon[34495]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:43.171 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:42 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/588743212' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:03:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:42 smithi171 ceph-mon[34138]: pgmap v412: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:03:43.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:42 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/588743212' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T22:03:43.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:03:43.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:03:43.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:43.395 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T22:03:43.450 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-09-10T22:03:43.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:43.454 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-09-10T22:03:43.490 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-09-10T22:03:43.490 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T22:03:43.490 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T22:03:43.490 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T22:03:43.490 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T22:03:43.490 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:03:43.490 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:03:43.490 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T22:03:43.491 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T22:03:43.491 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T22:03:43.491 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:03:43.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:03:43.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:03:43.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:43.688 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-09-10T22:03:43.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:43.725 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-09-10T22:03:43.800 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:03:43.800 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:03:43.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:43.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T22:03:43.800 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:03:43.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:03:43.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:03:43.978 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T22:03:43.978 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:03:43.978 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:03:43.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T22:03:43.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T22:03:43.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T22:03:43.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T22:03:43.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T22:03:43.978 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:03:44.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:03:44.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:03:44.197 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T22:03:44.197 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-09-10T22:03:44.197 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-09-10T22:03:44.198 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T22:03:44.198 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-09-10T22:03:44.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T22:03:44.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:03:44.363 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-09-10T22:03:44.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:03:44.363 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:44.398 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T22:03:44.398 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T22:03:44.398 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:44.473 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-09-10T22:03:44.570 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:03:44.606 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-09-10T22:03:44.606 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-09-10T22:03:44.606 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-09-10T22:03:44.606 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-09-10T22:03:44.607 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/aa67a9b51a02440af118fb4a34c332faaad40a1704ef6bee09f5ec373d88fc36/merged 2024-09-10T22:03:44.608 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d5d9b09148e2eea2bbe0e8ce312021e74fc0f1e8e09a4437adfa136749d3246b/merged 2024-09-10T22:03:44.609 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-09-10T22:03:44.609 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:03:44.609 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/a2dc6ee73be1567fd8f29eb80e26f1703773796ce8455114e9ea424c72750d7c/merged 2024-09-10T22:03:44.609 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f9c986101c7a2f5bb7fa1e8bd02849530b16017961224132afbbb799d1392c45/merged 2024-09-10T22:03:44.609 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e37fdfa87f4eac35931f91db6e1b46b42cc7a219cdc3b5db5ebc31f16ee9c77b/merged 2024-09-10T22:03:44.609 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c00ae7ca7df917931de3ddaff2d63373ed5b918da32f164d120bb626ebb226b5/merged 2024-09-10T22:03:44.609 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/bca883ef8c983cdd19c8073e9384d35dfcb291061c8b1a1238032a74f87ec46f/merged 2024-09-10T22:03:44.609 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1facfaa64ed970af42ecf309afb8987b16b91f2d32389ffe22e1c3cb3b1014b6/merged 2024-09-10T22:03:44.609 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/ef00c446100b225a6c68486a124b6d6f154b98394c2c94e127800d7ca868be21/merged 2024-09-10T22:03:44.610 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/937017d01ece5d353e6824b353f0a967ab18de84c0941cf309598207c4f60be5/merged 2024-09-10T22:03:44.610 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-09-10T22:03:44.610 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:44.610 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:44.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:44.611 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-09-10T22:03:44.686 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T22:03:44.686 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/15d452fc-d875-48e7-8057-49ebca12b7b4 --client_fs=cephfs) 2024-09-10T22:03:44.730 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-09-10T22:03:44.787 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-09-10T22:03:44.826 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-09-10T22:03:44.827 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/aa67a9b51a02440af118fb4a34c332faaad40a1704ef6bee09f5ec373d88fc36/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d5d9b09148e2eea2bbe0e8ce312021e74fc0f1e8e09a4437adfa136749d3246b/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/a2dc6ee73be1567fd8f29eb80e26f1703773796ce8455114e9ea424c72750d7c/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f9c986101c7a2f5bb7fa1e8bd02849530b16017961224132afbbb799d1392c45/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e37fdfa87f4eac35931f91db6e1b46b42cc7a219cdc3b5db5ebc31f16ee9c77b/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c00ae7ca7df917931de3ddaff2d63373ed5b918da32f164d120bb626ebb226b5/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/bca883ef8c983cdd19c8073e9384d35dfcb291061c8b1a1238032a74f87ec46f/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1facfaa64ed970af42ecf309afb8987b16b91f2d32389ffe22e1c3cb3b1014b6/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/ef00c446100b225a6c68486a124b6d6f154b98394c2c94e127800d7ca868be21/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/937017d01ece5d353e6824b353f0a967ab18de84c0941cf309598207c4f60be5/merged 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-09-10T22:03:44.828 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:44.829 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:44.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:44.830 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-09-10T22:03:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:44 smithi090 ceph-mon[30350]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:44.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:2024-09-10T22:03:44.879+0000 7f9ae7349480 -1 init, newargv = 0x55e56ff3b390 newargc=15 2024-09-10T22:03:44.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[89329]: starting ceph client 2024-09-10T22:03:44.912 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[89329]: starting fuse 2024-09-10T22:03:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:44 smithi154 ceph-mon[34495]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:45.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:44 smithi171 ceph-mon[34138]: pgmap v413: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:03:45.994 DEBUG:teuthology.orchestra.run.smithi090:> sudo modprobe fuse 2024-09-10T22:03:46.054 DEBUG:teuthology.orchestra.run.smithi090:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T22:03:46.091 INFO:teuthology.orchestra.run.smithi090.stdout:/proc 2024-09-10T22:03:46.091 INFO:teuthology.orchestra.run.smithi090.stdout:/sys 2024-09-10T22:03:46.091 INFO:teuthology.orchestra.run.smithi090.stdout:/dev 2024-09-10T22:03:46.091 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/security 2024-09-10T22:03:46.091 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/shm 2024-09-10T22:03:46.091 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/pts 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/run 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/cgroup 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/pstore 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/bpf 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/ 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/selinux 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/mqueue 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/dev/hugepages 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/debug 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/tracing 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T22:03:46.092 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/fs/fuse/connections 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/sys/kernel/config 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/1000 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/ceph 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/aa67a9b51a02440af118fb4a34c332faaad40a1704ef6bee09f5ec373d88fc36/merged 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/d5d9b09148e2eea2bbe0e8ce312021e74fc0f1e8e09a4437adfa136749d3246b/merged 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/run/user/0 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/a2dc6ee73be1567fd8f29eb80e26f1703773796ce8455114e9ea424c72750d7c/merged 2024-09-10T22:03:46.093 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/f9c986101c7a2f5bb7fa1e8bd02849530b16017961224132afbbb799d1392c45/merged 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/e37fdfa87f4eac35931f91db6e1b46b42cc7a219cdc3b5db5ebc31f16ee9c77b/merged 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/c00ae7ca7df917931de3ddaff2d63373ed5b918da32f164d120bb626ebb226b5/merged 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/bca883ef8c983cdd19c8073e9384d35dfcb291061c8b1a1238032a74f87ec46f/merged 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/1facfaa64ed970af42ecf309afb8987b16b91f2d32389ffe22e1c3cb3b1014b6/merged 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/ef00c446100b225a6c68486a124b6d6f154b98394c2c94e127800d7ca868be21/merged 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/var/lib/containers/storage/overlay/937017d01ece5d353e6824b353f0a967ab18de84c0941cf309598207c4f60be5/merged 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:03:46.094 INFO:teuthology.orchestra.run.smithi090.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:46.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:46.096 DEBUG:teuthology.orchestra.run.smithi090:> ls /sys/fs/fuse/connections 2024-09-10T22:03:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:45 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:03:46.141 INFO:teuthology.orchestra.run.smithi090.stdout:96 2024-09-10T22:03:46.142 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [96] 2024-09-10T22:03:46.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo stdin-killer -- python3 -c ' 2024-09-10T22:03:46.142 DEBUG:teuthology.orchestra.run.smithi090:> import glob 2024-09-10T22:03:46.142 DEBUG:teuthology.orchestra.run.smithi090:> import re 2024-09-10T22:03:46.142 DEBUG:teuthology.orchestra.run.smithi090:> import os 2024-09-10T22:03:46.142 DEBUG:teuthology.orchestra.run.smithi090:> import subprocess 2024-09-10T22:03:46.142 DEBUG:teuthology.orchestra.run.smithi090:> 2024-09-10T22:03:46.142 DEBUG:teuthology.orchestra.run.smithi090:> def _find_admin_socket(client_name): 2024-09-10T22:03:46.142 DEBUG:teuthology.orchestra.run.smithi090:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> files = glob.glob(asok_path) 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> # Given a non-glob path, it better be there 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> if "*" not in asok_path: 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> assert(len(files) == 1) 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> return files[0] 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> for f in files: 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> contents = proc_f.read() 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> if mountpoint in contents: 2024-09-10T22:03:46.143 DEBUG:teuthology.orchestra.run.smithi090:> return f 2024-09-10T22:03:46.144 DEBUG:teuthology.orchestra.run.smithi090:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T22:03:46.144 DEBUG:teuthology.orchestra.run.smithi090:> 2024-09-10T22:03:46.144 DEBUG:teuthology.orchestra.run.smithi090:> print(_find_admin_socket("client.0")) 2024-09-10T22:03:46.144 DEBUG:teuthology.orchestra.run.smithi090:> ' 2024-09-10T22:03:46.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:45 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:03:46.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:45 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:03:46.307 INFO:teuthology.orchestra.run.smithi090.stdout:/var/run/ceph/ceph-client.0.89329.asok 2024-09-10T22:03:46.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T22:03:46.324 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.89329.asok 2024-09-10T22:03:46.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:46.324 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.89329.asok status 2024-09-10T22:03:46.511 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "metadata": { 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "entity_id": "0", 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": "smithi090", 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "pid": "89329", 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "root": "/volumes/qa/sv_0/15d452fc-d875-48e7-8057-49ebca12b7b4" 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_count": 0, 2024-09-10T22:03:46.512 INFO:teuthology.orchestra.run.smithi090.stdout: "dentry_pinned_count": 0, 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 35276, 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "inst": { 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": { 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "client", 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "num": 35276 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2984236444 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:03:46.513 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": { 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2984236444 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "inst_str": "client.35276 172.21.15.90:0/2984236444", 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "addr_str": "172.21.15.90:0/2984236444", 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "inode_count": 5, 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_epoch": 29, 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch": 90, 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_epoch_barrier": 0, 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "blocklisted": false, 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs" 2024-09-10T22:03:46.514 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:03:46.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:46.525 DEBUG:teuthology.orchestra.run.smithi090:> stat --file-system '--printf=%T 2024-09-10T22:03:46.525 DEBUG:teuthology.orchestra.run.smithi090:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:46.562 INFO:teuthology.orchestra.run.smithi090.stdout:fuseblk 2024-09-10T22:03:46.563 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:46.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:03:46.564 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T22:03:46.660 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T22:03:46.669 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T22:03:46.669 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-09-10T22:03:46.670 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-10T22:03:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.822+0000 7efdd5600640 1 -- 172.21.15.90:0/1492022458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0108e80 msgr2=0x7efdd010f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.822+0000 7efdd5600640 1 --2- 172.21.15.90:0/1492022458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0108e80 0x7efdd010f680 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009be0 tx=0x7efdc802a6c0 comp rx=0 tx=0).stop 2024-09-10T22:03:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.822+0000 7efdd5600640 1 -- 172.21.15.90:0/1492022458 shutdown_connections 2024-09-10T22:03:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.822+0000 7efdd5600640 1 --2- 172.21.15.90:0/1492022458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0108e80 0x7efdd010f680 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.822+0000 7efdd5600640 1 --2- 172.21.15.90:0/1492022458 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdd0108500 0x7efdd0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.822+0000 7efdd5600640 1 --2- 172.21.15.90:0/1492022458 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdd0102b00 0x7efdd0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:46.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.822+0000 7efdd5600640 1 -- 172.21.15.90:0/1492022458 >> 172.21.15.90:0/1492022458 conn(0x7efdd00fe580 msgr2=0x7efdd01009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:46.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.822+0000 7efdd5600640 1 -- 172.21.15.90:0/1492022458 shutdown_connections 2024-09-10T22:03:46.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.822+0000 7efdd5600640 1 -- 172.21.15.90:0/1492022458 wait complete. 2024-09-10T22:03:46.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.823+0000 7efdd5600640 1 Processor -- start 2024-09-10T22:03:46.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.823+0000 7efdd5600640 1 -- start start 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.823+0000 7efdd5600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0102b00 0x7efdd019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.823+0000 7efdd5600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdd0108500 0x7efdd019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.823+0000 7efdd5600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdd0108e80 0x7efdd01a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.823+0000 7efdd5600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efdd01129f0 con 0x7efdd0102b00 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.823+0000 7efdd5600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efdd0112870 con 0x7efdd0108500 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.823+0000 7efdd5600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efdd0112b70 con 0x7efdd0108e80 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdcfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0102b00 0x7efdd019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdcfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0102b00 0x7efdd019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51588/0 (socket says 172.21.15.90:51588) 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdcfe00640 1 -- 172.21.15.90:0/593663799 learned_addr learned my addr 172.21.15.90:0/593663799 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdc7400640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdd0108500 0x7efdd019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdd4c00640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdd0108e80 0x7efdd01a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdcfe00640 1 -- 172.21.15.90:0/593663799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdd0108e80 msgr2=0x7efdd01a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:46.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdcfe00640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdd0108e80 0x7efdd01a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdcfe00640 1 -- 172.21.15.90:0/593663799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdd0108500 msgr2=0x7efdd019b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdcfe00640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdd0108500 0x7efdd019b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdcfe00640 1 -- 172.21.15.90:0/593663799 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd01a1210 con 0x7efdd0102b00 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.824+0000 7efdcfe00640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0102b00 0x7efdd019b170 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efdc000e9c0 tx=0x7efdc000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.825+0000 7efdcd600640 1 -- 172.21.15.90:0/593663799 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc000cde0 con 0x7efdd0102b00 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.825+0000 7efdcd600640 1 -- 172.21.15.90:0/593663799 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdc00035f0 con 0x7efdd0102b00 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.825+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd01a1500 con 0x7efdd0102b00 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.825+0000 7efdcd600640 1 -- 172.21.15.90:0/593663799 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0010690 con 0x7efdd0102b00 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.825+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd01a7bc0 con 0x7efdd0102b00 2024-09-10T22:03:46.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.826+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd0102ee0 con 0x7efdd0102b00 2024-09-10T22:03:46.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.828+0000 7efdcd600640 1 -- 172.21.15.90:0/593663799 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7efdc0004000 con 0x7efdd0102b00 2024-09-10T22:03:46.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.829+0000 7efdcd600640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efdb40b5330 0x7efdb40b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:46.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.829+0000 7efdcd600640 1 -- 172.21.15.90:0/593663799 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7efdc00e63a0 con 0x7efdd0102b00 2024-09-10T22:03:46.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.830+0000 7efdc7400640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efdb40b5330 0x7efdb40b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:46.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.830+0000 7efdc7400640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efdb40b5330 0x7efdb40b77f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efdd0106a70 tx=0x7efdb8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:46.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:46.833+0000 7efdcd600640 1 -- 172.21.15.90:0/593663799 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efdc00acd70 con 0x7efdd0102b00 2024-09-10T22:03:47.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.055+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7efdd002e5c0 con 0x7efdd0102b00 2024-09-10T22:03:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:46 smithi090 ceph-mon[30350]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:03:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:03:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:03:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:03:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:46 smithi154 ceph-mon[34495]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:03:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:03:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:03:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:03:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:46 smithi171 ceph-mon[34138]: pgmap v414: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:03:47.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:03:47.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:03:47.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:03:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.821+0000 7efdcd600640 1 -- 172.21.15.90:0/593663799 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7efdc00ac670 con 0x7efdd0102b00 2024-09-10T22:03:47.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.824+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efdb40b5330 msgr2=0x7efdb40b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:47.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.824+0000 7efdd5600640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efdb40b5330 0x7efdb40b77f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efdd0106a70 tx=0x7efdb8005f90 comp rx=0 tx=0).stop 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.824+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0102b00 msgr2=0x7efdd019b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.824+0000 7efdd5600640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0102b00 0x7efdd019b170 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efdc000e9c0 tx=0x7efdc000ee80 comp rx=0 tx=0).stop 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.825+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 shutdown_connections 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.825+0000 7efdd5600640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efdb40b5330 0x7efdb40b77f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.825+0000 7efdd5600640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efdd0108e80 0x7efdd01a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.825+0000 7efdd5600640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efdd0108500 0x7efdd019b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.825+0000 7efdd5600640 1 --2- 172.21.15.90:0/593663799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efdd0102b00 0x7efdd019b170 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.825+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 >> 172.21.15.90:0/593663799 conn(0x7efdd00fe580 msgr2=0x7efdd0104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.825+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 shutdown_connections 2024-09-10T22:03:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:47.825+0000 7efdd5600640 1 -- 172.21.15.90:0/593663799 wait complete. 2024-09-10T22:03:47.845 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T22:03:47.859 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T22:03:47.867 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T22:03:47.867 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-09-10T22:03:47.867 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-10T22:03:48.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.023+0000 7f4dedc00640 1 -- 172.21.15.90:0/1999974711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108500 msgr2=0x7f4de8108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.023+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1999974711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108500 0x7f4de8108940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0009900 tx=0x7f4dd002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.023+0000 7f4dedc00640 1 -- 172.21.15.90:0/1999974711 shutdown_connections 2024-09-10T22:03:48.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.023+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1999974711 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4de8108e80 0x7f4de810f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.023+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1999974711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108500 0x7f4de8108940 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.023+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1999974711 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de8102b00 0x7f4de8102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.023+0000 7f4dedc00640 1 -- 172.21.15.90:0/1999974711 >> 172.21.15.90:0/1999974711 conn(0x7f4de80fe580 msgr2=0x7f4de81009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:48.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.023+0000 7f4dedc00640 1 -- 172.21.15.90:0/1999974711 shutdown_connections 2024-09-10T22:03:48.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.023+0000 7f4dedc00640 1 -- 172.21.15.90:0/1999974711 wait complete. 2024-09-10T22:03:48.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.024+0000 7f4dedc00640 1 Processor -- start 2024-09-10T22:03:48.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.024+0000 7f4dedc00640 1 -- start start 2024-09-10T22:03:48.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.024+0000 7f4dedc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de8102b00 0x7f4de819b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:48.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.024+0000 7f4dedc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4de8108500 0x7f4de819b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:48.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4dedc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108e80 0x7f4de81a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:48.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4dedc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4de81129f0 con 0x7f4de8108e80 2024-09-10T22:03:48.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4dedc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112870 con 0x7f4de8102b00 2024-09-10T22:03:48.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4dedc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4de8112b70 con 0x7f4de8108500 2024-09-10T22:03:48.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ded200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108e80 0x7f4de81a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ded200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108e80 0x7f4de81a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60640/0 (socket says 172.21.15.90:60640) 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ded200640 1 -- 172.21.15.90:0/1331992840 learned_addr learned my addr 172.21.15.90:0/1331992840 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ddf400640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4de8108500 0x7f4de819b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4de7e00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de8102b00 0x7f4de819b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ded200640 1 -- 172.21.15.90:0/1331992840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4de8108500 msgr2=0x7f4de819b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ded200640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4de8108500 0x7f4de819b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ded200640 1 -- 172.21.15.90:0/1331992840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de8102b00 msgr2=0x7f4de819b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ded200640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de8102b00 0x7f4de819b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ded200640 1 -- 172.21.15.90:0/1331992840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4de81a11b0 con 0x7f4de8108e80 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4ddf400640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4de8108500 0x7f4de819b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.025+0000 7f4de7e00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de8102b00 0x7f4de819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:03:48.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.026+0000 7f4ded200640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108e80 0x7f4de81a09e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4de000c970 tx=0x7f4de000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:48.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.026+0000 7f4de5600640 1 -- 172.21.15.90:0/1331992840 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0007bb0 con 0x7f4de8108e80 2024-09-10T22:03:48.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.026+0000 7f4de5600640 1 -- 172.21.15.90:0/1331992840 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4de0007d50 con 0x7f4de8108e80 2024-09-10T22:03:48.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.026+0000 7f4de5600640 1 -- 172.21.15.90:0/1331992840 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0018660 con 0x7f4de8108e80 2024-09-10T22:03:48.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.026+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4de81a14a0 con 0x7f4de8108e80 2024-09-10T22:03:48.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.026+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4de81a7b60 con 0x7f4de8108e80 2024-09-10T22:03:48.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.027+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4de8102ee0 con 0x7f4de8108e80 2024-09-10T22:03:48.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.029+0000 7f4de5600640 1 -- 172.21.15.90:0/1331992840 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f4de0020030 con 0x7f4de8108e80 2024-09-10T22:03:48.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.031+0000 7f4de5600640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4dcc0b53b0 0x7f4dcc0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:48.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.031+0000 7f4de5600640 1 -- 172.21.15.90:0/1331992840 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f4de00e5f30 con 0x7f4de8108e80 2024-09-10T22:03:48.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.031+0000 7f4de7e00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4dcc0b53b0 0x7f4dcc0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:48.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.032+0000 7f4de7e00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4dcc0b53b0 0x7f4dcc0b7870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8005fd0 tx=0x7f4dd8009510 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:48.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.033+0000 7f4de5600640 1 -- 172.21.15.90:0/1331992840 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4de0056220 con 0x7f4de8108e80 2024-09-10T22:03:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:47 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/593663799' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-10T22:03:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:47 smithi090 ceph-mon[30350]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:48.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:47 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/593663799' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-10T22:03:48.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:47 smithi154 ceph-mon[34495]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:48.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.238+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f4de8063590 con 0x7f4dcc0b53b0 2024-09-10T22:03:48.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:47 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/593663799' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-10T22:03:48.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:47 smithi171 ceph-mon[34138]: pgmap v415: 129 pgs: 129 active+clean; 647 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:03:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.427+0000 7f4de5600640 1 -- 172.21.15.90:0/1331992840 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4de8063590 con 0x7f4dcc0b53b0 2024-09-10T22:03:48.429 INFO:teuthology.orchestra.run.smithi090.stdout:Schedule set for path / 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4dcc0b53b0 msgr2=0x7f4dcc0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4dcc0b53b0 0x7f4dcc0b7870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8005fd0 tx=0x7f4dd8009510 comp rx=0 tx=0).stop 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108e80 msgr2=0x7f4de81a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108e80 0x7f4de81a09e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4de000c970 tx=0x7f4de000ce30 comp rx=0 tx=0).stop 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 shutdown_connections 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4dcc0b53b0 0x7f4dcc0b7870 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4de8108e80 0x7f4de81a09e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4de8108500 0x7f4de819b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 --2- 172.21.15.90:0/1331992840 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4de8102b00 0x7f4de819b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 >> 172.21.15.90:0/1331992840 conn(0x7f4de80fe580 msgr2=0x7f4de8104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:48.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 shutdown_connections 2024-09-10T22:03:48.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.430+0000 7f4dedc00640 1 -- 172.21.15.90:0/1331992840 wait complete. 2024-09-10T22:03:48.447 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-10T22:03:48.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.600+0000 7f254e800640 1 -- 172.21.15.90:0/4019293232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f254810f2d0 msgr2=0x7f25481116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.600+0000 7f254e800640 1 --2- 172.21.15.90:0/4019293232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f254810f2d0 0x7f25481116c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2540009ca0 tx=0x7f254002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.600+0000 7f254e800640 1 -- 172.21.15.90:0/4019293232 shutdown_connections 2024-09-10T22:03:48.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.600+0000 7f254e800640 1 --2- 172.21.15.90:0/4019293232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f254810f2d0 0x7f25481116c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.600+0000 7f254e800640 1 --2- 172.21.15.90:0/4019293232 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2548075e00 0x7f254810ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.600+0000 7f254e800640 1 --2- 172.21.15.90:0/4019293232 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25480754e0 0x7f25480758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.600+0000 7f254e800640 1 -- 172.21.15.90:0/4019293232 >> 172.21.15.90:0/4019293232 conn(0x7f25480fe730 msgr2=0x7f2548100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:48.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.600+0000 7f254e800640 1 -- 172.21.15.90:0/4019293232 shutdown_connections 2024-09-10T22:03:48.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.600+0000 7f254e800640 1 -- 172.21.15.90:0/4019293232 wait complete. 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.601+0000 7f254e800640 1 Processor -- start 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.601+0000 7f254e800640 1 -- start start 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.601+0000 7f254e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25480754e0 0x7f254819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.601+0000 7f254e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2548075e00 0x7f254819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.601+0000 7f254e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f254810f2d0 0x7f25481a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.601+0000 7f254e800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2548114db0 con 0x7f2548075e00 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.601+0000 7f254e800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2548114c30 con 0x7f254810f2d0 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.601+0000 7f254e800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2548114f30 con 0x7f25480754e0 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f253fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2548075e00 0x7f254819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f253fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2548075e00 0x7f254819b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60674/0 (socket says 172.21.15.90:60674) 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f253fe00640 1 -- 172.21.15.90:0/1013593563 learned_addr learned my addr 172.21.15.90:0/1013593563 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:48.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f2547400640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25480754e0 0x7f254819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f2547e00640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f254810f2d0 0x7f25481a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f253fe00640 1 -- 172.21.15.90:0/1013593563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25480754e0 msgr2=0x7f254819b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f253fe00640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25480754e0 0x7f254819b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f253fe00640 1 -- 172.21.15.90:0/1013593563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f254810f2d0 msgr2=0x7f25481a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f253fe00640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f254810f2d0 0x7f25481a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f253fe00640 1 -- 172.21.15.90:0/1013593563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25481a1330 con 0x7f2548075e00 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f253fe00640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2548075e00 0x7f254819b7d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f253000c920 tx=0x7f253000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.602+0000 7f2544c00640 1 -- 172.21.15.90:0/1013593563 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530007c70 con 0x7f2548075e00 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.603+0000 7f2544c00640 1 -- 172.21.15.90:0/1013593563 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2530007e10 con 0x7f2548075e00 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.603+0000 7f2544c00640 1 -- 172.21.15.90:0/1013593563 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2530018710 con 0x7f2548075e00 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.603+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25481a1620 con 0x7f2548075e00 2024-09-10T22:03:48.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.603+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25481a7ce0 con 0x7f2548075e00 2024-09-10T22:03:48.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.604+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25480758c0 con 0x7f2548075e00 2024-09-10T22:03:48.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.607+0000 7f2544c00640 1 -- 172.21.15.90:0/1013593563 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f253000f450 con 0x7f2548075e00 2024-09-10T22:03:48.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.610+0000 7f2544c00640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f252c0b53b0 0x7f252c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:48.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.610+0000 7f2544c00640 1 -- 172.21.15.90:0/1013593563 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f25300e6320 con 0x7f2548075e00 2024-09-10T22:03:48.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.610+0000 7f2547400640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f252c0b53b0 0x7f252c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:48.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.611+0000 7f2547400640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f252c0b53b0 0x7f252c0b7870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f253800bbe0 tx=0x7f253800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:48.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.614+0000 7f2544c00640 1 -- 172.21.15.90:0/1013593563 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2530014030 con 0x7f2548075e00 2024-09-10T22:03:48.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.825+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f2548100a50 con 0x7f252c0b53b0 2024-09-10T22:03:48.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.895+0000 7f2544c00640 1 -- 172.21.15.90:0/1013593563 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f2548100a50 con 0x7f252c0b53b0 2024-09-10T22:03:48.897 INFO:teuthology.orchestra.run.smithi090.stdout:Retention added to path / 2024-09-10T22:03:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.897+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f252c0b53b0 msgr2=0x7f252c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.897+0000 7f254e800640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f252c0b53b0 0x7f252c0b7870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f253800bbe0 tx=0x7f253800b390 comp rx=0 tx=0).stop 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2548075e00 msgr2=0x7f254819b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2548075e00 0x7f254819b7d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f253000c920 tx=0x7f253000cde0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 shutdown_connections 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f252c0b53b0 0x7f252c0b7870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f254810f2d0 0x7f25481a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2548075e00 0x7f254819b7d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 --2- 172.21.15.90:0/1013593563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f25480754e0 0x7f254819b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 >> 172.21.15.90:0/1013593563 conn(0x7f25480fe730 msgr2=0x7f25480ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 shutdown_connections 2024-09-10T22:03:48.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:48.898+0000 7f254e800640 1 -- 172.21.15.90:0/1013593563 wait complete. 2024-09-10T22:03:48.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-10T22:03:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.064+0000 7f3d36e00640 1 -- 172.21.15.90:0/1723279668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d3010a8f0 msgr2=0x7f3d3010ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.064+0000 7f3d36e00640 1 --2- 172.21.15.90:0/1723279668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d3010a8f0 0x7f3d3010ad30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009900 tx=0x7f3d1c02a410 comp rx=0 tx=0).stop 2024-09-10T22:03:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.065+0000 7f3d36e00640 1 -- 172.21.15.90:0/1723279668 shutdown_connections 2024-09-10T22:03:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.065+0000 7f3d36e00640 1 --2- 172.21.15.90:0/1723279668 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d3010b270 0x7f3d30111a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.065+0000 7f3d36e00640 1 --2- 172.21.15.90:0/1723279668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d3010a8f0 0x7f3d3010ad30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.065+0000 7f3d36e00640 1 --2- 172.21.15.90:0/1723279668 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d30104ef0 0x7f3d301052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.065+0000 7f3d36e00640 1 -- 172.21.15.90:0/1723279668 >> 172.21.15.90:0/1723279668 conn(0x7f3d30100910 msgr2=0x7f3d30102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.065+0000 7f3d36e00640 1 -- 172.21.15.90:0/1723279668 shutdown_connections 2024-09-10T22:03:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.065+0000 7f3d36e00640 1 -- 172.21.15.90:0/1723279668 wait complete. 2024-09-10T22:03:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.066+0000 7f3d36e00640 1 Processor -- start 2024-09-10T22:03:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.066+0000 7f3d36e00640 1 -- start start 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.066+0000 7f3d36e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d30104ef0 0x7f3d3019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.066+0000 7f3d36e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d3010a8f0 0x7f3d3019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.066+0000 7f3d36e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d3010b270 0x7f3d301a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.066+0000 7f3d36e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30114de0 con 0x7f3d3010b270 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.066+0000 7f3d36e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30114c60 con 0x7f3d3010a8f0 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.066+0000 7f3d36e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30114f60 con 0x7f3d30104ef0 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d34e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d3010b270 0x7f3d301a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d30104ef0 0x7f3d3019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d30104ef0 0x7f3d3019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:57008/0 (socket says 172.21.15.90:57008) 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2fe00640 1 -- 172.21.15.90:0/3182683811 learned_addr learned my addr 172.21.15.90:0/3182683811 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2f400640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d3010a8f0 0x7f3d3019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2fe00640 1 -- 172.21.15.90:0/3182683811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d3010a8f0 msgr2=0x7f3d3019b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2fe00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d3010a8f0 0x7f3d3019b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2fe00640 1 -- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d3010b270 msgr2=0x7f3d301a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2fe00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d3010b270 0x7f3d301a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2fe00640 1 -- 172.21.15.90:0/3182683811 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d301a1480 con 0x7f3d30104ef0 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d34e00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d3010b270 0x7f3d301a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.067+0000 7f3d2fe00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d30104ef0 0x7f3d3019b3e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000ea80 tx=0x7f3d2000ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.068+0000 7f3d2cc00640 1 -- 172.21.15.90:0/3182683811 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2000ce40 con 0x7f3d30104ef0 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.068+0000 7f3d2cc00640 1 -- 172.21.15.90:0/3182683811 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d20005020 con 0x7f3d30104ef0 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.068+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d301a1770 con 0x7f3d30104ef0 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.068+0000 7f3d2cc00640 1 -- 172.21.15.90:0/3182683811 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200116d0 con 0x7f3d30104ef0 2024-09-10T22:03:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.068+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d301a7eb0 con 0x7f3d30104ef0 2024-09-10T22:03:49.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.069+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d301052d0 con 0x7f3d30104ef0 2024-09-10T22:03:49.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.071+0000 7f3d2cc00640 1 -- 172.21.15.90:0/3182683811 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7f3d2000f040 con 0x7f3d30104ef0 2024-09-10T22:03:49.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.072+0000 7f3d2cc00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d140b53b0 0x7f3d140b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:49.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.073+0000 7f3d2cc00640 1 -- 172.21.15.90:0/3182683811 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3d200e62d0 con 0x7f3d30104ef0 2024-09-10T22:03:49.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.073+0000 7f3d2f400640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d140b53b0 0x7f3d140b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:49.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.073+0000 7f3d2f400640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d140b53b0 0x7f3d140b7870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c004770 tx=0x7f3d1c005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:49.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.074+0000 7f3d2cc00640 1 -- 172.21.15.90:0/3182683811 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3d20016030 con 0x7f3d30104ef0 2024-09-10T22:03:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:48 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/593663799' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T22:03:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:48 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:03:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:48 smithi090 ceph-mon[30350]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:48 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/593663799' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T22:03:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:48 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:03:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:48 smithi154 ceph-mon[34495]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:48 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/593663799' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T22:03:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:48 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:03:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:48 smithi171 ceph-mon[34138]: from='client.15561 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:49.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.281+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f3d3002e5f0 con 0x7f3d140b53b0 2024-09-10T22:03:49.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.288+0000 7f3d2cc00640 1 -- 172.21.15.90:0/3182683811 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f3d3002e5f0 con 0x7f3d140b53b0 2024-09-10T22:03:49.290 INFO:teuthology.orchestra.run.smithi090.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-10T00:00:00", "created": "2024-09-10T22:03:48", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-10T22:03:49.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d140b53b0 msgr2=0x7f3d140b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d140b53b0 0x7f3d140b7870 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c004770 tx=0x7f3d1c005fd0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d30104ef0 msgr2=0x7f3d3019b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d30104ef0 0x7f3d3019b3e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000ea80 tx=0x7f3d2000ef40 comp rx=0 tx=0).stop 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 shutdown_connections 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d140b53b0 0x7f3d140b7870 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d3010b270 0x7f3d301a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d3010a8f0 0x7f3d3019b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 --2- 172.21.15.90:0/3182683811 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d30104ef0 0x7f3d3019b3e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 >> 172.21.15.90:0/3182683811 conn(0x7f3d30100910 msgr2=0x7f3d30102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:49.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 shutdown_connections 2024-09-10T22:03:49.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.291+0000 7f3d36e00640 1 -- 172.21.15.90:0/3182683811 wait complete. 2024-09-10T22:03:49.308 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-10T22:03:49.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.458+0000 7fa8b9800640 1 -- 172.21.15.90:0/1196047834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 msgr2=0x7fa8b4105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.458+0000 7fa8b9800640 1 --2- 172.21.15.90:0/1196047834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 0x7fa8b4105100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009900 tx=0x7fa8a402a400 comp rx=0 tx=0).stop 2024-09-10T22:03:49.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.458+0000 7fa8b9800640 1 -- 172.21.15.90:0/1196047834 shutdown_connections 2024-09-10T22:03:49.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.458+0000 7fa8b9800640 1 --2- 172.21.15.90:0/1196047834 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8b410b0a0 0x7fa8b41118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.458+0000 7fa8b9800640 1 --2- 172.21.15.90:0/1196047834 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b410a720 0x7fa8b410ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.458+0000 7fa8b9800640 1 --2- 172.21.15.90:0/1196047834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 0x7fa8b4105100 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.458+0000 7fa8b9800640 1 -- 172.21.15.90:0/1196047834 >> 172.21.15.90:0/1196047834 conn(0x7fa8b4100740 msgr2=0x7fa8b4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:49.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.458+0000 7fa8b9800640 1 -- 172.21.15.90:0/1196047834 shutdown_connections 2024-09-10T22:03:49.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.458+0000 7fa8b9800640 1 -- 172.21.15.90:0/1196047834 wait complete. 2024-09-10T22:03:49.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.459+0000 7fa8b9800640 1 Processor -- start 2024-09-10T22:03:49.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.459+0000 7fa8b9800640 1 -- start start 2024-09-10T22:03:49.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.459+0000 7fa8b9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 0x7fa8b419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.459+0000 7fa8b9800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b410a720 0x7fa8b419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b9800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8b410b0a0 0x7fa8b41a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b9800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b4114c20 con 0x7fa8b4104d20 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b9800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b4114aa0 con 0x7fa8b410a720 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b9800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b4114da0 con 0x7fa8b410b0a0 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 0x7fa8b419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 0x7fa8b419b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60710/0 (socket says 172.21.15.90:60710) 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b3e00640 1 -- 172.21.15.90:0/460927943 learned_addr learned my addr 172.21.15.90:0/460927943 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b3400640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b410a720 0x7fa8b419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b8e00640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8b410b0a0 0x7fa8b41a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b3e00640 1 -- 172.21.15.90:0/460927943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8b410b0a0 msgr2=0x7fa8b41a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b3e00640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8b410b0a0 0x7fa8b41a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b3e00640 1 -- 172.21.15.90:0/460927943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b410a720 msgr2=0x7fa8b419b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b3e00640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b410a720 0x7fa8b419b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b3e00640 1 -- 172.21.15.90:0/460927943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b41a1120 con 0x7fa8b4104d20 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.460+0000 7fa8b8e00640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8b410b0a0 0x7fa8b41a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.461+0000 7fa8b3e00640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 0x7fa8b419b080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa8a402a920 tx=0x7fa8a4003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.461+0000 7fa8b0c00640 1 -- 172.21.15.90:0/460927943 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a403f070 con 0x7fa8b4104d20 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.461+0000 7fa8b0c00640 1 -- 172.21.15.90:0/460927943 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8a403bd00 con 0x7fa8b4104d20 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.461+0000 7fa8b0c00640 1 -- 172.21.15.90:0/460927943 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4035030 con 0x7fa8b4104d20 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.461+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b41a13b0 con 0x7fa8b4104d20 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.461+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b41a7ad0 con 0x7fa8b4104d20 2024-09-10T22:03:49.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.462+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b4105100 con 0x7fa8b4104d20 2024-09-10T22:03:49.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.464+0000 7fa8b0c00640 1 -- 172.21.15.90:0/460927943 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 22) ==== 153267+0+0 (secure 0 0 0) 0x7fa8a4031400 con 0x7fa8b4104d20 2024-09-10T22:03:49.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.465+0000 7fa8b0c00640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8980b5330 0x7fa8980b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:49.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.465+0000 7fa8b0c00640 1 -- 172.21.15.90:0/460927943 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa8a4106e10 con 0x7fa8b4104d20 2024-09-10T22:03:49.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.466+0000 7fa8b3400640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8980b5330 0x7fa8980b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:49.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.466+0000 7fa8b3400640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8980b5330 0x7fa8980b77f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4103fd0 tx=0x7fa8a000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:49.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.469+0000 7fa8b0c00640 1 -- 172.21.15.90:0/460927943 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa8a410a030 con 0x7fa8b4104d20 2024-09-10T22:03:49.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.673+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa8b4063590 con 0x7fa8980b5330 2024-09-10T22:03:49.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.680+0000 7fa8b0c00640 1 -- 172.21.15.90:0/460927943 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa8b4063590 con 0x7fa8980b5330 2024-09-10T22:03:49.682 INFO:teuthology.orchestra.run.smithi090.stdout:/ 1m 6m3h 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8980b5330 msgr2=0x7fa8980b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8980b5330 0x7fa8980b77f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4103fd0 tx=0x7fa8a000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 msgr2=0x7fa8b419b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 0x7fa8b419b080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa8a402a920 tx=0x7fa8a4003d60 comp rx=0 tx=0).stop 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 shutdown_connections 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8980b5330 0x7fa8980b77f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8b410b0a0 0x7fa8b41a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8b410a720 0x7fa8b419b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 --2- 172.21.15.90:0/460927943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b4104d20 0x7fa8b419b080 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 >> 172.21.15.90:0/460927943 conn(0x7fa8b4100740 msgr2=0x7fa8b4102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 shutdown_connections 2024-09-10T22:03:49.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:49.683+0000 7fa8b9800640 1 -- 172.21.15.90:0/460927943 wait complete. 2024-09-10T22:03:49.700 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-10T22:03:49.759 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-10T22:03:49.770 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-10T22:03:49.790 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T22:03:49.790 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:03:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:49 smithi090 ceph-mon[30350]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:49 smithi090 ceph-mon[30350]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:49 smithi090 ceph-mon[30350]: pgmap v416: 129 pgs: 129 active+clean; 1.0 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 91 KiB/s wr, 34 op/s 2024-09-10T22:03:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:49 smithi090 ceph-mon[30350]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:49 smithi154 ceph-mon[34495]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:49 smithi154 ceph-mon[34495]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:49 smithi154 ceph-mon[34495]: pgmap v416: 129 pgs: 129 active+clean; 1.0 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 91 KiB/s wr, 34 op/s 2024-09-10T22:03:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:49 smithi154 ceph-mon[34495]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:49 smithi171 ceph-mon[34138]: from='client.15573 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:49 smithi171 ceph-mon[34138]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:49 smithi171 ceph-mon[34138]: pgmap v416: 129 pgs: 129 active+clean; 1.0 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 91 KiB/s wr, 34 op/s 2024-09-10T22:03:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:49 smithi171 ceph-mon[34138]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:03:50.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.781+0000 7f2f4be00640 1 -- 172.21.15.90:0/3835134037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108620 msgr2=0x7f2f4c108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:50.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.781+0000 7f2f4be00640 1 --2- 172.21.15.90:0/3835134037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108620 0x7f2f4c108a60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009900 tx=0x7f2f3c02a410 comp rx=0 tx=0).stop 2024-09-10T22:03:50.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.781+0000 7f2f4be00640 1 -- 172.21.15.90:0/3835134037 shutdown_connections 2024-09-10T22:03:50.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.781+0000 7f2f4be00640 1 --2- 172.21.15.90:0/3835134037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c108fa0 0x7f2f4c10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:50.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.781+0000 7f2f4be00640 1 --2- 172.21.15.90:0/3835134037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108620 0x7f2f4c108a60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:50.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.781+0000 7f2f4be00640 1 --2- 172.21.15.90:0/3835134037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f4c102c10 0x7f2f4c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:50.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.781+0000 7f2f4be00640 1 -- 172.21.15.90:0/3835134037 >> 172.21.15.90:0/3835134037 conn(0x7f2f4c0fe660 msgr2=0x7f2f4c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:50.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.781+0000 7f2f4be00640 1 -- 172.21.15.90:0/3835134037 shutdown_connections 2024-09-10T22:03:50.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.781+0000 7f2f4be00640 1 -- 172.21.15.90:0/3835134037 wait complete. 2024-09-10T22:03:50.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.782+0000 7f2f4be00640 1 Processor -- start 2024-09-10T22:03:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4be00640 1 -- start start 2024-09-10T22:03:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c102c10 0x7f2f4c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f4c108620 0x7f2f4c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108fa0 0x7f2f4c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c112ae0 con 0x7f2f4c108fa0 2024-09-10T22:03:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c112960 con 0x7f2f4c102c10 2024-09-10T22:03:50.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c112c60 con 0x7f2f4c108620 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108fa0 0x7f2f4c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108fa0 0x7f2f4c075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60716/0 (socket says 172.21.15.90:60716) 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4b400640 1 -- 172.21.15.90:0/857460652 learned_addr learned my addr 172.21.15.90:0/857460652 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4aa00640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c102c10 0x7f2f4c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.783+0000 7f2f4a000640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f4c108620 0x7f2f4c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f4b400640 1 -- 172.21.15.90:0/857460652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f4c108620 msgr2=0x7f2f4c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f4b400640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f4c108620 0x7f2f4c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f4b400640 1 -- 172.21.15.90:0/857460652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c102c10 msgr2=0x7f2f4c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f4b400640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c102c10 0x7f2f4c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f4b400640 1 -- 172.21.15.90:0/857460652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4c1a73d0 con 0x7f2f4c108fa0 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f4b400640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108fa0 0x7f2f4c075f20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009fd0 tx=0x7f2f4400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f43400640 1 -- 172.21.15.90:0/857460652 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44017070 con 0x7f2f4c108fa0 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f43400640 1 -- 172.21.15.90:0/857460652 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f44003e00 con 0x7f2f4c108fa0 2024-09-10T22:03:50.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f43400640 1 -- 172.21.15.90:0/857460652 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f44010460 con 0x7f2f4c108fa0 2024-09-10T22:03:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.784+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4c1a76c0 con 0x7f2f4c108fa0 2024-09-10T22:03:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.785+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f4c1a7bd0 con 0x7f2f4c108fa0 2024-09-10T22:03:50.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.786+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f4c102ff0 con 0x7f2f4c108fa0 2024-09-10T22:03:50.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.788+0000 7f2f43400640 1 -- 172.21.15.90:0/857460652 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f2f44003970 con 0x7f2f4c108fa0 2024-09-10T22:03:50.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.789+0000 7f2f43400640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f300b54e0 0x7f2f300b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:50.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.789+0000 7f2f43400640 1 -- 172.21.15.90:0/857460652 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f2f440e6300 con 0x7f2f4c108fa0 2024-09-10T22:03:50.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.789+0000 7f2f4aa00640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f300b54e0 0x7f2f300b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:50.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.790+0000 7f2f4aa00640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f300b54e0 0x7f2f300b79a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005fd0 tx=0x7f2f3400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:50.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.793+0000 7f2f43400640 1 -- 172.21.15.90:0/857460652 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2f440ac520 con 0x7f2f4c108fa0 2024-09-10T22:03:50.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.977+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f4c063590 con 0x7f2f300b54e0 2024-09-10T22:03:50.979 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:50 smithi090 ceph-mon[30350]: mgrmap e23: x(active, since 11m), standbys: z, y 2024-09-10T22:03:50.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.985+0000 7f2f43400640 1 -- 172.21.15.90:0/857460652 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340841 (secure 0 0 0) 0x7f2f4c063590 con 0x7f2f300b54e0 2024-09-10T22:03:50.991 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:50.992 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-09-10T22:03:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f300b54e0 msgr2=0x7f2f300b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f300b54e0 0x7f2f300b79a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005fd0 tx=0x7f2f3400b390 comp rx=0 tx=0).stop 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108fa0 msgr2=0x7f2f4c075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108fa0 0x7f2f4c075f20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009fd0 tx=0x7f2f4400b260 comp rx=0 tx=0).stop 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 shutdown_connections 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f300b54e0 0x7f2f300b79a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f4c108fa0 0x7f2f4c075f20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f4c108620 0x7f2f4c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 --2- 172.21.15.90:0/857460652 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f4c102c10 0x7f2f4c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 >> 172.21.15.90:0/857460652 conn(0x7f2f4c0fe660 msgr2=0x7f2f4c100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 shutdown_connections 2024-09-10T22:03:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:50.989+0000 7f2f4be00640 1 -- 172.21.15.90:0/857460652 wait complete. 2024-09-10T22:03:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:50 smithi154 ceph-mon[34495]: mgrmap e23: x(active, since 11m), standbys: z, y 2024-09-10T22:03:51.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:50 smithi171 ceph-mon[34138]: mgrmap e23: x(active, since 11m), standbys: z, y 2024-09-10T22:03:52.079 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:51 smithi090 ceph-mon[30350]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:03:52.080 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:51 smithi090 ceph-mon[30350]: pgmap v417: 129 pgs: 129 active+clean; 1.0 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 92 KiB/s wr, 34 op/s 2024-09-10T22:03:52.117 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-10T22:03:49.633145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1057246,"num_objects":103,"num_object_clones":0,"num_object_copies":311,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":1638,"num_read_kb":987,"num_write":496,"num_write_kb":3402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":883,"ondisk_log_size":883,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349076,"kb_used_data":12308,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592748,"statfs":{"total":1151940427776,"available":1151582973952,"internally_reserved":0,"allocated":12603392,"data_stored":8388750,"data_compressed":498330,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":395188,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":210,"num_write":109,"num_write_kb":1095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003254"},"pg_stats":[{"pgid":"2.1f","version":"90'15","reported_seq":197,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:46.979191+0000","last_change":"2024-09-10T22:00:30.824265+0000","last_active":"2024-09-10T22:03:46.979191+0000","last_peered":"2024-09-10T22:03:46.979191+0000","last_clean":"2024-09-10T22:03:46.979191+0000","last_became_active":"2024-09-10T22:00:30.823905+0000","last_became_peered":"2024-09-10T22:00:30.823905+0000","last_unstale":"2024-09-10T22:03:46.979191+0000","last_undegraded":"2024-09-10T22:03:46.979191+0000","last_fullsized":"2024-09-10T22:03:46.979191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:16.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837001+0000","last_change":"2024-09-10T22:00:34.497719+0000","last_active":"2024-09-10T22:03:03.837001+0000","last_peered":"2024-09-10T22:03:03.837001+0000","last_clean":"2024-09-10T22:03:03.837001+0000","last_became_active":"2024-09-10T22:00:34.497380+0000","last_became_peered":"2024-09-10T22:00:34.497380+0000","last_unstale":"2024-09-10T22:03:03.837001+0000","last_undegraded":"2024-09-10T22:03:03.837001+0000","last_fullsized":"2024-09-10T22:03:03.837001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:26.336299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723331+0000","last_change":"2024-09-10T22:00:43.710835+0000","last_active":"2024-09-10T22:02:35.723331+0000","last_peered":"2024-09-10T22:02:35.723331+0000","last_clean":"2024-09-10T22:02:35.723331+0000","last_became_active":"2024-09-10T22:00:43.710363+0000","last_became_peered":"2024-09-10T22:00:43.710363+0000","last_unstale":"2024-09-10T22:02:35.723331+0000","last_undegraded":"2024-09-10T22:02:35.723331+0000","last_fullsized":"2024-09-10T22:02:35.723331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:28.173482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723977+0000","last_change":"2024-09-10T22:00:31.083228+0000","last_active":"2024-09-10T22:02:35.723977+0000","last_peered":"2024-09-10T22:02:35.723977+0000","last_clean":"2024-09-10T22:02:35.723977+0000","last_became_active":"2024-09-10T22:00:31.082911+0000","last_became_peered":"2024-09-10T22:00:31.082911+0000","last_unstale":"2024-09-10T22:02:35.723977+0000","last_undegraded":"2024-09-10T22:02:35.723977+0000","last_fullsized":"2024-09-10T22:02:35.723977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:28.598628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853513+0000","last_change":"2024-09-10T22:00:34.509939+0000","last_active":"2024-09-10T22:03:03.853513+0000","last_peered":"2024-09-10T22:03:03.853513+0000","last_clean":"2024-09-10T22:03:03.853513+0000","last_became_active":"2024-09-10T22:00:34.509536+0000","last_became_peered":"2024-09-10T22:00:34.509536+0000","last_unstale":"2024-09-10T22:03:03.853513+0000","last_undegraded":"2024-09-10T22:03:03.853513+0000","last_fullsized":"2024-09-10T22:03:03.853513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:27.602378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897125+0000","last_change":"2024-09-10T22:00:43.714662+0000","last_active":"2024-09-10T22:02:35.897125+0000","last_peered":"2024-09-10T22:02:35.897125+0000","last_clean":"2024-09-10T22:02:35.897125+0000","last_became_active":"2024-09-10T22:00:43.714074+0000","last_became_peered":"2024-09-10T22:00:43.714074+0000","last_unstale":"2024-09-10T22:02:35.897125+0000","last_undegraded":"2024-09-10T22:02:35.897125+0000","last_fullsized":"2024-09-10T22:02:35.897125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:41.204519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948479+0000","last_change":"2024-09-10T22:01:35.209979+0000","last_active":"2024-09-10T22:02:35.948479+0000","last_peered":"2024-09-10T22:02:35.948479+0000","last_clean":"2024-09-10T22:02:35.948479+0000","last_became_active":"2024-09-10T22:01:35.197192+0000","last_became_peered":"2024-09-10T22:01:35.197192+0000","last_unstale":"2024-09-10T22:02:35.948479+0000","last_undegraded":"2024-09-10T22:02:35.948479+0000","last_fullsized":"2024-09-10T22:02:35.948479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:47.707679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853796+0000","last_change":"2024-09-10T22:00:34.502795+0000","last_active":"2024-09-10T22:03:03.853796+0000","last_peered":"2024-09-10T22:03:03.853796+0000","last_clean":"2024-09-10T22:03:03.853796+0000","last_became_active":"2024-09-10T22:00:34.501112+0000","last_became_peered":"2024-09-10T22:00:34.501112+0000","last_unstale":"2024-09-10T22:03:03.853796+0000","last_undegraded":"2024-09-10T22:03:03.853796+0000","last_fullsized":"2024-09-10T22:03:03.853796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:45.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897363+0000","last_change":"2024-09-10T22:01:35.619765+0000","last_active":"2024-09-10T22:02:35.897363+0000","last_peered":"2024-09-10T22:02:35.897363+0000","last_clean":"2024-09-10T22:02:35.897363+0000","last_became_active":"2024-09-10T22:01:35.619407+0000","last_became_peered":"2024-09-10T22:01:35.619407+0000","last_unstale":"2024-09-10T22:02:35.897363+0000","last_undegraded":"2024-09-10T22:02:35.897363+0000","last_fullsized":"2024-09-10T22:02:35.897363+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:17.064585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726660+0000","last_change":"2024-09-10T22:00:30.818224+0000","last_active":"2024-09-10T22:02:35.726660+0000","last_peered":"2024-09-10T22:02:35.726660+0000","last_clean":"2024-09-10T22:02:35.726660+0000","last_became_active":"2024-09-10T22:00:30.817781+0000","last_became_peered":"2024-09-10T22:00:30.817781+0000","last_unstale":"2024-09-10T22:02:35.726660+0000","last_undegraded":"2024-09-10T22:02:35.726660+0000","last_fullsized":"2024-09-10T22:02:35.726660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:47.097642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851346+0000","last_change":"2024-09-10T22:02:35.949974+0000","last_active":"2024-09-10T22:03:03.851346+0000","last_peered":"2024-09-10T22:03:03.851346+0000","last_clean":"2024-09-10T22:03:03.851346+0000","last_became_active":"2024-09-10T22:02:35.949618+0000","last_became_peered":"2024-09-10T22:02:35.949618+0000","last_unstale":"2024-09-10T22:03:03.851346+0000","last_undegraded":"2024-09-10T22:03:03.851346+0000","last_fullsized":"2024-09-10T22:03:03.851346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:02.877650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897288+0000","last_change":"2024-09-10T22:00:43.712066+0000","last_active":"2024-09-10T22:02:35.897288+0000","last_peered":"2024-09-10T22:02:35.897288+0000","last_clean":"2024-09-10T22:02:35.897288+0000","last_became_active":"2024-09-10T22:00:43.711614+0000","last_became_peered":"2024-09-10T22:00:43.711614+0000","last_unstale":"2024-09-10T22:02:35.897288+0000","last_undegraded":"2024-09-10T22:02:35.897288+0000","last_fullsized":"2024-09-10T22:02:35.897288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:42.403367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.854262+0000","last_change":"2024-09-10T22:00:34.501943+0000","last_active":"2024-09-10T22:03:03.854262+0000","last_peered":"2024-09-10T22:03:03.854262+0000","last_clean":"2024-09-10T22:03:03.854262+0000","last_became_active":"2024-09-10T22:00:34.501602+0000","last_became_peered":"2024-09-10T22:00:34.501602+0000","last_unstale":"2024-09-10T22:03:03.854262+0000","last_undegraded":"2024-09-10T22:03:03.854262+0000","last_fullsized":"2024-09-10T22:03:03.854262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:53.200974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726593+0000","last_change":"2024-09-10T22:01:35.200791+0000","last_active":"2024-09-10T22:02:35.726593+0000","last_peered":"2024-09-10T22:02:35.726593+0000","last_clean":"2024-09-10T22:02:35.726593+0000","last_became_active":"2024-09-10T22:01:35.196059+0000","last_became_peered":"2024-09-10T22:01:35.196059+0000","last_unstale":"2024-09-10T22:02:35.726593+0000","last_undegraded":"2024-09-10T22:02:35.726593+0000","last_fullsized":"2024-09-10T22:02:35.726593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:51.349216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897010+0000","last_change":"2024-09-10T22:00:43.760567+0000","last_active":"2024-09-10T22:02:35.897010+0000","last_peered":"2024-09-10T22:02:35.897010+0000","last_clean":"2024-09-10T22:02:35.897010+0000","last_became_active":"2024-09-10T22:00:43.760064+0000","last_became_peered":"2024-09-10T22:00:43.760064+0000","last_unstale":"2024-09-10T22:02:35.897010+0000","last_undegraded":"2024-09-10T22:02:35.897010+0000","last_fullsized":"2024-09-10T22:02:35.897010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:55.469525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.843933+0000","last_change":"2024-09-10T22:00:34.486225+0000","last_active":"2024-09-10T22:03:03.843933+0000","last_peered":"2024-09-10T22:03:03.843933+0000","last_clean":"2024-09-10T22:03:03.843933+0000","last_became_active":"2024-09-10T22:00:34.485896+0000","last_became_peered":"2024-09-10T22:00:34.485896+0000","last_unstale":"2024-09-10T22:03:03.843933+0000","last_undegraded":"2024-09-10T22:03:03.843933+0000","last_fullsized":"2024-09-10T22:03:03.843933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:00.151927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:43.663404+0000","last_change":"2024-09-10T22:00:30.817958+0000","last_active":"2024-09-10T22:02:43.663404+0000","last_peered":"2024-09-10T22:02:43.663404+0000","last_clean":"2024-09-10T22:02:43.663404+0000","last_became_active":"2024-09-10T22:00:30.817646+0000","last_became_peered":"2024-09-10T22:00:30.817646+0000","last_unstale":"2024-09-10T22:02:43.663404+0000","last_undegraded":"2024-09-10T22:02:43.663404+0000","last_fullsized":"2024-09-10T22:02:43.663404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:27.722184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897658+0000","last_change":"2024-09-10T22:00:43.716465+0000","last_active":"2024-09-10T22:02:35.897658+0000","last_peered":"2024-09-10T22:02:35.897658+0000","last_clean":"2024-09-10T22:02:35.897658+0000","last_became_active":"2024-09-10T22:00:43.716165+0000","last_became_peered":"2024-09-10T22:00:43.716165+0000","last_unstale":"2024-09-10T22:02:35.897658+0000","last_undegraded":"2024-09-10T22:02:35.897658+0000","last_fullsized":"2024-09-10T22:02:35.897658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:18.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841664+0000","last_change":"2024-09-10T22:00:34.493992+0000","last_active":"2024-09-10T22:03:03.841664+0000","last_peered":"2024-09-10T22:03:03.841664+0000","last_clean":"2024-09-10T22:03:03.841664+0000","last_became_active":"2024-09-10T22:00:34.493670+0000","last_became_peered":"2024-09-10T22:00:34.493670+0000","last_unstale":"2024-09-10T22:03:03.841664+0000","last_undegraded":"2024-09-10T22:03:03.841664+0000","last_fullsized":"2024-09-10T22:03:03.841664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:04.991819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722649+0000","last_change":"2024-09-10T22:00:30.819830+0000","last_active":"2024-09-10T22:02:35.722649+0000","last_peered":"2024-09-10T22:02:35.722649+0000","last_clean":"2024-09-10T22:02:35.722649+0000","last_became_active":"2024-09-10T22:00:30.819401+0000","last_became_peered":"2024-09-10T22:00:30.819401+0000","last_unstale":"2024-09-10T22:02:35.722649+0000","last_undegraded":"2024-09-10T22:02:35.722649+0000","last_fullsized":"2024-09-10T22:02:35.722649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:37.823107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724263+0000","last_change":"2024-09-10T22:00:43.718500+0000","last_active":"2024-09-10T22:02:35.724263+0000","last_peered":"2024-09-10T22:02:35.724263+0000","last_clean":"2024-09-10T22:02:35.724263+0000","last_became_active":"2024-09-10T22:00:43.717667+0000","last_became_peered":"2024-09-10T22:00:43.717667+0000","last_unstale":"2024-09-10T22:02:35.724263+0000","last_undegraded":"2024-09-10T22:02:35.724263+0000","last_fullsized":"2024-09-10T22:02:35.724263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:23.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.792059+0000","last_change":"2024-09-10T22:00:34.494162+0000","last_active":"2024-09-10T22:03:03.792059+0000","last_peered":"2024-09-10T22:03:03.792059+0000","last_clean":"2024-09-10T22:03:03.792059+0000","last_became_active":"2024-09-10T22:00:34.493819+0000","last_became_peered":"2024-09-10T22:00:34.493819+0000","last_unstale":"2024-09-10T22:03:03.792059+0000","last_undegraded":"2024-09-10T22:03:03.792059+0000","last_fullsized":"2024-09-10T22:03:03.792059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:43.264363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948452+0000","last_change":"2024-09-10T22:00:31.081559+0000","last_active":"2024-09-10T22:02:35.948452+0000","last_peered":"2024-09-10T22:02:35.948452+0000","last_clean":"2024-09-10T22:02:35.948452+0000","last_became_active":"2024-09-10T22:00:31.081257+0000","last_became_peered":"2024-09-10T22:00:31.081257+0000","last_unstale":"2024-09-10T22:02:35.948452+0000","last_undegraded":"2024-09-10T22:02:35.948452+0000","last_fullsized":"2024-09-10T22:02:35.948452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:53.235765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897682+0000","last_change":"2024-09-10T22:01:35.445171+0000","last_active":"2024-09-10T22:02:35.897682+0000","last_peered":"2024-09-10T22:02:35.897682+0000","last_clean":"2024-09-10T22:02:35.897682+0000","last_became_active":"2024-09-10T22:01:35.444651+0000","last_became_peered":"2024-09-10T22:01:35.444651+0000","last_unstale":"2024-09-10T22:02:35.897682+0000","last_undegraded":"2024-09-10T22:02:35.897682+0000","last_fullsized":"2024-09-10T22:02:35.897682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:36.457151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.847341+0000","last_change":"2024-09-10T22:01:35.750982+0000","last_active":"2024-09-10T22:03:03.847341+0000","last_peered":"2024-09-10T22:03:03.847341+0000","last_clean":"2024-09-10T22:03:03.847341+0000","last_became_active":"2024-09-10T22:01:35.750572+0000","last_became_peered":"2024-09-10T22:01:35.750572+0000","last_unstale":"2024-09-10T22:03:03.847341+0000","last_undegraded":"2024-09-10T22:03:03.847341+0000","last_fullsized":"2024-09-10T22:03:03.847341+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:36.596313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897938+0000","last_change":"2024-09-10T22:00:31.311864+0000","last_active":"2024-09-10T22:02:35.897938+0000","last_peered":"2024-09-10T22:02:35.897938+0000","last_clean":"2024-09-10T22:02:35.897938+0000","last_became_active":"2024-09-10T22:00:31.311611+0000","last_became_peered":"2024-09-10T22:00:31.311611+0000","last_unstale":"2024-09-10T22:02:35.897938+0000","last_undegraded":"2024-09-10T22:02:35.897938+0000","last_fullsized":"2024-09-10T22:02:35.897938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:27.969999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723479+0000","last_change":"2024-09-10T22:00:43.708023+0000","last_active":"2024-09-10T22:02:35.723479+0000","last_peered":"2024-09-10T22:02:35.723479+0000","last_clean":"2024-09-10T22:02:35.723479+0000","last_became_active":"2024-09-10T22:00:43.707086+0000","last_became_peered":"2024-09-10T22:00:43.707086+0000","last_unstale":"2024-09-10T22:02:35.723479+0000","last_undegraded":"2024-09-10T22:02:35.723479+0000","last_fullsized":"2024-09-10T22:02:35.723479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:15.585104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.846819+0000","last_change":"2024-09-10T22:00:34.494104+0000","last_active":"2024-09-10T22:03:03.846819+0000","last_peered":"2024-09-10T22:03:03.846819+0000","last_clean":"2024-09-10T22:03:03.846819+0000","last_became_active":"2024-09-10T22:00:34.493404+0000","last_became_peered":"2024-09-10T22:00:34.493404+0000","last_unstale":"2024-09-10T22:03:03.846819+0000","last_undegraded":"2024-09-10T22:03:03.846819+0000","last_fullsized":"2024-09-10T22:03:03.846819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:47.444100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'72","reported_seq":167,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:48.425868+0000","last_change":"2024-09-10T22:00:30.819427+0000","last_active":"2024-09-10T22:03:48.425868+0000","last_peered":"2024-09-10T22:03:48.425868+0000","last_clean":"2024-09-10T22:03:48.425868+0000","last_became_active":"2024-09-10T22:00:30.819098+0000","last_became_peered":"2024-09-10T22:00:30.819098+0000","last_unstale":"2024-09-10T22:03:48.425868+0000","last_undegraded":"2024-09-10T22:03:48.425868+0000","last_fullsized":"2024-09-10T22:03:48.425868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:23.068155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":172,"num_read_kb":151,"num_write":130,"num_write_kb":1094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722303+0000","last_change":"2024-09-10T22:00:43.760602+0000","last_active":"2024-09-10T22:02:35.722303+0000","last_peered":"2024-09-10T22:02:35.722303+0000","last_clean":"2024-09-10T22:02:35.722303+0000","last_became_active":"2024-09-10T22:00:43.760172+0000","last_became_peered":"2024-09-10T22:00:43.760172+0000","last_unstale":"2024-09-10T22:02:35.722303+0000","last_undegraded":"2024-09-10T22:02:35.722303+0000","last_fullsized":"2024-09-10T22:02:35.722303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:40.332101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.836073+0000","last_change":"2024-09-10T22:00:34.501632+0000","last_active":"2024-09-10T22:03:03.836073+0000","last_peered":"2024-09-10T22:03:03.836073+0000","last_clean":"2024-09-10T22:03:03.836073+0000","last_became_active":"2024-09-10T22:00:34.500924+0000","last_became_peered":"2024-09-10T22:00:34.500924+0000","last_unstale":"2024-09-10T22:03:03.836073+0000","last_undegraded":"2024-09-10T22:03:03.836073+0000","last_fullsized":"2024-09-10T22:03:03.836073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:28.052570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726206+0000","last_change":"2024-09-10T22:00:31.081391+0000","last_active":"2024-09-10T22:02:35.726206+0000","last_peered":"2024-09-10T22:02:35.726206+0000","last_clean":"2024-09-10T22:02:35.726206+0000","last_became_active":"2024-09-10T22:00:31.081018+0000","last_became_peered":"2024-09-10T22:00:31.081018+0000","last_unstale":"2024-09-10T22:02:35.726206+0000","last_undegraded":"2024-09-10T22:02:35.726206+0000","last_fullsized":"2024-09-10T22:02:35.726206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:09.450785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948812+0000","last_change":"2024-09-10T22:00:43.711592+0000","last_active":"2024-09-10T22:02:35.948812+0000","last_peered":"2024-09-10T22:02:35.948812+0000","last_clean":"2024-09-10T22:02:35.948812+0000","last_became_active":"2024-09-10T22:00:43.711144+0000","last_became_peered":"2024-09-10T22:00:43.711144+0000","last_unstale":"2024-09-10T22:02:35.948812+0000","last_undegraded":"2024-09-10T22:02:35.948812+0000","last_fullsized":"2024-09-10T22:02:35.948812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:01.818907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840894+0000","last_change":"2024-09-10T22:00:34.494070+0000","last_active":"2024-09-10T22:03:03.840894+0000","last_peered":"2024-09-10T22:03:03.840894+0000","last_clean":"2024-09-10T22:03:03.840894+0000","last_became_active":"2024-09-10T22:00:34.493324+0000","last_became_peered":"2024-09-10T22:00:34.493324+0000","last_unstale":"2024-09-10T22:03:03.840894+0000","last_undegraded":"2024-09-10T22:03:03.840894+0000","last_fullsized":"2024-09-10T22:03:03.840894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:29.969743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723888+0000","last_change":"2024-09-10T22:00:31.082439+0000","last_active":"2024-09-10T22:02:35.723888+0000","last_peered":"2024-09-10T22:02:35.723888+0000","last_clean":"2024-09-10T22:02:35.723888+0000","last_became_active":"2024-09-10T22:00:31.082025+0000","last_became_peered":"2024-09-10T22:00:31.082025+0000","last_unstale":"2024-09-10T22:02:35.723888+0000","last_undegraded":"2024-09-10T22:02:35.723888+0000","last_fullsized":"2024-09-10T22:02:35.723888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:21.933419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726116+0000","last_change":"2024-09-10T22:00:43.760997+0000","last_active":"2024-09-10T22:02:35.726116+0000","last_peered":"2024-09-10T22:02:35.726116+0000","last_clean":"2024-09-10T22:02:35.726116+0000","last_became_active":"2024-09-10T22:00:43.760124+0000","last_became_peered":"2024-09-10T22:00:43.760124+0000","last_unstale":"2024-09-10T22:02:35.726116+0000","last_undegraded":"2024-09-10T22:02:35.726116+0000","last_fullsized":"2024-09-10T22:02:35.726116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:29.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.843650+0000","last_change":"2024-09-10T22:00:34.511730+0000","last_active":"2024-09-10T22:03:03.843650+0000","last_peered":"2024-09-10T22:03:03.843650+0000","last_clean":"2024-09-10T22:03:03.843650+0000","last_became_active":"2024-09-10T22:00:34.511440+0000","last_became_peered":"2024-09-10T22:00:34.511440+0000","last_unstale":"2024-09-10T22:03:03.843650+0000","last_undegraded":"2024-09-10T22:03:03.843650+0000","last_fullsized":"2024-09-10T22:03:03.843650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:09.053835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'8","reported_seq":194,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:47.981333+0000","last_change":"2024-09-10T22:00:30.825999+0000","last_active":"2024-09-10T22:03:47.981333+0000","last_peered":"2024-09-10T22:03:47.981333+0000","last_clean":"2024-09-10T22:03:47.981333+0000","last_became_active":"2024-09-10T22:00:30.825334+0000","last_became_peered":"2024-09-10T22:00:30.825334+0000","last_unstale":"2024-09-10T22:03:47.981333+0000","last_undegraded":"2024-09-10T22:03:47.981333+0000","last_fullsized":"2024-09-10T22:03:47.981333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:14.684222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":141,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894213+0000","last_change":"2024-09-10T22:00:43.710517+0000","last_active":"2024-09-10T22:02:35.894213+0000","last_peered":"2024-09-10T22:02:35.894213+0000","last_clean":"2024-09-10T22:02:35.894213+0000","last_became_active":"2024-09-10T22:00:43.709252+0000","last_became_peered":"2024-09-10T22:00:43.709252+0000","last_unstale":"2024-09-10T22:02:35.894213+0000","last_undegraded":"2024-09-10T22:02:35.894213+0000","last_fullsized":"2024-09-10T22:02:35.894213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:21.355152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838763+0000","last_change":"2024-09-10T22:02:35.898878+0000","last_active":"2024-09-10T22:03:03.838763+0000","last_peered":"2024-09-10T22:03:03.838763+0000","last_clean":"2024-09-10T22:03:03.838763+0000","last_became_active":"2024-09-10T22:02:35.898531+0000","last_became_peered":"2024-09-10T22:02:35.898531+0000","last_unstale":"2024-09-10T22:03:03.838763+0000","last_undegraded":"2024-09-10T22:03:03.838763+0000","last_fullsized":"2024-09-10T22:03:03.838763+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:02.735208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":182,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:43.027365+0000","last_change":"2024-09-10T22:00:31.081590+0000","last_active":"2024-09-10T22:03:43.027365+0000","last_peered":"2024-09-10T22:03:43.027365+0000","last_clean":"2024-09-10T22:03:43.027365+0000","last_became_active":"2024-09-10T22:00:31.081250+0000","last_became_peered":"2024-09-10T22:00:31.081250+0000","last_unstale":"2024-09-10T22:03:43.027365+0000","last_undegraded":"2024-09-10T22:03:43.027365+0000","last_fullsized":"2024-09-10T22:03:43.027365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:12.872832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893781+0000","last_change":"2024-09-10T22:00:43.761351+0000","last_active":"2024-09-10T22:02:35.893781+0000","last_peered":"2024-09-10T22:02:35.893781+0000","last_clean":"2024-09-10T22:02:35.893781+0000","last_became_active":"2024-09-10T22:00:43.760903+0000","last_became_peered":"2024-09-10T22:00:43.760903+0000","last_unstale":"2024-09-10T22:02:35.893781+0000","last_undegraded":"2024-09-10T22:02:35.893781+0000","last_fullsized":"2024-09-10T22:02:35.893781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:14.113974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838302+0000","last_change":"2024-09-10T22:00:34.504671+0000","last_active":"2024-09-10T22:03:03.838302+0000","last_peered":"2024-09-10T22:03:03.838302+0000","last_clean":"2024-09-10T22:03:03.838302+0000","last_became_active":"2024-09-10T22:00:34.504368+0000","last_became_peered":"2024-09-10T22:00:34.504368+0000","last_unstale":"2024-09-10T22:03:03.838302+0000","last_undegraded":"2024-09-10T22:03:03.838302+0000","last_fullsized":"2024-09-10T22:03:03.838302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:59.771589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863056+0000","last_change":"2024-09-10T22:00:30.820401+0000","last_active":"2024-09-10T22:02:35.863056+0000","last_peered":"2024-09-10T22:02:35.863056+0000","last_clean":"2024-09-10T22:02:35.863056+0000","last_became_active":"2024-09-10T22:00:30.820194+0000","last_became_peered":"2024-09-10T22:00:30.820194+0000","last_unstale":"2024-09-10T22:02:35.863056+0000","last_undegraded":"2024-09-10T22:02:35.863056+0000","last_fullsized":"2024-09-10T22:02:35.863056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:47.767102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727473+0000","last_change":"2024-09-10T22:00:43.762001+0000","last_active":"2024-09-10T22:02:35.727473+0000","last_peered":"2024-09-10T22:02:35.727473+0000","last_clean":"2024-09-10T22:02:35.727473+0000","last_became_active":"2024-09-10T22:00:43.761323+0000","last_became_peered":"2024-09-10T22:00:43.761323+0000","last_unstale":"2024-09-10T22:02:35.727473+0000","last_undegraded":"2024-09-10T22:02:35.727473+0000","last_fullsized":"2024-09-10T22:02:35.727473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:50.113545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.847877+0000","last_change":"2024-09-10T22:00:34.498591+0000","last_active":"2024-09-10T22:03:03.847877+0000","last_peered":"2024-09-10T22:03:03.847877+0000","last_clean":"2024-09-10T22:03:03.847877+0000","last_became_active":"2024-09-10T22:00:34.496449+0000","last_became_peered":"2024-09-10T22:00:34.496449+0000","last_unstale":"2024-09-10T22:03:03.847877+0000","last_undegraded":"2024-09-10T22:03:03.847877+0000","last_fullsized":"2024-09-10T22:03:03.847877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:54.815268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897905+0000","last_change":"2024-09-10T22:00:30.817201+0000","last_active":"2024-09-10T22:02:35.897905+0000","last_peered":"2024-09-10T22:02:35.897905+0000","last_clean":"2024-09-10T22:02:35.897905+0000","last_became_active":"2024-09-10T22:00:30.816792+0000","last_became_peered":"2024-09-10T22:00:30.816792+0000","last_unstale":"2024-09-10T22:02:35.897905+0000","last_undegraded":"2024-09-10T22:02:35.897905+0000","last_fullsized":"2024-09-10T22:02:35.897905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:48.239811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863189+0000","last_change":"2024-09-10T21:59:01.830452+0000","last_active":"2024-09-10T22:02:35.863189+0000","last_peered":"2024-09-10T22:02:35.863189+0000","last_clean":"2024-09-10T22:02:35.863189+0000","last_became_active":"2024-09-10T21:59:01.823744+0000","last_became_peered":"2024-09-10T21:59:01.823744+0000","last_unstale":"2024-09-10T22:02:35.863189+0000","last_undegraded":"2024-09-10T22:02:35.863189+0000","last_fullsized":"2024-09-10T22:02:35.863189+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_clean_scrub_stamp":"2024-09-10T21:54:48.621299+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:39.573257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863562+0000","last_change":"2024-09-10T22:00:43.713888+0000","last_active":"2024-09-10T22:02:35.863562+0000","last_peered":"2024-09-10T22:02:35.863562+0000","last_clean":"2024-09-10T22:02:35.863562+0000","last_became_active":"2024-09-10T22:00:43.713329+0000","last_became_peered":"2024-09-10T22:00:43.713329+0000","last_unstale":"2024-09-10T22:02:35.863562+0000","last_undegraded":"2024-09-10T22:02:35.863562+0000","last_fullsized":"2024-09-10T22:02:35.863562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:01.062089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.852867+0000","last_change":"2024-09-10T22:00:34.502589+0000","last_active":"2024-09-10T22:03:03.852867+0000","last_peered":"2024-09-10T22:03:03.852867+0000","last_clean":"2024-09-10T22:03:03.852867+0000","last_became_active":"2024-09-10T22:00:34.502216+0000","last_became_peered":"2024-09-10T22:00:34.502216+0000","last_unstale":"2024-09-10T22:03:03.852867+0000","last_undegraded":"2024-09-10T22:03:03.852867+0000","last_fullsized":"2024-09-10T22:03:03.852867+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:44.959375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":193,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:47.001644+0000","last_change":"2024-09-10T22:00:31.311807+0000","last_active":"2024-09-10T22:03:47.001644+0000","last_peered":"2024-09-10T22:03:47.001644+0000","last_clean":"2024-09-10T22:03:47.001644+0000","last_became_active":"2024-09-10T22:00:31.311424+0000","last_became_peered":"2024-09-10T22:00:31.311424+0000","last_unstale":"2024-09-10T22:03:47.001644+0000","last_undegraded":"2024-09-10T22:03:47.001644+0000","last_fullsized":"2024-09-10T22:03:47.001644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:09.754777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2195,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948273+0000","last_change":"2024-09-10T22:00:43.711882+0000","last_active":"2024-09-10T22:02:35.948273+0000","last_peered":"2024-09-10T22:02:35.948273+0000","last_clean":"2024-09-10T22:02:35.948273+0000","last_became_active":"2024-09-10T22:00:43.711408+0000","last_became_peered":"2024-09-10T22:00:43.711408+0000","last_unstale":"2024-09-10T22:02:35.948273+0000","last_undegraded":"2024-09-10T22:02:35.948273+0000","last_fullsized":"2024-09-10T22:02:35.948273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:31.155723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841900+0000","last_change":"2024-09-10T22:00:34.505653+0000","last_active":"2024-09-10T22:03:03.841900+0000","last_peered":"2024-09-10T22:03:03.841900+0000","last_clean":"2024-09-10T22:03:03.841900+0000","last_became_active":"2024-09-10T22:00:34.505317+0000","last_became_peered":"2024-09-10T22:00:34.505317+0000","last_unstale":"2024-09-10T22:03:03.841900+0000","last_undegraded":"2024-09-10T22:03:03.841900+0000","last_fullsized":"2024-09-10T22:03:03.841900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:35.050892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727968+0000","last_change":"2024-09-10T22:00:30.817431+0000","last_active":"2024-09-10T22:02:35.727968+0000","last_peered":"2024-09-10T22:02:35.727968+0000","last_clean":"2024-09-10T22:02:35.727968+0000","last_became_active":"2024-09-10T22:00:30.816952+0000","last_became_peered":"2024-09-10T22:00:30.816952+0000","last_unstale":"2024-09-10T22:02:35.727968+0000","last_undegraded":"2024-09-10T22:02:35.727968+0000","last_fullsized":"2024-09-10T22:02:35.727968+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:37.881641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947502+0000","last_change":"2024-09-10T22:00:43.720627+0000","last_active":"2024-09-10T22:02:35.947502+0000","last_peered":"2024-09-10T22:02:35.947502+0000","last_clean":"2024-09-10T22:02:35.947502+0000","last_became_active":"2024-09-10T22:00:43.720235+0000","last_became_peered":"2024-09-10T22:00:43.720235+0000","last_unstale":"2024-09-10T22:02:35.947502+0000","last_undegraded":"2024-09-10T22:02:35.947502+0000","last_fullsized":"2024-09-10T22:02:35.947502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:50.549536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841647+0000","last_change":"2024-09-10T22:00:34.501649+0000","last_active":"2024-09-10T22:03:03.841647+0000","last_peered":"2024-09-10T22:03:03.841647+0000","last_clean":"2024-09-10T22:03:03.841647+0000","last_became_active":"2024-09-10T22:00:34.498882+0000","last_became_peered":"2024-09-10T22:00:34.498882+0000","last_unstale":"2024-09-10T22:03:03.841647+0000","last_undegraded":"2024-09-10T22:03:03.841647+0000","last_fullsized":"2024-09-10T22:03:03.841647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894508+0000","last_change":"2024-09-10T22:00:30.821943+0000","last_active":"2024-09-10T22:02:35.894508+0000","last_peered":"2024-09-10T22:02:35.894508+0000","last_clean":"2024-09-10T22:02:35.894508+0000","last_became_active":"2024-09-10T22:00:30.821372+0000","last_became_peered":"2024-09-10T22:00:30.821372+0000","last_unstale":"2024-09-10T22:02:35.894508+0000","last_undegraded":"2024-09-10T22:02:35.894508+0000","last_fullsized":"2024-09-10T22:02:35.894508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:04.360383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727340+0000","last_change":"2024-09-10T22:00:43.710045+0000","last_active":"2024-09-10T22:02:35.727340+0000","last_peered":"2024-09-10T22:02:35.727340+0000","last_clean":"2024-09-10T22:02:35.727340+0000","last_became_active":"2024-09-10T22:00:43.709416+0000","last_became_peered":"2024-09-10T22:00:43.709416+0000","last_unstale":"2024-09-10T22:02:35.727340+0000","last_undegraded":"2024-09-10T22:02:35.727340+0000","last_fullsized":"2024-09-10T22:02:35.727340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:56.005549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.834717+0000","last_change":"2024-09-10T22:00:34.496163+0000","last_active":"2024-09-10T22:03:03.834717+0000","last_peered":"2024-09-10T22:03:03.834717+0000","last_clean":"2024-09-10T22:03:03.834717+0000","last_became_active":"2024-09-10T22:00:34.495814+0000","last_became_peered":"2024-09-10T22:00:34.495814+0000","last_unstale":"2024-09-10T22:03:03.834717+0000","last_undegraded":"2024-09-10T22:03:03.834717+0000","last_fullsized":"2024-09-10T22:03:03.834717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:05.398154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726434+0000","last_change":"2024-09-10T22:00:30.812702+0000","last_active":"2024-09-10T22:02:35.726434+0000","last_peered":"2024-09-10T22:02:35.726434+0000","last_clean":"2024-09-10T22:02:35.726434+0000","last_became_active":"2024-09-10T22:00:30.812369+0000","last_became_peered":"2024-09-10T22:00:30.812369+0000","last_unstale":"2024-09-10T22:02:35.726434+0000","last_undegraded":"2024-09-10T22:02:35.726434+0000","last_fullsized":"2024-09-10T22:02:35.726434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:52.332876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727291+0000","last_change":"2024-09-10T22:00:43.715573+0000","last_active":"2024-09-10T22:02:35.727291+0000","last_peered":"2024-09-10T22:02:35.727291+0000","last_clean":"2024-09-10T22:02:35.727291+0000","last_became_active":"2024-09-10T22:00:43.714475+0000","last_became_peered":"2024-09-10T22:00:43.714475+0000","last_unstale":"2024-09-10T22:02:35.727291+0000","last_undegraded":"2024-09-10T22:02:35.727291+0000","last_fullsized":"2024-09-10T22:02:35.727291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:55.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.844868+0000","last_change":"2024-09-10T22:02:35.949769+0000","last_active":"2024-09-10T22:03:03.844868+0000","last_peered":"2024-09-10T22:03:03.844868+0000","last_clean":"2024-09-10T22:03:03.844868+0000","last_became_active":"2024-09-10T22:02:35.949440+0000","last_became_peered":"2024-09-10T22:02:35.949440+0000","last_unstale":"2024-09-10T22:03:03.844868+0000","last_undegraded":"2024-09-10T22:03:03.844868+0000","last_fullsized":"2024-09-10T22:03:03.844868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:03.639412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948309+0000","last_change":"2024-09-10T22:01:35.903381+0000","last_active":"2024-09-10T22:02:35.948309+0000","last_peered":"2024-09-10T22:02:35.948309+0000","last_clean":"2024-09-10T22:02:35.948309+0000","last_became_active":"2024-09-10T22:01:35.878262+0000","last_became_peered":"2024-09-10T22:01:35.878262+0000","last_unstale":"2024-09-10T22:02:35.948309+0000","last_undegraded":"2024-09-10T22:02:35.948309+0000","last_fullsized":"2024-09-10T22:02:35.948309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:45.419449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893771+0000","last_change":"2024-09-10T22:00:43.720728+0000","last_active":"2024-09-10T22:02:35.893771+0000","last_peered":"2024-09-10T22:02:35.893771+0000","last_clean":"2024-09-10T22:02:35.893771+0000","last_became_active":"2024-09-10T22:00:43.720305+0000","last_became_peered":"2024-09-10T22:00:43.720305+0000","last_unstale":"2024-09-10T22:02:35.893771+0000","last_undegraded":"2024-09-10T22:02:35.893771+0000","last_fullsized":"2024-09-10T22:02:35.893771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:54.986822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841861+0000","last_change":"2024-09-10T22:00:34.501571+0000","last_active":"2024-09-10T22:03:03.841861+0000","last_peered":"2024-09-10T22:03:03.841861+0000","last_clean":"2024-09-10T22:03:03.841861+0000","last_became_active":"2024-09-10T22:00:34.501211+0000","last_became_peered":"2024-09-10T22:00:34.501211+0000","last_unstale":"2024-09-10T22:03:03.841861+0000","last_undegraded":"2024-09-10T22:03:03.841861+0000","last_fullsized":"2024-09-10T22:03:03.841861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:05.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":194,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:46.034697+0000","last_change":"2024-09-10T22:00:30.824296+0000","last_active":"2024-09-10T22:03:46.034697+0000","last_peered":"2024-09-10T22:03:46.034697+0000","last_clean":"2024-09-10T22:03:46.034697+0000","last_became_active":"2024-09-10T22:00:30.823886+0000","last_became_peered":"2024-09-10T22:00:30.823886+0000","last_unstale":"2024-09-10T22:03:46.034697+0000","last_undegraded":"2024-09-10T22:03:46.034697+0000","last_fullsized":"2024-09-10T22:03:46.034697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:26.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724371+0000","last_change":"2024-09-10T22:00:43.717580+0000","last_active":"2024-09-10T22:02:35.724371+0000","last_peered":"2024-09-10T22:02:35.724371+0000","last_clean":"2024-09-10T22:02:35.724371+0000","last_became_active":"2024-09-10T22:00:43.708377+0000","last_became_peered":"2024-09-10T22:00:43.708377+0000","last_unstale":"2024-09-10T22:02:35.724371+0000","last_undegraded":"2024-09-10T22:02:35.724371+0000","last_fullsized":"2024-09-10T22:02:35.724371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:54.682292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.848412+0000","last_change":"2024-09-10T22:00:34.514348+0000","last_active":"2024-09-10T22:03:03.848412+0000","last_peered":"2024-09-10T22:03:03.848412+0000","last_clean":"2024-09-10T22:03:03.848412+0000","last_became_active":"2024-09-10T22:00:34.513797+0000","last_became_peered":"2024-09-10T22:00:34.513797+0000","last_unstale":"2024-09-10T22:03:03.848412+0000","last_undegraded":"2024-09-10T22:03:03.848412+0000","last_fullsized":"2024-09-10T22:03:03.848412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:12.797643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728093+0000","last_change":"2024-09-10T22:00:30.826486+0000","last_active":"2024-09-10T22:02:35.728093+0000","last_peered":"2024-09-10T22:02:35.728093+0000","last_clean":"2024-09-10T22:02:35.728093+0000","last_became_active":"2024-09-10T22:00:30.826111+0000","last_became_peered":"2024-09-10T22:00:30.826111+0000","last_unstale":"2024-09-10T22:02:35.728093+0000","last_undegraded":"2024-09-10T22:02:35.728093+0000","last_fullsized":"2024-09-10T22:02:35.728093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:45.609925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948482+0000","last_change":"2024-09-10T22:00:43.709432+0000","last_active":"2024-09-10T22:02:35.948482+0000","last_peered":"2024-09-10T22:02:35.948482+0000","last_clean":"2024-09-10T22:02:35.948482+0000","last_became_active":"2024-09-10T22:00:43.708627+0000","last_became_peered":"2024-09-10T22:00:43.708627+0000","last_unstale":"2024-09-10T22:02:35.948482+0000","last_undegraded":"2024-09-10T22:02:35.948482+0000","last_fullsized":"2024-09-10T22:02:35.948482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:12.799976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.848267+0000","last_change":"2024-09-10T22:00:34.499203+0000","last_active":"2024-09-10T22:03:03.848267+0000","last_peered":"2024-09-10T22:03:03.848267+0000","last_clean":"2024-09-10T22:03:03.848267+0000","last_became_active":"2024-09-10T22:00:34.497877+0000","last_became_peered":"2024-09-10T22:00:34.497877+0000","last_unstale":"2024-09-10T22:03:03.848267+0000","last_undegraded":"2024-09-10T22:03:03.848267+0000","last_fullsized":"2024-09-10T22:03:03.848267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:41.897603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723973+0000","last_change":"2024-09-10T22:00:30.820852+0000","last_active":"2024-09-10T22:02:35.723973+0000","last_peered":"2024-09-10T22:02:35.723973+0000","last_clean":"2024-09-10T22:02:35.723973+0000","last_became_active":"2024-09-10T22:00:30.820023+0000","last_became_peered":"2024-09-10T22:00:30.820023+0000","last_unstale":"2024-09-10T22:02:35.723973+0000","last_undegraded":"2024-09-10T22:02:35.723973+0000","last_fullsized":"2024-09-10T22:02:35.723973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:54.245142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727813+0000","last_change":"2024-09-10T22:00:43.716323+0000","last_active":"2024-09-10T22:02:35.727813+0000","last_peered":"2024-09-10T22:02:35.727813+0000","last_clean":"2024-09-10T22:02:35.727813+0000","last_became_active":"2024-09-10T22:00:43.715827+0000","last_became_peered":"2024-09-10T22:00:43.715827+0000","last_unstale":"2024-09-10T22:02:35.727813+0000","last_undegraded":"2024-09-10T22:02:35.727813+0000","last_fullsized":"2024-09-10T22:02:35.727813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:10.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.839342+0000","last_change":"2024-09-10T22:00:34.500114+0000","last_active":"2024-09-10T22:03:03.839342+0000","last_peered":"2024-09-10T22:03:03.839342+0000","last_clean":"2024-09-10T22:03:03.839342+0000","last_became_active":"2024-09-10T22:00:34.499334+0000","last_became_peered":"2024-09-10T22:00:34.499334+0000","last_unstale":"2024-09-10T22:03:03.839342+0000","last_undegraded":"2024-09-10T22:03:03.839342+0000","last_fullsized":"2024-09-10T22:03:03.839342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:11.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":354,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:43.995239+0000","last_change":"2024-09-10T22:00:30.822561+0000","last_active":"2024-09-10T22:03:43.995239+0000","last_peered":"2024-09-10T22:03:43.995239+0000","last_clean":"2024-09-10T22:03:43.995239+0000","last_became_active":"2024-09-10T22:00:30.822201+0000","last_became_peered":"2024-09-10T22:00:30.822201+0000","last_unstale":"2024-09-10T22:03:43.995239+0000","last_undegraded":"2024-09-10T22:03:43.995239+0000","last_fullsized":"2024-09-10T22:03:43.995239+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:41.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62906,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":273,"num_read_kb":198,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871420+0000","last_change":"2024-09-10T22:01:35.444988+0000","last_active":"2024-09-10T22:02:35.871420+0000","last_peered":"2024-09-10T22:02:35.871420+0000","last_clean":"2024-09-10T22:02:35.871420+0000","last_became_active":"2024-09-10T22:01:35.444498+0000","last_became_peered":"2024-09-10T22:01:35.444498+0000","last_unstale":"2024-09-10T22:02:35.871420+0000","last_undegraded":"2024-09-10T22:02:35.871420+0000","last_fullsized":"2024-09-10T22:02:35.871420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:35.017660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.852407+0000","last_change":"2024-09-10T22:00:34.503234+0000","last_active":"2024-09-10T22:03:03.852407+0000","last_peered":"2024-09-10T22:03:03.852407+0000","last_clean":"2024-09-10T22:03:03.852407+0000","last_became_active":"2024-09-10T22:00:34.501501+0000","last_became_peered":"2024-09-10T22:00:34.501501+0000","last_unstale":"2024-09-10T22:03:03.852407+0000","last_undegraded":"2024-09-10T22:03:03.852407+0000","last_fullsized":"2024-09-10T22:03:03.852407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:23.772491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728102+0000","last_change":"2024-09-10T22:00:31.082131+0000","last_active":"2024-09-10T22:02:35.728102+0000","last_peered":"2024-09-10T22:02:35.728102+0000","last_clean":"2024-09-10T22:02:35.728102+0000","last_became_active":"2024-09-10T22:00:31.081405+0000","last_became_peered":"2024-09-10T22:00:31.081405+0000","last_unstale":"2024-09-10T22:02:35.728102+0000","last_undegraded":"2024-09-10T22:02:35.728102+0000","last_fullsized":"2024-09-10T22:02:35.728102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:02.400004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897415+0000","last_change":"2024-09-10T22:00:43.705732+0000","last_active":"2024-09-10T22:02:35.897415+0000","last_peered":"2024-09-10T22:02:35.897415+0000","last_clean":"2024-09-10T22:02:35.897415+0000","last_became_active":"2024-09-10T22:00:43.705274+0000","last_became_peered":"2024-09-10T22:00:43.705274+0000","last_unstale":"2024-09-10T22:02:35.897415+0000","last_undegraded":"2024-09-10T22:02:35.897415+0000","last_fullsized":"2024-09-10T22:02:35.897415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:54.707636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.845781+0000","last_change":"2024-09-10T22:00:34.499585+0000","last_active":"2024-09-10T22:03:03.845781+0000","last_peered":"2024-09-10T22:03:03.845781+0000","last_clean":"2024-09-10T22:03:03.845781+0000","last_became_active":"2024-09-10T22:00:34.499261+0000","last_became_peered":"2024-09-10T22:00:34.499261+0000","last_unstale":"2024-09-10T22:03:03.845781+0000","last_undegraded":"2024-09-10T22:03:03.845781+0000","last_fullsized":"2024-09-10T22:03:03.845781+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:41.156093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863703+0000","last_change":"2024-09-10T22:00:30.823322+0000","last_active":"2024-09-10T22:02:35.863703+0000","last_peered":"2024-09-10T22:02:35.863703+0000","last_clean":"2024-09-10T22:02:35.863703+0000","last_became_active":"2024-09-10T22:00:30.822987+0000","last_became_peered":"2024-09-10T22:00:30.822987+0000","last_unstale":"2024-09-10T22:02:35.863703+0000","last_undegraded":"2024-09-10T22:02:35.863703+0000","last_fullsized":"2024-09-10T22:02:35.863703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:33.605627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723255+0000","last_change":"2024-09-10T22:00:43.708746+0000","last_active":"2024-09-10T22:02:35.723255+0000","last_peered":"2024-09-10T22:02:35.723255+0000","last_clean":"2024-09-10T22:02:35.723255+0000","last_became_active":"2024-09-10T22:00:43.708283+0000","last_became_peered":"2024-09-10T22:00:43.708283+0000","last_unstale":"2024-09-10T22:02:35.723255+0000","last_undegraded":"2024-09-10T22:02:35.723255+0000","last_fullsized":"2024-09-10T22:02:35.723255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:30.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.854268+0000","last_change":"2024-09-10T22:00:34.501472+0000","last_active":"2024-09-10T22:03:03.854268+0000","last_peered":"2024-09-10T22:03:03.854268+0000","last_clean":"2024-09-10T22:03:03.854268+0000","last_became_active":"2024-09-10T22:00:34.501131+0000","last_became_peered":"2024-09-10T22:00:34.501131+0000","last_unstale":"2024-09-10T22:03:03.854268+0000","last_undegraded":"2024-09-10T22:03:03.854268+0000","last_fullsized":"2024-09-10T22:03:03.854268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:11.281413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":314,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:43.994905+0000","last_change":"2024-09-10T22:00:30.828335+0000","last_active":"2024-09-10T22:03:43.994905+0000","last_peered":"2024-09-10T22:03:43.994905+0000","last_clean":"2024-09-10T22:03:43.994905+0000","last_became_active":"2024-09-10T22:00:30.828028+0000","last_became_peered":"2024-09-10T22:00:30.828028+0000","last_unstale":"2024-09-10T22:03:43.994905+0000","last_undegraded":"2024-09-10T22:03:43.994905+0000","last_fullsized":"2024-09-10T22:03:43.994905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:56.399002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":264,"num_read_kb":264,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722469+0000","last_change":"2024-09-10T22:00:43.710086+0000","last_active":"2024-09-10T22:02:35.722469+0000","last_peered":"2024-09-10T22:02:35.722469+0000","last_clean":"2024-09-10T22:02:35.722469+0000","last_became_active":"2024-09-10T22:00:43.709518+0000","last_became_peered":"2024-09-10T22:00:43.709518+0000","last_unstale":"2024-09-10T22:02:35.722469+0000","last_undegraded":"2024-09-10T22:02:35.722469+0000","last_fullsized":"2024-09-10T22:02:35.722469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:07.636887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947965+0000","last_change":"2024-09-10T22:00:43.711620+0000","last_active":"2024-09-10T22:02:35.947965+0000","last_peered":"2024-09-10T22:02:35.947965+0000","last_clean":"2024-09-10T22:02:35.947965+0000","last_became_active":"2024-09-10T22:00:43.711224+0000","last_became_peered":"2024-09-10T22:00:43.711224+0000","last_unstale":"2024-09-10T22:02:35.947965+0000","last_undegraded":"2024-09-10T22:02:35.947965+0000","last_fullsized":"2024-09-10T22:02:35.947965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:43.860286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841267+0000","last_change":"2024-09-10T22:00:34.494408+0000","last_active":"2024-09-10T22:03:03.841267+0000","last_peered":"2024-09-10T22:03:03.841267+0000","last_clean":"2024-09-10T22:03:03.841267+0000","last_became_active":"2024-09-10T22:00:34.494066+0000","last_became_peered":"2024-09-10T22:00:34.494066+0000","last_unstale":"2024-09-10T22:03:03.841267+0000","last_undegraded":"2024-09-10T22:03:03.841267+0000","last_fullsized":"2024-09-10T22:03:03.841267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:49.565755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727709+0000","last_change":"2024-09-10T22:00:30.821842+0000","last_active":"2024-09-10T22:02:35.727709+0000","last_peered":"2024-09-10T22:02:35.727709+0000","last_clean":"2024-09-10T22:02:35.727709+0000","last_became_active":"2024-09-10T22:00:30.821499+0000","last_became_peered":"2024-09-10T22:00:30.821499+0000","last_unstale":"2024-09-10T22:02:35.727709+0000","last_undegraded":"2024-09-10T22:02:35.727709+0000","last_fullsized":"2024-09-10T22:02:35.727709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:19.125052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948928+0000","last_change":"2024-09-10T22:00:43.706689+0000","last_active":"2024-09-10T22:02:35.948928+0000","last_peered":"2024-09-10T22:02:35.948928+0000","last_clean":"2024-09-10T22:02:35.948928+0000","last_became_active":"2024-09-10T22:00:43.705954+0000","last_became_peered":"2024-09-10T22:00:43.705954+0000","last_unstale":"2024-09-10T22:02:35.948928+0000","last_undegraded":"2024-09-10T22:02:35.948928+0000","last_fullsized":"2024-09-10T22:02:35.948928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:30.800710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.831092+0000","last_change":"2024-09-10T22:00:34.490853+0000","last_active":"2024-09-10T22:03:03.831092+0000","last_peered":"2024-09-10T22:03:03.831092+0000","last_clean":"2024-09-10T22:03:03.831092+0000","last_became_active":"2024-09-10T22:00:34.489908+0000","last_became_peered":"2024-09-10T22:00:34.489908+0000","last_unstale":"2024-09-10T22:03:03.831092+0000","last_undegraded":"2024-09-10T22:03:03.831092+0000","last_fullsized":"2024-09-10T22:03:03.831092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:57.105004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893936+0000","last_change":"2024-09-10T22:00:30.816497+0000","last_active":"2024-09-10T22:02:35.893936+0000","last_peered":"2024-09-10T22:02:35.893936+0000","last_clean":"2024-09-10T22:02:35.893936+0000","last_became_active":"2024-09-10T22:00:30.816181+0000","last_became_peered":"2024-09-10T22:00:30.816181+0000","last_unstale":"2024-09-10T22:02:35.893936+0000","last_undegraded":"2024-09-10T22:02:35.893936+0000","last_fullsized":"2024-09-10T22:02:35.893936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:12.779113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871850+0000","last_change":"2024-09-10T22:00:30.821402+0000","last_active":"2024-09-10T22:02:35.871850+0000","last_peered":"2024-09-10T22:02:35.871850+0000","last_clean":"2024-09-10T22:02:35.871850+0000","last_became_active":"2024-09-10T22:00:30.821105+0000","last_became_peered":"2024-09-10T22:00:30.821105+0000","last_unstale":"2024-09-10T22:02:35.871850+0000","last_undegraded":"2024-09-10T22:02:35.871850+0000","last_fullsized":"2024-09-10T22:02:35.871850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:36.090579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840274+0000","last_change":"2024-09-10T22:00:34.511911+0000","last_active":"2024-09-10T22:03:03.840274+0000","last_peered":"2024-09-10T22:03:03.840274+0000","last_clean":"2024-09-10T22:03:03.840274+0000","last_became_active":"2024-09-10T22:00:34.511545+0000","last_became_peered":"2024-09-10T22:00:34.511545+0000","last_unstale":"2024-09-10T22:03:03.840274+0000","last_undegraded":"2024-09-10T22:03:03.840274+0000","last_fullsized":"2024-09-10T22:03:03.840274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:12.484733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723568+0000","last_change":"2024-09-10T22:01:35.750458+0000","last_active":"2024-09-10T22:02:35.723568+0000","last_peered":"2024-09-10T22:02:35.723568+0000","last_clean":"2024-09-10T22:02:35.723568+0000","last_became_active":"2024-09-10T22:01:35.749946+0000","last_became_peered":"2024-09-10T22:01:35.749946+0000","last_unstale":"2024-09-10T22:02:35.723568+0000","last_undegraded":"2024-09-10T22:02:35.723568+0000","last_fullsized":"2024-09-10T22:02:35.723568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.362221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":193,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:46.000213+0000","last_change":"2024-09-10T22:00:30.826427+0000","last_active":"2024-09-10T22:03:46.000213+0000","last_peered":"2024-09-10T22:03:46.000213+0000","last_clean":"2024-09-10T22:03:46.000213+0000","last_became_active":"2024-09-10T22:00:30.826038+0000","last_became_peered":"2024-09-10T22:00:30.826038+0000","last_unstale":"2024-09-10T22:03:46.000213+0000","last_undegraded":"2024-09-10T22:03:46.000213+0000","last_fullsized":"2024-09-10T22:03:46.000213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:45.075208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":140,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.833092+0000","last_change":"2024-09-10T22:00:34.494479+0000","last_active":"2024-09-10T22:03:03.833092+0000","last_peered":"2024-09-10T22:03:03.833092+0000","last_clean":"2024-09-10T22:03:03.833092+0000","last_became_active":"2024-09-10T22:00:34.494087+0000","last_became_peered":"2024-09-10T22:00:34.494087+0000","last_unstale":"2024-09-10T22:03:03.833092+0000","last_undegraded":"2024-09-10T22:03:03.833092+0000","last_fullsized":"2024-09-10T22:03:03.833092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:28.548999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723643+0000","last_change":"2024-09-10T22:00:43.714568+0000","last_active":"2024-09-10T22:02:35.723643+0000","last_peered":"2024-09-10T22:02:35.723643+0000","last_clean":"2024-09-10T22:02:35.723643+0000","last_became_active":"2024-09-10T22:00:43.712923+0000","last_became_peered":"2024-09-10T22:00:43.712923+0000","last_unstale":"2024-09-10T22:02:35.723643+0000","last_undegraded":"2024-09-10T22:02:35.723643+0000","last_fullsized":"2024-09-10T22:02:35.723643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:27.945799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.833182+0000","last_change":"2024-09-10T22:00:34.496942+0000","last_active":"2024-09-10T22:03:03.833182+0000","last_peered":"2024-09-10T22:03:03.833182+0000","last_clean":"2024-09-10T22:03:03.833182+0000","last_became_active":"2024-09-10T22:00:34.496475+0000","last_became_peered":"2024-09-10T22:00:34.496475+0000","last_unstale":"2024-09-10T22:03:03.833182+0000","last_undegraded":"2024-09-10T22:03:03.833182+0000","last_fullsized":"2024-09-10T22:03:03.833182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:33.277346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837137+0000","last_change":"2024-09-10T22:00:34.506571+0000","last_active":"2024-09-10T22:03:03.837137+0000","last_peered":"2024-09-10T22:03:03.837137+0000","last_clean":"2024-09-10T22:03:03.837137+0000","last_became_active":"2024-09-10T22:00:34.505718+0000","last_became_peered":"2024-09-10T22:00:34.505718+0000","last_unstale":"2024-09-10T22:03:03.837137+0000","last_undegraded":"2024-09-10T22:03:03.837137+0000","last_fullsized":"2024-09-10T22:03:03.837137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:10.111415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837418+0000","last_change":"2024-09-10T22:00:34.502030+0000","last_active":"2024-09-10T22:03:03.837418+0000","last_peered":"2024-09-10T22:03:03.837418+0000","last_clean":"2024-09-10T22:03:03.837418+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:03:03.837418+0000","last_undegraded":"2024-09-10T22:03:03.837418+0000","last_fullsized":"2024-09-10T22:03:03.837418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:09.532523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.846513+0000","last_change":"2024-09-10T22:00:34.513630+0000","last_active":"2024-09-10T22:03:03.846513+0000","last_peered":"2024-09-10T22:03:03.846513+0000","last_clean":"2024-09-10T22:03:03.846513+0000","last_became_active":"2024-09-10T22:00:34.513348+0000","last_became_peered":"2024-09-10T22:00:34.513348+0000","last_unstale":"2024-09-10T22:03:03.846513+0000","last_undegraded":"2024-09-10T22:03:03.846513+0000","last_fullsized":"2024-09-10T22:03:03.846513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:53.942004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.844287+0000","last_change":"2024-09-10T22:00:34.508697+0000","last_active":"2024-09-10T22:03:03.844287+0000","last_peered":"2024-09-10T22:03:03.844287+0000","last_clean":"2024-09-10T22:03:03.844287+0000","last_became_active":"2024-09-10T22:00:34.508371+0000","last_became_peered":"2024-09-10T22:00:34.508371+0000","last_unstale":"2024-09-10T22:03:03.844287+0000","last_undegraded":"2024-09-10T22:03:03.844287+0000","last_fullsized":"2024-09-10T22:03:03.844287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:32.843517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.832359+0000","last_change":"2024-09-10T22:00:34.496021+0000","last_active":"2024-09-10T22:03:03.832359+0000","last_peered":"2024-09-10T22:03:03.832359+0000","last_clean":"2024-09-10T22:03:03.832359+0000","last_became_active":"2024-09-10T22:00:34.495663+0000","last_became_peered":"2024-09-10T22:00:34.495663+0000","last_unstale":"2024-09-10T22:03:03.832359+0000","last_undegraded":"2024-09-10T22:03:03.832359+0000","last_fullsized":"2024-09-10T22:03:03.832359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:16.512182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851192+0000","last_change":"2024-09-10T22:00:34.506589+0000","last_active":"2024-09-10T22:03:03.851192+0000","last_peered":"2024-09-10T22:03:03.851192+0000","last_clean":"2024-09-10T22:03:03.851192+0000","last_became_active":"2024-09-10T22:00:34.506214+0000","last_became_peered":"2024-09-10T22:00:34.506214+0000","last_unstale":"2024-09-10T22:03:03.851192+0000","last_undegraded":"2024-09-10T22:03:03.851192+0000","last_fullsized":"2024-09-10T22:03:03.851192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:47.072458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.825874+0000","last_change":"2024-09-10T22:00:34.503471+0000","last_active":"2024-09-10T22:03:03.825874+0000","last_peered":"2024-09-10T22:03:03.825874+0000","last_clean":"2024-09-10T22:03:03.825874+0000","last_became_active":"2024-09-10T22:00:34.501145+0000","last_became_peered":"2024-09-10T22:00:34.501145+0000","last_unstale":"2024-09-10T22:03:03.825874+0000","last_undegraded":"2024-09-10T22:03:03.825874+0000","last_fullsized":"2024-09-10T22:03:03.825874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:24.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840484+0000","last_change":"2024-09-10T22:00:34.490177+0000","last_active":"2024-09-10T22:03:03.840484+0000","last_peered":"2024-09-10T22:03:03.840484+0000","last_clean":"2024-09-10T22:03:03.840484+0000","last_became_active":"2024-09-10T22:00:34.488428+0000","last_became_peered":"2024-09-10T22:00:34.488428+0000","last_unstale":"2024-09-10T22:03:03.840484+0000","last_undegraded":"2024-09-10T22:03:03.840484+0000","last_fullsized":"2024-09-10T22:03:03.840484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:21.111826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841640+0000","last_change":"2024-09-10T22:00:34.499174+0000","last_active":"2024-09-10T22:03:03.841640+0000","last_peered":"2024-09-10T22:03:03.841640+0000","last_clean":"2024-09-10T22:03:03.841640+0000","last_became_active":"2024-09-10T22:00:34.498824+0000","last_became_peered":"2024-09-10T22:00:34.498824+0000","last_unstale":"2024-09-10T22:03:03.841640+0000","last_undegraded":"2024-09-10T22:03:03.841640+0000","last_fullsized":"2024-09-10T22:03:03.841640+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:15.946270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853015+0000","last_change":"2024-09-10T22:00:34.505748+0000","last_active":"2024-09-10T22:03:03.853015+0000","last_peered":"2024-09-10T22:03:03.853015+0000","last_clean":"2024-09-10T22:03:03.853015+0000","last_became_active":"2024-09-10T22:00:34.500441+0000","last_became_peered":"2024-09-10T22:00:34.500441+0000","last_unstale":"2024-09-10T22:03:03.853015+0000","last_undegraded":"2024-09-10T22:03:03.853015+0000","last_fullsized":"2024-09-10T22:03:03.853015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:22.177566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.850314+0000","last_change":"2024-09-10T22:00:34.502319+0000","last_active":"2024-09-10T22:03:03.850314+0000","last_peered":"2024-09-10T22:03:03.850314+0000","last_clean":"2024-09-10T22:03:03.850314+0000","last_became_active":"2024-09-10T22:00:34.501427+0000","last_became_peered":"2024-09-10T22:00:34.501427+0000","last_unstale":"2024-09-10T22:03:03.850314+0000","last_undegraded":"2024-09-10T22:03:03.850314+0000","last_fullsized":"2024-09-10T22:03:03.850314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:18.463319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.842219+0000","last_change":"2024-09-10T22:00:34.497571+0000","last_active":"2024-09-10T22:03:03.842219+0000","last_peered":"2024-09-10T22:03:03.842219+0000","last_clean":"2024-09-10T22:03:03.842219+0000","last_became_active":"2024-09-10T22:00:34.497205+0000","last_became_peered":"2024-09-10T22:00:34.497205+0000","last_unstale":"2024-09-10T22:03:03.842219+0000","last_undegraded":"2024-09-10T22:03:03.842219+0000","last_fullsized":"2024-09-10T22:03:03.842219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:49.924518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837147+0000","last_change":"2024-09-10T22:00:34.504641+0000","last_active":"2024-09-10T22:03:03.837147+0000","last_peered":"2024-09-10T22:03:03.837147+0000","last_clean":"2024-09-10T22:03:03.837147+0000","last_became_active":"2024-09-10T22:00:34.504183+0000","last_became_peered":"2024-09-10T22:00:34.504183+0000","last_unstale":"2024-09-10T22:03:03.837147+0000","last_undegraded":"2024-09-10T22:03:03.837147+0000","last_fullsized":"2024-09-10T22:03:03.837147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:06.704972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851050+0000","last_change":"2024-09-10T22:00:34.503884+0000","last_active":"2024-09-10T22:03:03.851050+0000","last_peered":"2024-09-10T22:03:03.851050+0000","last_clean":"2024-09-10T22:03:03.851050+0000","last_became_active":"2024-09-10T22:00:34.502950+0000","last_became_peered":"2024-09-10T22:00:34.502950+0000","last_unstale":"2024-09-10T22:03:03.851050+0000","last_undegraded":"2024-09-10T22:03:03.851050+0000","last_fullsized":"2024-09-10T22:03:03.851050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:45.560958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.850635+0000","last_change":"2024-09-10T22:00:34.504981+0000","last_active":"2024-09-10T22:03:03.850635+0000","last_peered":"2024-09-10T22:03:03.850635+0000","last_clean":"2024-09-10T22:03:03.850635+0000","last_became_active":"2024-09-10T22:00:34.500474+0000","last_became_peered":"2024-09-10T22:00:34.500474+0000","last_unstale":"2024-09-10T22:03:03.850635+0000","last_undegraded":"2024-09-10T22:03:03.850635+0000","last_fullsized":"2024-09-10T22:03:03.850635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:20.892605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853045+0000","last_change":"2024-09-10T22:00:34.506650+0000","last_active":"2024-09-10T22:03:03.853045+0000","last_peered":"2024-09-10T22:03:03.853045+0000","last_clean":"2024-09-10T22:03:03.853045+0000","last_became_active":"2024-09-10T22:00:34.505698+0000","last_became_peered":"2024-09-10T22:00:34.505698+0000","last_unstale":"2024-09-10T22:03:03.853045+0000","last_undegraded":"2024-09-10T22:03:03.853045+0000","last_fullsized":"2024-09-10T22:03:03.853045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:29.221693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841202+0000","last_change":"2024-09-10T22:00:34.502238+0000","last_active":"2024-09-10T22:03:03.841202+0000","last_peered":"2024-09-10T22:03:03.841202+0000","last_clean":"2024-09-10T22:03:03.841202+0000","last_became_active":"2024-09-10T22:00:34.500049+0000","last_became_peered":"2024-09-10T22:00:34.500049+0000","last_unstale":"2024-09-10T22:03:03.841202+0000","last_undegraded":"2024-09-10T22:03:03.841202+0000","last_fullsized":"2024-09-10T22:03:03.841202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:05.669784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.839971+0000","last_change":"2024-09-10T22:00:34.499340+0000","last_active":"2024-09-10T22:03:03.839971+0000","last_peered":"2024-09-10T22:03:03.839971+0000","last_clean":"2024-09-10T22:03:03.839971+0000","last_became_active":"2024-09-10T22:00:34.497655+0000","last_became_peered":"2024-09-10T22:00:34.497655+0000","last_unstale":"2024-09-10T22:03:03.839971+0000","last_undegraded":"2024-09-10T22:03:03.839971+0000","last_fullsized":"2024-09-10T22:03:03.839971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:34.115829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838094+0000","last_change":"2024-09-10T22:00:34.500705+0000","last_active":"2024-09-10T22:03:03.838094+0000","last_peered":"2024-09-10T22:03:03.838094+0000","last_clean":"2024-09-10T22:03:03.838094+0000","last_became_active":"2024-09-10T22:00:34.500071+0000","last_became_peered":"2024-09-10T22:00:34.500071+0000","last_unstale":"2024-09-10T22:03:03.838094+0000","last_undegraded":"2024-09-10T22:03:03.838094+0000","last_fullsized":"2024-09-10T22:03:03.838094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:40.519976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838120+0000","last_change":"2024-09-10T22:00:34.510570+0000","last_active":"2024-09-10T22:03:03.838120+0000","last_peered":"2024-09-10T22:03:03.838120+0000","last_clean":"2024-09-10T22:03:03.838120+0000","last_became_active":"2024-09-10T22:00:34.506922+0000","last_became_peered":"2024-09-10T22:00:34.506922+0000","last_unstale":"2024-09-10T22:03:03.838120+0000","last_undegraded":"2024-09-10T22:03:03.838120+0000","last_fullsized":"2024-09-10T22:03:03.838120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.489031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853043+0000","last_change":"2024-09-10T22:00:34.509985+0000","last_active":"2024-09-10T22:03:03.853043+0000","last_peered":"2024-09-10T22:03:03.853043+0000","last_clean":"2024-09-10T22:03:03.853043+0000","last_became_active":"2024-09-10T22:00:34.509527+0000","last_became_peered":"2024-09-10T22:00:34.509527+0000","last_unstale":"2024-09-10T22:03:03.853043+0000","last_undegraded":"2024-09-10T22:03:03.853043+0000","last_fullsized":"2024-09-10T22:03:03.853043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:49.463334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.839704+0000","last_change":"2024-09-10T22:00:34.497986+0000","last_active":"2024-09-10T22:03:03.839704+0000","last_peered":"2024-09-10T22:03:03.839704+0000","last_clean":"2024-09-10T22:03:03.839704+0000","last_became_active":"2024-09-10T22:00:34.497612+0000","last_became_peered":"2024-09-10T22:00:34.497612+0000","last_unstale":"2024-09-10T22:03:03.839704+0000","last_undegraded":"2024-09-10T22:03:03.839704+0000","last_fullsized":"2024-09-10T22:03:03.839704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:15.696988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851846+0000","last_change":"2024-09-10T22:00:34.498482+0000","last_active":"2024-09-10T22:03:03.851846+0000","last_peered":"2024-09-10T22:03:03.851846+0000","last_clean":"2024-09-10T22:03:03.851846+0000","last_became_active":"2024-09-10T22:00:34.498139+0000","last_became_peered":"2024-09-10T22:00:34.498139+0000","last_unstale":"2024-09-10T22:03:03.851846+0000","last_undegraded":"2024-09-10T22:03:03.851846+0000","last_fullsized":"2024-09-10T22:03:03.851846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:23.427253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841157+0000","last_change":"2024-09-10T22:00:34.491325+0000","last_active":"2024-09-10T22:03:03.841157+0000","last_peered":"2024-09-10T22:03:03.841157+0000","last_clean":"2024-09-10T22:03:03.841157+0000","last_became_active":"2024-09-10T22:00:34.491026+0000","last_became_peered":"2024-09-10T22:00:34.491026+0000","last_unstale":"2024-09-10T22:03:03.841157+0000","last_undegraded":"2024-09-10T22:03:03.841157+0000","last_fullsized":"2024-09-10T22:03:03.841157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:07.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.843338+0000","last_change":"2024-09-10T22:00:34.503975+0000","last_active":"2024-09-10T22:03:03.843338+0000","last_peered":"2024-09-10T22:03:03.843338+0000","last_clean":"2024-09-10T22:03:03.843338+0000","last_became_active":"2024-09-10T22:00:34.503624+0000","last_became_peered":"2024-09-10T22:00:34.503624+0000","last_unstale":"2024-09-10T22:03:03.843338+0000","last_undegraded":"2024-09-10T22:03:03.843338+0000","last_fullsized":"2024-09-10T22:03:03.843338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:17.678811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837903+0000","last_change":"2024-09-10T22:01:35.596824+0000","last_active":"2024-09-10T22:03:03.837903+0000","last_peered":"2024-09-10T22:03:03.837903+0000","last_clean":"2024-09-10T22:03:03.837903+0000","last_became_active":"2024-09-10T22:01:35.596550+0000","last_became_peered":"2024-09-10T22:01:35.596550+0000","last_unstale":"2024-09-10T22:03:03.837903+0000","last_undegraded":"2024-09-10T22:03:03.837903+0000","last_fullsized":"2024-09-10T22:03:03.837903+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:19.617649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.832879+0000","last_change":"2024-09-10T22:02:35.895282+0000","last_active":"2024-09-10T22:03:03.832879+0000","last_peered":"2024-09-10T22:03:03.832879+0000","last_clean":"2024-09-10T22:03:03.832879+0000","last_became_active":"2024-09-10T22:02:35.894850+0000","last_became_peered":"2024-09-10T22:02:35.894850+0000","last_unstale":"2024-09-10T22:03:03.832879+0000","last_undegraded":"2024-09-10T22:03:03.832879+0000","last_fullsized":"2024-09-10T22:03:03.832879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:50.109012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840009+0000","last_change":"2024-09-10T22:00:34.513134+0000","last_active":"2024-09-10T22:03:03.840009+0000","last_peered":"2024-09-10T22:03:03.840009+0000","last_clean":"2024-09-10T22:03:03.840009+0000","last_became_active":"2024-09-10T22:00:34.507578+0000","last_became_peered":"2024-09-10T22:00:34.507578+0000","last_unstale":"2024-09-10T22:03:03.840009+0000","last_undegraded":"2024-09-10T22:03:03.840009+0000","last_fullsized":"2024-09-10T22:03:03.840009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:33.387337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851019+0000","last_change":"2024-09-10T22:00:34.507645+0000","last_active":"2024-09-10T22:03:03.851019+0000","last_peered":"2024-09-10T22:03:03.851019+0000","last_clean":"2024-09-10T22:03:03.851019+0000","last_became_active":"2024-09-10T22:00:34.503614+0000","last_became_peered":"2024-09-10T22:00:34.503614+0000","last_unstale":"2024-09-10T22:03:03.851019+0000","last_undegraded":"2024-09-10T22:03:03.851019+0000","last_fullsized":"2024-09-10T22:03:03.851019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:53.033523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841836+0000","last_change":"2024-09-10T22:00:34.501891+0000","last_active":"2024-09-10T22:03:03.841836+0000","last_peered":"2024-09-10T22:03:03.841836+0000","last_clean":"2024-09-10T22:03:03.841836+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:03:03.841836+0000","last_undegraded":"2024-09-10T22:03:03.841836+0000","last_fullsized":"2024-09-10T22:03:03.841836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:30.068461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.842854+0000","last_change":"2024-09-10T22:00:34.501475+0000","last_active":"2024-09-10T22:03:03.842854+0000","last_peered":"2024-09-10T22:03:03.842854+0000","last_clean":"2024-09-10T22:03:03.842854+0000","last_became_active":"2024-09-10T22:00:34.500909+0000","last_became_peered":"2024-09-10T22:00:34.500909+0000","last_unstale":"2024-09-10T22:03:03.842854+0000","last_undegraded":"2024-09-10T22:03:03.842854+0000","last_fullsized":"2024-09-10T22:03:03.842854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:39.616773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":466644,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1557,"num_read_kb":918,"num_write":371,"num_write_kb":1250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606208,"data_stored":616828,"data_compressed":19272,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},"log_size":289,"ondisk_log_size":289,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246272,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678071,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":536890,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874299,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":1186007,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037826,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":539779,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234057,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":535949,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430288,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":1135242,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593814,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":531719,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790045,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":1128774,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953571,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":536286,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149800,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1015808,"data_stored":530895,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378735,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":597774,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542261,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1081344,"data_stored":595767,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705787,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":533668,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19272,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:03:52.123 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:03:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:51 smithi154 ceph-mon[34495]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:03:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:51 smithi154 ceph-mon[34495]: pgmap v417: 129 pgs: 129 active+clean; 1.0 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 92 KiB/s wr, 34 op/s 2024-09-10T22:03:52.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:51 smithi171 ceph-mon[34138]: from='client.15591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:03:52.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:51 smithi171 ceph-mon[34138]: pgmap v417: 129 pgs: 129 active+clean; 1.0 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 92 KiB/s wr, 34 op/s 2024-09-10T22:03:52.492 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:53.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.078+0000 7f9073e00640 1 -- 172.21.15.90:0/2258683760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074075540 msgr2=0x7f9074075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:53.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.078+0000 7f9073e00640 1 --2- 172.21.15.90:0/2258683760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074075540 0x7f9074075920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9064009900 tx=0x7f906402a400 comp rx=0 tx=0).stop 2024-09-10T22:03:53.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.078+0000 7f9073e00640 1 -- 172.21.15.90:0/2258683760 shutdown_connections 2024-09-10T22:03:53.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.078+0000 7f9073e00640 1 --2- 172.21.15.90:0/2258683760 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9074113970 0x7f9074115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.078+0000 7f9073e00640 1 --2- 172.21.15.90:0/2258683760 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9074075e60 0x7f9074113430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.078+0000 7f9073e00640 1 --2- 172.21.15.90:0/2258683760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074075540 0x7f9074075920 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.078+0000 7f9073e00640 1 -- 172.21.15.90:0/2258683760 >> 172.21.15.90:0/2258683760 conn(0x7f90740fe6c0 msgr2=0x7f9074100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.078+0000 7f9073e00640 1 -- 172.21.15.90:0/2258683760 shutdown_connections 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.078+0000 7f9073e00640 1 -- 172.21.15.90:0/2258683760 wait complete. 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.079+0000 7f9073e00640 1 Processor -- start 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.079+0000 7f9073e00640 1 -- start start 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.079+0000 7f9073e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9074075540 0x7f907419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.079+0000 7f9073e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9074075e60 0x7f907419fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.079+0000 7f9073e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074113970 0x7f90741a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.079+0000 7f9073e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f90741190a0 con 0x7f9074113970 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.079+0000 7f9073e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9074118f20 con 0x7f9074075540 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.079+0000 7f9073e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9074119220 con 0x7f9074075e60 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f9073400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074113970 0x7f90741a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f9073400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074113970 0x7f90741a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60744/0 (socket says 172.21.15.90:60744) 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f9073400640 1 -- 172.21.15.90:0/1359388491 learned_addr learned my addr 172.21.15.90:0/1359388491 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:53.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f9072a00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9074075540 0x7f907419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f906a000640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9074075e60 0x7f907419fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f9073400640 1 -- 172.21.15.90:0/1359388491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9074075e60 msgr2=0x7f907419fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f9073400640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9074075e60 0x7f907419fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f9073400640 1 -- 172.21.15.90:0/1359388491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9074075540 msgr2=0x7f907419f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f9073400640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9074075540 0x7f907419f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.080+0000 7f9073400640 1 -- 172.21.15.90:0/1359388491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90741a55e0 con 0x7f9074113970 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.081+0000 7f9073400640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074113970 0x7f90741a4e10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f906c00ccb0 tx=0x7f906c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.081+0000 7f906be00640 1 -- 172.21.15.90:0/1359388491 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c00f450 con 0x7f9074113970 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.081+0000 7f906be00640 1 -- 172.21.15.90:0/1359388491 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f906c00fa80 con 0x7f9074113970 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.081+0000 7f906be00640 1 -- 172.21.15.90:0/1359388491 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c018a40 con 0x7f9074113970 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.081+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90741a58d0 con 0x7f9074113970 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.081+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90741abf90 con 0x7f9074113970 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.082+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9074075920 con 0x7f9074113970 2024-09-10T22:03:53.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.084+0000 7f906be00640 1 -- 172.21.15.90:0/1359388491 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f906c00f5f0 con 0x7f9074113970 2024-09-10T22:03:53.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.085+0000 7f906be00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90580b54e0 0x7f90580b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:53.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.086+0000 7f906be00640 1 -- 172.21.15.90:0/1359388491 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f906c0e6520 con 0x7f9074113970 2024-09-10T22:03:53.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.086+0000 7f9072a00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90580b54e0 0x7f90580b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:53.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.086+0000 7f9072a00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90580b54e0 0x7f90580b79a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f90640057c0 tx=0x7f90640058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:53.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.089+0000 7f906be00640 1 -- 172.21.15.90:0/1359388491 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f906c014030 con 0x7f9074113970 2024-09-10T22:03:53.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.271+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90741009e0 con 0x7f90580b54e0 2024-09-10T22:03:53.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.277+0000 7f906be00640 1 -- 172.21.15.90:0/1359388491 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340898 (secure 0 0 0) 0x7f90741009e0 con 0x7f90580b54e0 2024-09-10T22:03:53.281 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:53.283 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90580b54e0 msgr2=0x7f90580b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90580b54e0 0x7f90580b79a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f90640057c0 tx=0x7f90640058d0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074113970 msgr2=0x7f90741a4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074113970 0x7f90741a4e10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f906c00ccb0 tx=0x7f906c00a620 comp rx=0 tx=0).stop 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 shutdown_connections 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90580b54e0 0x7f90580b79a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9074113970 0x7f90741a4e10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9074075e60 0x7f907419fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 --2- 172.21.15.90:0/1359388491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9074075540 0x7f907419f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 >> 172.21.15.90:0/1359388491 conn(0x7f90740fe6c0 msgr2=0x7f90740ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:53.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 shutdown_connections 2024-09-10T22:03:53.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:53.282+0000 7f9073e00640 1 -- 172.21.15.90:0/1359388491 wait complete. 2024-09-10T22:03:54.172 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-10T22:03:51.633691+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1062053,"num_objects":103,"num_object_clones":0,"num_object_copies":311,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":0,"num_read":1665,"num_read_kb":1013,"num_write":504,"num_write_kb":3413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":890,"ondisk_log_size":890,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349244,"kb_used_data":12468,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592580,"statfs":{"total":1151940427776,"available":1151582801920,"internally_reserved":0,"allocated":12767232,"data_stored":9973827,"data_compressed":573090,"data_compressed_allocated":774144,"data_compressed_original":4374528,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":399995,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":303,"num_read_kb":231,"num_write":117,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003046"},"pg_stats":[{"pgid":"2.1f","version":"90'15","reported_seq":197,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:46.979191+0000","last_change":"2024-09-10T22:00:30.824265+0000","last_active":"2024-09-10T22:03:46.979191+0000","last_peered":"2024-09-10T22:03:46.979191+0000","last_clean":"2024-09-10T22:03:46.979191+0000","last_became_active":"2024-09-10T22:00:30.823905+0000","last_became_peered":"2024-09-10T22:00:30.823905+0000","last_unstale":"2024-09-10T22:03:46.979191+0000","last_undegraded":"2024-09-10T22:03:46.979191+0000","last_fullsized":"2024-09-10T22:03:46.979191+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:16.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837001+0000","last_change":"2024-09-10T22:00:34.497719+0000","last_active":"2024-09-10T22:03:03.837001+0000","last_peered":"2024-09-10T22:03:03.837001+0000","last_clean":"2024-09-10T22:03:03.837001+0000","last_became_active":"2024-09-10T22:00:34.497380+0000","last_became_peered":"2024-09-10T22:00:34.497380+0000","last_unstale":"2024-09-10T22:03:03.837001+0000","last_undegraded":"2024-09-10T22:03:03.837001+0000","last_fullsized":"2024-09-10T22:03:03.837001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:26.336299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723331+0000","last_change":"2024-09-10T22:00:43.710835+0000","last_active":"2024-09-10T22:02:35.723331+0000","last_peered":"2024-09-10T22:02:35.723331+0000","last_clean":"2024-09-10T22:02:35.723331+0000","last_became_active":"2024-09-10T22:00:43.710363+0000","last_became_peered":"2024-09-10T22:00:43.710363+0000","last_unstale":"2024-09-10T22:02:35.723331+0000","last_undegraded":"2024-09-10T22:02:35.723331+0000","last_fullsized":"2024-09-10T22:02:35.723331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:28.173482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723977+0000","last_change":"2024-09-10T22:00:31.083228+0000","last_active":"2024-09-10T22:02:35.723977+0000","last_peered":"2024-09-10T22:02:35.723977+0000","last_clean":"2024-09-10T22:02:35.723977+0000","last_became_active":"2024-09-10T22:00:31.082911+0000","last_became_peered":"2024-09-10T22:00:31.082911+0000","last_unstale":"2024-09-10T22:02:35.723977+0000","last_undegraded":"2024-09-10T22:02:35.723977+0000","last_fullsized":"2024-09-10T22:02:35.723977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:28.598628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853513+0000","last_change":"2024-09-10T22:00:34.509939+0000","last_active":"2024-09-10T22:03:03.853513+0000","last_peered":"2024-09-10T22:03:03.853513+0000","last_clean":"2024-09-10T22:03:03.853513+0000","last_became_active":"2024-09-10T22:00:34.509536+0000","last_became_peered":"2024-09-10T22:00:34.509536+0000","last_unstale":"2024-09-10T22:03:03.853513+0000","last_undegraded":"2024-09-10T22:03:03.853513+0000","last_fullsized":"2024-09-10T22:03:03.853513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:27.602378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897125+0000","last_change":"2024-09-10T22:00:43.714662+0000","last_active":"2024-09-10T22:02:35.897125+0000","last_peered":"2024-09-10T22:02:35.897125+0000","last_clean":"2024-09-10T22:02:35.897125+0000","last_became_active":"2024-09-10T22:00:43.714074+0000","last_became_peered":"2024-09-10T22:00:43.714074+0000","last_unstale":"2024-09-10T22:02:35.897125+0000","last_undegraded":"2024-09-10T22:02:35.897125+0000","last_fullsized":"2024-09-10T22:02:35.897125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:41.204519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948479+0000","last_change":"2024-09-10T22:01:35.209979+0000","last_active":"2024-09-10T22:02:35.948479+0000","last_peered":"2024-09-10T22:02:35.948479+0000","last_clean":"2024-09-10T22:02:35.948479+0000","last_became_active":"2024-09-10T22:01:35.197192+0000","last_became_peered":"2024-09-10T22:01:35.197192+0000","last_unstale":"2024-09-10T22:02:35.948479+0000","last_undegraded":"2024-09-10T22:02:35.948479+0000","last_fullsized":"2024-09-10T22:02:35.948479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:47.707679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853796+0000","last_change":"2024-09-10T22:00:34.502795+0000","last_active":"2024-09-10T22:03:03.853796+0000","last_peered":"2024-09-10T22:03:03.853796+0000","last_clean":"2024-09-10T22:03:03.853796+0000","last_became_active":"2024-09-10T22:00:34.501112+0000","last_became_peered":"2024-09-10T22:00:34.501112+0000","last_unstale":"2024-09-10T22:03:03.853796+0000","last_undegraded":"2024-09-10T22:03:03.853796+0000","last_fullsized":"2024-09-10T22:03:03.853796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:45.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897363+0000","last_change":"2024-09-10T22:01:35.619765+0000","last_active":"2024-09-10T22:02:35.897363+0000","last_peered":"2024-09-10T22:02:35.897363+0000","last_clean":"2024-09-10T22:02:35.897363+0000","last_became_active":"2024-09-10T22:01:35.619407+0000","last_became_peered":"2024-09-10T22:01:35.619407+0000","last_unstale":"2024-09-10T22:02:35.897363+0000","last_undegraded":"2024-09-10T22:02:35.897363+0000","last_fullsized":"2024-09-10T22:02:35.897363+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:17.064585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726660+0000","last_change":"2024-09-10T22:00:30.818224+0000","last_active":"2024-09-10T22:02:35.726660+0000","last_peered":"2024-09-10T22:02:35.726660+0000","last_clean":"2024-09-10T22:02:35.726660+0000","last_became_active":"2024-09-10T22:00:30.817781+0000","last_became_peered":"2024-09-10T22:00:30.817781+0000","last_unstale":"2024-09-10T22:02:35.726660+0000","last_undegraded":"2024-09-10T22:02:35.726660+0000","last_fullsized":"2024-09-10T22:02:35.726660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:47.097642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851346+0000","last_change":"2024-09-10T22:02:35.949974+0000","last_active":"2024-09-10T22:03:03.851346+0000","last_peered":"2024-09-10T22:03:03.851346+0000","last_clean":"2024-09-10T22:03:03.851346+0000","last_became_active":"2024-09-10T22:02:35.949618+0000","last_became_peered":"2024-09-10T22:02:35.949618+0000","last_unstale":"2024-09-10T22:03:03.851346+0000","last_undegraded":"2024-09-10T22:03:03.851346+0000","last_fullsized":"2024-09-10T22:03:03.851346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:02.877650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897288+0000","last_change":"2024-09-10T22:00:43.712066+0000","last_active":"2024-09-10T22:02:35.897288+0000","last_peered":"2024-09-10T22:02:35.897288+0000","last_clean":"2024-09-10T22:02:35.897288+0000","last_became_active":"2024-09-10T22:00:43.711614+0000","last_became_peered":"2024-09-10T22:00:43.711614+0000","last_unstale":"2024-09-10T22:02:35.897288+0000","last_undegraded":"2024-09-10T22:02:35.897288+0000","last_fullsized":"2024-09-10T22:02:35.897288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:42.403367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.854262+0000","last_change":"2024-09-10T22:00:34.501943+0000","last_active":"2024-09-10T22:03:03.854262+0000","last_peered":"2024-09-10T22:03:03.854262+0000","last_clean":"2024-09-10T22:03:03.854262+0000","last_became_active":"2024-09-10T22:00:34.501602+0000","last_became_peered":"2024-09-10T22:00:34.501602+0000","last_unstale":"2024-09-10T22:03:03.854262+0000","last_undegraded":"2024-09-10T22:03:03.854262+0000","last_fullsized":"2024-09-10T22:03:03.854262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:53.200974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726593+0000","last_change":"2024-09-10T22:01:35.200791+0000","last_active":"2024-09-10T22:02:35.726593+0000","last_peered":"2024-09-10T22:02:35.726593+0000","last_clean":"2024-09-10T22:02:35.726593+0000","last_became_active":"2024-09-10T22:01:35.196059+0000","last_became_peered":"2024-09-10T22:01:35.196059+0000","last_unstale":"2024-09-10T22:02:35.726593+0000","last_undegraded":"2024-09-10T22:02:35.726593+0000","last_fullsized":"2024-09-10T22:02:35.726593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:51.349216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897010+0000","last_change":"2024-09-10T22:00:43.760567+0000","last_active":"2024-09-10T22:02:35.897010+0000","last_peered":"2024-09-10T22:02:35.897010+0000","last_clean":"2024-09-10T22:02:35.897010+0000","last_became_active":"2024-09-10T22:00:43.760064+0000","last_became_peered":"2024-09-10T22:00:43.760064+0000","last_unstale":"2024-09-10T22:02:35.897010+0000","last_undegraded":"2024-09-10T22:02:35.897010+0000","last_fullsized":"2024-09-10T22:02:35.897010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:55.469525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.843933+0000","last_change":"2024-09-10T22:00:34.486225+0000","last_active":"2024-09-10T22:03:03.843933+0000","last_peered":"2024-09-10T22:03:03.843933+0000","last_clean":"2024-09-10T22:03:03.843933+0000","last_became_active":"2024-09-10T22:00:34.485896+0000","last_became_peered":"2024-09-10T22:00:34.485896+0000","last_unstale":"2024-09-10T22:03:03.843933+0000","last_undegraded":"2024-09-10T22:03:03.843933+0000","last_fullsized":"2024-09-10T22:03:03.843933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:00.151927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:43.663404+0000","last_change":"2024-09-10T22:00:30.817958+0000","last_active":"2024-09-10T22:02:43.663404+0000","last_peered":"2024-09-10T22:02:43.663404+0000","last_clean":"2024-09-10T22:02:43.663404+0000","last_became_active":"2024-09-10T22:00:30.817646+0000","last_became_peered":"2024-09-10T22:00:30.817646+0000","last_unstale":"2024-09-10T22:02:43.663404+0000","last_undegraded":"2024-09-10T22:02:43.663404+0000","last_fullsized":"2024-09-10T22:02:43.663404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:27.722184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897658+0000","last_change":"2024-09-10T22:00:43.716465+0000","last_active":"2024-09-10T22:02:35.897658+0000","last_peered":"2024-09-10T22:02:35.897658+0000","last_clean":"2024-09-10T22:02:35.897658+0000","last_became_active":"2024-09-10T22:00:43.716165+0000","last_became_peered":"2024-09-10T22:00:43.716165+0000","last_unstale":"2024-09-10T22:02:35.897658+0000","last_undegraded":"2024-09-10T22:02:35.897658+0000","last_fullsized":"2024-09-10T22:02:35.897658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:18.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841664+0000","last_change":"2024-09-10T22:00:34.493992+0000","last_active":"2024-09-10T22:03:03.841664+0000","last_peered":"2024-09-10T22:03:03.841664+0000","last_clean":"2024-09-10T22:03:03.841664+0000","last_became_active":"2024-09-10T22:00:34.493670+0000","last_became_peered":"2024-09-10T22:00:34.493670+0000","last_unstale":"2024-09-10T22:03:03.841664+0000","last_undegraded":"2024-09-10T22:03:03.841664+0000","last_fullsized":"2024-09-10T22:03:03.841664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:04.991819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722649+0000","last_change":"2024-09-10T22:00:30.819830+0000","last_active":"2024-09-10T22:02:35.722649+0000","last_peered":"2024-09-10T22:02:35.722649+0000","last_clean":"2024-09-10T22:02:35.722649+0000","last_became_active":"2024-09-10T22:00:30.819401+0000","last_became_peered":"2024-09-10T22:00:30.819401+0000","last_unstale":"2024-09-10T22:02:35.722649+0000","last_undegraded":"2024-09-10T22:02:35.722649+0000","last_fullsized":"2024-09-10T22:02:35.722649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:37.823107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724263+0000","last_change":"2024-09-10T22:00:43.718500+0000","last_active":"2024-09-10T22:02:35.724263+0000","last_peered":"2024-09-10T22:02:35.724263+0000","last_clean":"2024-09-10T22:02:35.724263+0000","last_became_active":"2024-09-10T22:00:43.717667+0000","last_became_peered":"2024-09-10T22:00:43.717667+0000","last_unstale":"2024-09-10T22:02:35.724263+0000","last_undegraded":"2024-09-10T22:02:35.724263+0000","last_fullsized":"2024-09-10T22:02:35.724263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:23.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.792059+0000","last_change":"2024-09-10T22:00:34.494162+0000","last_active":"2024-09-10T22:03:03.792059+0000","last_peered":"2024-09-10T22:03:03.792059+0000","last_clean":"2024-09-10T22:03:03.792059+0000","last_became_active":"2024-09-10T22:00:34.493819+0000","last_became_peered":"2024-09-10T22:00:34.493819+0000","last_unstale":"2024-09-10T22:03:03.792059+0000","last_undegraded":"2024-09-10T22:03:03.792059+0000","last_fullsized":"2024-09-10T22:03:03.792059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:43.264363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948452+0000","last_change":"2024-09-10T22:00:31.081559+0000","last_active":"2024-09-10T22:02:35.948452+0000","last_peered":"2024-09-10T22:02:35.948452+0000","last_clean":"2024-09-10T22:02:35.948452+0000","last_became_active":"2024-09-10T22:00:31.081257+0000","last_became_peered":"2024-09-10T22:00:31.081257+0000","last_unstale":"2024-09-10T22:02:35.948452+0000","last_undegraded":"2024-09-10T22:02:35.948452+0000","last_fullsized":"2024-09-10T22:02:35.948452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:53.235765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897682+0000","last_change":"2024-09-10T22:01:35.445171+0000","last_active":"2024-09-10T22:02:35.897682+0000","last_peered":"2024-09-10T22:02:35.897682+0000","last_clean":"2024-09-10T22:02:35.897682+0000","last_became_active":"2024-09-10T22:01:35.444651+0000","last_became_peered":"2024-09-10T22:01:35.444651+0000","last_unstale":"2024-09-10T22:02:35.897682+0000","last_undegraded":"2024-09-10T22:02:35.897682+0000","last_fullsized":"2024-09-10T22:02:35.897682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:36.457151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.847341+0000","last_change":"2024-09-10T22:01:35.750982+0000","last_active":"2024-09-10T22:03:03.847341+0000","last_peered":"2024-09-10T22:03:03.847341+0000","last_clean":"2024-09-10T22:03:03.847341+0000","last_became_active":"2024-09-10T22:01:35.750572+0000","last_became_peered":"2024-09-10T22:01:35.750572+0000","last_unstale":"2024-09-10T22:03:03.847341+0000","last_undegraded":"2024-09-10T22:03:03.847341+0000","last_fullsized":"2024-09-10T22:03:03.847341+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:36.596313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897938+0000","last_change":"2024-09-10T22:00:31.311864+0000","last_active":"2024-09-10T22:02:35.897938+0000","last_peered":"2024-09-10T22:02:35.897938+0000","last_clean":"2024-09-10T22:02:35.897938+0000","last_became_active":"2024-09-10T22:00:31.311611+0000","last_became_peered":"2024-09-10T22:00:31.311611+0000","last_unstale":"2024-09-10T22:02:35.897938+0000","last_undegraded":"2024-09-10T22:02:35.897938+0000","last_fullsized":"2024-09-10T22:02:35.897938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:27.969999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723479+0000","last_change":"2024-09-10T22:00:43.708023+0000","last_active":"2024-09-10T22:02:35.723479+0000","last_peered":"2024-09-10T22:02:35.723479+0000","last_clean":"2024-09-10T22:02:35.723479+0000","last_became_active":"2024-09-10T22:00:43.707086+0000","last_became_peered":"2024-09-10T22:00:43.707086+0000","last_unstale":"2024-09-10T22:02:35.723479+0000","last_undegraded":"2024-09-10T22:02:35.723479+0000","last_fullsized":"2024-09-10T22:02:35.723479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:15.585104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.846819+0000","last_change":"2024-09-10T22:00:34.494104+0000","last_active":"2024-09-10T22:03:03.846819+0000","last_peered":"2024-09-10T22:03:03.846819+0000","last_clean":"2024-09-10T22:03:03.846819+0000","last_became_active":"2024-09-10T22:00:34.493404+0000","last_became_peered":"2024-09-10T22:00:34.493404+0000","last_unstale":"2024-09-10T22:03:03.846819+0000","last_undegraded":"2024-09-10T22:03:03.846819+0000","last_fullsized":"2024-09-10T22:03:03.846819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:47.444100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'72","reported_seq":167,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:48.425868+0000","last_change":"2024-09-10T22:00:30.819427+0000","last_active":"2024-09-10T22:03:48.425868+0000","last_peered":"2024-09-10T22:03:48.425868+0000","last_clean":"2024-09-10T22:03:48.425868+0000","last_became_active":"2024-09-10T22:00:30.819098+0000","last_became_peered":"2024-09-10T22:00:30.819098+0000","last_unstale":"2024-09-10T22:03:48.425868+0000","last_undegraded":"2024-09-10T22:03:48.425868+0000","last_fullsized":"2024-09-10T22:03:48.425868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:23.068155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":172,"num_read_kb":151,"num_write":130,"num_write_kb":1094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722303+0000","last_change":"2024-09-10T22:00:43.760602+0000","last_active":"2024-09-10T22:02:35.722303+0000","last_peered":"2024-09-10T22:02:35.722303+0000","last_clean":"2024-09-10T22:02:35.722303+0000","last_became_active":"2024-09-10T22:00:43.760172+0000","last_became_peered":"2024-09-10T22:00:43.760172+0000","last_unstale":"2024-09-10T22:02:35.722303+0000","last_undegraded":"2024-09-10T22:02:35.722303+0000","last_fullsized":"2024-09-10T22:02:35.722303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:40.332101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.836073+0000","last_change":"2024-09-10T22:00:34.501632+0000","last_active":"2024-09-10T22:03:03.836073+0000","last_peered":"2024-09-10T22:03:03.836073+0000","last_clean":"2024-09-10T22:03:03.836073+0000","last_became_active":"2024-09-10T22:00:34.500924+0000","last_became_peered":"2024-09-10T22:00:34.500924+0000","last_unstale":"2024-09-10T22:03:03.836073+0000","last_undegraded":"2024-09-10T22:03:03.836073+0000","last_fullsized":"2024-09-10T22:03:03.836073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:28.052570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726206+0000","last_change":"2024-09-10T22:00:31.081391+0000","last_active":"2024-09-10T22:02:35.726206+0000","last_peered":"2024-09-10T22:02:35.726206+0000","last_clean":"2024-09-10T22:02:35.726206+0000","last_became_active":"2024-09-10T22:00:31.081018+0000","last_became_peered":"2024-09-10T22:00:31.081018+0000","last_unstale":"2024-09-10T22:02:35.726206+0000","last_undegraded":"2024-09-10T22:02:35.726206+0000","last_fullsized":"2024-09-10T22:02:35.726206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:09.450785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948812+0000","last_change":"2024-09-10T22:00:43.711592+0000","last_active":"2024-09-10T22:02:35.948812+0000","last_peered":"2024-09-10T22:02:35.948812+0000","last_clean":"2024-09-10T22:02:35.948812+0000","last_became_active":"2024-09-10T22:00:43.711144+0000","last_became_peered":"2024-09-10T22:00:43.711144+0000","last_unstale":"2024-09-10T22:02:35.948812+0000","last_undegraded":"2024-09-10T22:02:35.948812+0000","last_fullsized":"2024-09-10T22:02:35.948812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:01.818907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840894+0000","last_change":"2024-09-10T22:00:34.494070+0000","last_active":"2024-09-10T22:03:03.840894+0000","last_peered":"2024-09-10T22:03:03.840894+0000","last_clean":"2024-09-10T22:03:03.840894+0000","last_became_active":"2024-09-10T22:00:34.493324+0000","last_became_peered":"2024-09-10T22:00:34.493324+0000","last_unstale":"2024-09-10T22:03:03.840894+0000","last_undegraded":"2024-09-10T22:03:03.840894+0000","last_fullsized":"2024-09-10T22:03:03.840894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:29.969743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723888+0000","last_change":"2024-09-10T22:00:31.082439+0000","last_active":"2024-09-10T22:02:35.723888+0000","last_peered":"2024-09-10T22:02:35.723888+0000","last_clean":"2024-09-10T22:02:35.723888+0000","last_became_active":"2024-09-10T22:00:31.082025+0000","last_became_peered":"2024-09-10T22:00:31.082025+0000","last_unstale":"2024-09-10T22:02:35.723888+0000","last_undegraded":"2024-09-10T22:02:35.723888+0000","last_fullsized":"2024-09-10T22:02:35.723888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:21.933419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726116+0000","last_change":"2024-09-10T22:00:43.760997+0000","last_active":"2024-09-10T22:02:35.726116+0000","last_peered":"2024-09-10T22:02:35.726116+0000","last_clean":"2024-09-10T22:02:35.726116+0000","last_became_active":"2024-09-10T22:00:43.760124+0000","last_became_peered":"2024-09-10T22:00:43.760124+0000","last_unstale":"2024-09-10T22:02:35.726116+0000","last_undegraded":"2024-09-10T22:02:35.726116+0000","last_fullsized":"2024-09-10T22:02:35.726116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:29.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.843650+0000","last_change":"2024-09-10T22:00:34.511730+0000","last_active":"2024-09-10T22:03:03.843650+0000","last_peered":"2024-09-10T22:03:03.843650+0000","last_clean":"2024-09-10T22:03:03.843650+0000","last_became_active":"2024-09-10T22:00:34.511440+0000","last_became_peered":"2024-09-10T22:00:34.511440+0000","last_unstale":"2024-09-10T22:03:03.843650+0000","last_undegraded":"2024-09-10T22:03:03.843650+0000","last_fullsized":"2024-09-10T22:03:03.843650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:09.053835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'8","reported_seq":194,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:47.981333+0000","last_change":"2024-09-10T22:00:30.825999+0000","last_active":"2024-09-10T22:03:47.981333+0000","last_peered":"2024-09-10T22:03:47.981333+0000","last_clean":"2024-09-10T22:03:47.981333+0000","last_became_active":"2024-09-10T22:00:30.825334+0000","last_became_peered":"2024-09-10T22:00:30.825334+0000","last_unstale":"2024-09-10T22:03:47.981333+0000","last_undegraded":"2024-09-10T22:03:47.981333+0000","last_fullsized":"2024-09-10T22:03:47.981333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:14.684222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":141,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894213+0000","last_change":"2024-09-10T22:00:43.710517+0000","last_active":"2024-09-10T22:02:35.894213+0000","last_peered":"2024-09-10T22:02:35.894213+0000","last_clean":"2024-09-10T22:02:35.894213+0000","last_became_active":"2024-09-10T22:00:43.709252+0000","last_became_peered":"2024-09-10T22:00:43.709252+0000","last_unstale":"2024-09-10T22:02:35.894213+0000","last_undegraded":"2024-09-10T22:02:35.894213+0000","last_fullsized":"2024-09-10T22:02:35.894213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:21.355152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838763+0000","last_change":"2024-09-10T22:02:35.898878+0000","last_active":"2024-09-10T22:03:03.838763+0000","last_peered":"2024-09-10T22:03:03.838763+0000","last_clean":"2024-09-10T22:03:03.838763+0000","last_became_active":"2024-09-10T22:02:35.898531+0000","last_became_peered":"2024-09-10T22:02:35.898531+0000","last_unstale":"2024-09-10T22:03:03.838763+0000","last_undegraded":"2024-09-10T22:03:03.838763+0000","last_fullsized":"2024-09-10T22:03:03.838763+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:02.735208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":188,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:48.037068+0000","last_change":"2024-09-10T22:00:31.081590+0000","last_active":"2024-09-10T22:03:48.037068+0000","last_peered":"2024-09-10T22:03:48.037068+0000","last_clean":"2024-09-10T22:03:48.037068+0000","last_became_active":"2024-09-10T22:00:31.081250+0000","last_became_peered":"2024-09-10T22:00:31.081250+0000","last_unstale":"2024-09-10T22:03:48.037068+0000","last_undegraded":"2024-09-10T22:03:48.037068+0000","last_fullsized":"2024-09-10T22:03:48.037068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:12.872832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893781+0000","last_change":"2024-09-10T22:00:43.761351+0000","last_active":"2024-09-10T22:02:35.893781+0000","last_peered":"2024-09-10T22:02:35.893781+0000","last_clean":"2024-09-10T22:02:35.893781+0000","last_became_active":"2024-09-10T22:00:43.760903+0000","last_became_peered":"2024-09-10T22:00:43.760903+0000","last_unstale":"2024-09-10T22:02:35.893781+0000","last_undegraded":"2024-09-10T22:02:35.893781+0000","last_fullsized":"2024-09-10T22:02:35.893781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:14.113974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838302+0000","last_change":"2024-09-10T22:00:34.504671+0000","last_active":"2024-09-10T22:03:03.838302+0000","last_peered":"2024-09-10T22:03:03.838302+0000","last_clean":"2024-09-10T22:03:03.838302+0000","last_became_active":"2024-09-10T22:00:34.504368+0000","last_became_peered":"2024-09-10T22:00:34.504368+0000","last_unstale":"2024-09-10T22:03:03.838302+0000","last_undegraded":"2024-09-10T22:03:03.838302+0000","last_fullsized":"2024-09-10T22:03:03.838302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:59.771589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863056+0000","last_change":"2024-09-10T22:00:30.820401+0000","last_active":"2024-09-10T22:02:35.863056+0000","last_peered":"2024-09-10T22:02:35.863056+0000","last_clean":"2024-09-10T22:02:35.863056+0000","last_became_active":"2024-09-10T22:00:30.820194+0000","last_became_peered":"2024-09-10T22:00:30.820194+0000","last_unstale":"2024-09-10T22:02:35.863056+0000","last_undegraded":"2024-09-10T22:02:35.863056+0000","last_fullsized":"2024-09-10T22:02:35.863056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:47.767102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727473+0000","last_change":"2024-09-10T22:00:43.762001+0000","last_active":"2024-09-10T22:02:35.727473+0000","last_peered":"2024-09-10T22:02:35.727473+0000","last_clean":"2024-09-10T22:02:35.727473+0000","last_became_active":"2024-09-10T22:00:43.761323+0000","last_became_peered":"2024-09-10T22:00:43.761323+0000","last_unstale":"2024-09-10T22:02:35.727473+0000","last_undegraded":"2024-09-10T22:02:35.727473+0000","last_fullsized":"2024-09-10T22:02:35.727473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:50.113545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.847877+0000","last_change":"2024-09-10T22:00:34.498591+0000","last_active":"2024-09-10T22:03:03.847877+0000","last_peered":"2024-09-10T22:03:03.847877+0000","last_clean":"2024-09-10T22:03:03.847877+0000","last_became_active":"2024-09-10T22:00:34.496449+0000","last_became_peered":"2024-09-10T22:00:34.496449+0000","last_unstale":"2024-09-10T22:03:03.847877+0000","last_undegraded":"2024-09-10T22:03:03.847877+0000","last_fullsized":"2024-09-10T22:03:03.847877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:54.815268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897905+0000","last_change":"2024-09-10T22:00:30.817201+0000","last_active":"2024-09-10T22:02:35.897905+0000","last_peered":"2024-09-10T22:02:35.897905+0000","last_clean":"2024-09-10T22:02:35.897905+0000","last_became_active":"2024-09-10T22:00:30.816792+0000","last_became_peered":"2024-09-10T22:00:30.816792+0000","last_unstale":"2024-09-10T22:02:35.897905+0000","last_undegraded":"2024-09-10T22:02:35.897905+0000","last_fullsized":"2024-09-10T22:02:35.897905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:48.239811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863189+0000","last_change":"2024-09-10T21:59:01.830452+0000","last_active":"2024-09-10T22:02:35.863189+0000","last_peered":"2024-09-10T22:02:35.863189+0000","last_clean":"2024-09-10T22:02:35.863189+0000","last_became_active":"2024-09-10T21:59:01.823744+0000","last_became_peered":"2024-09-10T21:59:01.823744+0000","last_unstale":"2024-09-10T22:02:35.863189+0000","last_undegraded":"2024-09-10T22:02:35.863189+0000","last_fullsized":"2024-09-10T22:02:35.863189+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_clean_scrub_stamp":"2024-09-10T21:54:48.621299+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:39.573257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863562+0000","last_change":"2024-09-10T22:00:43.713888+0000","last_active":"2024-09-10T22:02:35.863562+0000","last_peered":"2024-09-10T22:02:35.863562+0000","last_clean":"2024-09-10T22:02:35.863562+0000","last_became_active":"2024-09-10T22:00:43.713329+0000","last_became_peered":"2024-09-10T22:00:43.713329+0000","last_unstale":"2024-09-10T22:02:35.863562+0000","last_undegraded":"2024-09-10T22:02:35.863562+0000","last_fullsized":"2024-09-10T22:02:35.863562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:01.062089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.852867+0000","last_change":"2024-09-10T22:00:34.502589+0000","last_active":"2024-09-10T22:03:03.852867+0000","last_peered":"2024-09-10T22:03:03.852867+0000","last_clean":"2024-09-10T22:03:03.852867+0000","last_became_active":"2024-09-10T22:00:34.502216+0000","last_became_peered":"2024-09-10T22:00:34.502216+0000","last_unstale":"2024-09-10T22:03:03.852867+0000","last_undegraded":"2024-09-10T22:03:03.852867+0000","last_fullsized":"2024-09-10T22:03:03.852867+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:44.959375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":193,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:47.001644+0000","last_change":"2024-09-10T22:00:31.311807+0000","last_active":"2024-09-10T22:03:47.001644+0000","last_peered":"2024-09-10T22:03:47.001644+0000","last_clean":"2024-09-10T22:03:47.001644+0000","last_became_active":"2024-09-10T22:00:31.311424+0000","last_became_peered":"2024-09-10T22:00:31.311424+0000","last_unstale":"2024-09-10T22:03:47.001644+0000","last_undegraded":"2024-09-10T22:03:47.001644+0000","last_fullsized":"2024-09-10T22:03:47.001644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:09.754777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2195,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948273+0000","last_change":"2024-09-10T22:00:43.711882+0000","last_active":"2024-09-10T22:02:35.948273+0000","last_peered":"2024-09-10T22:02:35.948273+0000","last_clean":"2024-09-10T22:02:35.948273+0000","last_became_active":"2024-09-10T22:00:43.711408+0000","last_became_peered":"2024-09-10T22:00:43.711408+0000","last_unstale":"2024-09-10T22:02:35.948273+0000","last_undegraded":"2024-09-10T22:02:35.948273+0000","last_fullsized":"2024-09-10T22:02:35.948273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:31.155723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841900+0000","last_change":"2024-09-10T22:00:34.505653+0000","last_active":"2024-09-10T22:03:03.841900+0000","last_peered":"2024-09-10T22:03:03.841900+0000","last_clean":"2024-09-10T22:03:03.841900+0000","last_became_active":"2024-09-10T22:00:34.505317+0000","last_became_peered":"2024-09-10T22:00:34.505317+0000","last_unstale":"2024-09-10T22:03:03.841900+0000","last_undegraded":"2024-09-10T22:03:03.841900+0000","last_fullsized":"2024-09-10T22:03:03.841900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:35.050892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727968+0000","last_change":"2024-09-10T22:00:30.817431+0000","last_active":"2024-09-10T22:02:35.727968+0000","last_peered":"2024-09-10T22:02:35.727968+0000","last_clean":"2024-09-10T22:02:35.727968+0000","last_became_active":"2024-09-10T22:00:30.816952+0000","last_became_peered":"2024-09-10T22:00:30.816952+0000","last_unstale":"2024-09-10T22:02:35.727968+0000","last_undegraded":"2024-09-10T22:02:35.727968+0000","last_fullsized":"2024-09-10T22:02:35.727968+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:37.881641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947502+0000","last_change":"2024-09-10T22:00:43.720627+0000","last_active":"2024-09-10T22:02:35.947502+0000","last_peered":"2024-09-10T22:02:35.947502+0000","last_clean":"2024-09-10T22:02:35.947502+0000","last_became_active":"2024-09-10T22:00:43.720235+0000","last_became_peered":"2024-09-10T22:00:43.720235+0000","last_unstale":"2024-09-10T22:02:35.947502+0000","last_undegraded":"2024-09-10T22:02:35.947502+0000","last_fullsized":"2024-09-10T22:02:35.947502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:50.549536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841647+0000","last_change":"2024-09-10T22:00:34.501649+0000","last_active":"2024-09-10T22:03:03.841647+0000","last_peered":"2024-09-10T22:03:03.841647+0000","last_clean":"2024-09-10T22:03:03.841647+0000","last_became_active":"2024-09-10T22:00:34.498882+0000","last_became_peered":"2024-09-10T22:00:34.498882+0000","last_unstale":"2024-09-10T22:03:03.841647+0000","last_undegraded":"2024-09-10T22:03:03.841647+0000","last_fullsized":"2024-09-10T22:03:03.841647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894508+0000","last_change":"2024-09-10T22:00:30.821943+0000","last_active":"2024-09-10T22:02:35.894508+0000","last_peered":"2024-09-10T22:02:35.894508+0000","last_clean":"2024-09-10T22:02:35.894508+0000","last_became_active":"2024-09-10T22:00:30.821372+0000","last_became_peered":"2024-09-10T22:00:30.821372+0000","last_unstale":"2024-09-10T22:02:35.894508+0000","last_undegraded":"2024-09-10T22:02:35.894508+0000","last_fullsized":"2024-09-10T22:02:35.894508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:04.360383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727340+0000","last_change":"2024-09-10T22:00:43.710045+0000","last_active":"2024-09-10T22:02:35.727340+0000","last_peered":"2024-09-10T22:02:35.727340+0000","last_clean":"2024-09-10T22:02:35.727340+0000","last_became_active":"2024-09-10T22:00:43.709416+0000","last_became_peered":"2024-09-10T22:00:43.709416+0000","last_unstale":"2024-09-10T22:02:35.727340+0000","last_undegraded":"2024-09-10T22:02:35.727340+0000","last_fullsized":"2024-09-10T22:02:35.727340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:56.005549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.834717+0000","last_change":"2024-09-10T22:00:34.496163+0000","last_active":"2024-09-10T22:03:03.834717+0000","last_peered":"2024-09-10T22:03:03.834717+0000","last_clean":"2024-09-10T22:03:03.834717+0000","last_became_active":"2024-09-10T22:00:34.495814+0000","last_became_peered":"2024-09-10T22:00:34.495814+0000","last_unstale":"2024-09-10T22:03:03.834717+0000","last_undegraded":"2024-09-10T22:03:03.834717+0000","last_fullsized":"2024-09-10T22:03:03.834717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:05.398154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:45.682374+0000","last_change":"2024-09-10T22:00:30.812702+0000","last_active":"2024-09-10T22:02:45.682374+0000","last_peered":"2024-09-10T22:02:45.682374+0000","last_clean":"2024-09-10T22:02:45.682374+0000","last_became_active":"2024-09-10T22:00:30.812369+0000","last_became_peered":"2024-09-10T22:00:30.812369+0000","last_unstale":"2024-09-10T22:02:45.682374+0000","last_undegraded":"2024-09-10T22:02:45.682374+0000","last_fullsized":"2024-09-10T22:02:45.682374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:52.332876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727291+0000","last_change":"2024-09-10T22:00:43.715573+0000","last_active":"2024-09-10T22:02:35.727291+0000","last_peered":"2024-09-10T22:02:35.727291+0000","last_clean":"2024-09-10T22:02:35.727291+0000","last_became_active":"2024-09-10T22:00:43.714475+0000","last_became_peered":"2024-09-10T22:00:43.714475+0000","last_unstale":"2024-09-10T22:02:35.727291+0000","last_undegraded":"2024-09-10T22:02:35.727291+0000","last_fullsized":"2024-09-10T22:02:35.727291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:55.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.844868+0000","last_change":"2024-09-10T22:02:35.949769+0000","last_active":"2024-09-10T22:03:03.844868+0000","last_peered":"2024-09-10T22:03:03.844868+0000","last_clean":"2024-09-10T22:03:03.844868+0000","last_became_active":"2024-09-10T22:02:35.949440+0000","last_became_peered":"2024-09-10T22:02:35.949440+0000","last_unstale":"2024-09-10T22:03:03.844868+0000","last_undegraded":"2024-09-10T22:03:03.844868+0000","last_fullsized":"2024-09-10T22:03:03.844868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:03.639412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948309+0000","last_change":"2024-09-10T22:01:35.903381+0000","last_active":"2024-09-10T22:02:35.948309+0000","last_peered":"2024-09-10T22:02:35.948309+0000","last_clean":"2024-09-10T22:02:35.948309+0000","last_became_active":"2024-09-10T22:01:35.878262+0000","last_became_peered":"2024-09-10T22:01:35.878262+0000","last_unstale":"2024-09-10T22:02:35.948309+0000","last_undegraded":"2024-09-10T22:02:35.948309+0000","last_fullsized":"2024-09-10T22:02:35.948309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:45.419449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893771+0000","last_change":"2024-09-10T22:00:43.720728+0000","last_active":"2024-09-10T22:02:35.893771+0000","last_peered":"2024-09-10T22:02:35.893771+0000","last_clean":"2024-09-10T22:02:35.893771+0000","last_became_active":"2024-09-10T22:00:43.720305+0000","last_became_peered":"2024-09-10T22:00:43.720305+0000","last_unstale":"2024-09-10T22:02:35.893771+0000","last_undegraded":"2024-09-10T22:02:35.893771+0000","last_fullsized":"2024-09-10T22:02:35.893771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:54.986822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841861+0000","last_change":"2024-09-10T22:00:34.501571+0000","last_active":"2024-09-10T22:03:03.841861+0000","last_peered":"2024-09-10T22:03:03.841861+0000","last_clean":"2024-09-10T22:03:03.841861+0000","last_became_active":"2024-09-10T22:00:34.501211+0000","last_became_peered":"2024-09-10T22:00:34.501211+0000","last_unstale":"2024-09-10T22:03:03.841861+0000","last_undegraded":"2024-09-10T22:03:03.841861+0000","last_fullsized":"2024-09-10T22:03:03.841861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:05.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":194,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:46.034697+0000","last_change":"2024-09-10T22:00:30.824296+0000","last_active":"2024-09-10T22:03:46.034697+0000","last_peered":"2024-09-10T22:03:46.034697+0000","last_clean":"2024-09-10T22:03:46.034697+0000","last_became_active":"2024-09-10T22:00:30.823886+0000","last_became_peered":"2024-09-10T22:00:30.823886+0000","last_unstale":"2024-09-10T22:03:46.034697+0000","last_undegraded":"2024-09-10T22:03:46.034697+0000","last_fullsized":"2024-09-10T22:03:46.034697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:26.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724371+0000","last_change":"2024-09-10T22:00:43.717580+0000","last_active":"2024-09-10T22:02:35.724371+0000","last_peered":"2024-09-10T22:02:35.724371+0000","last_clean":"2024-09-10T22:02:35.724371+0000","last_became_active":"2024-09-10T22:00:43.708377+0000","last_became_peered":"2024-09-10T22:00:43.708377+0000","last_unstale":"2024-09-10T22:02:35.724371+0000","last_undegraded":"2024-09-10T22:02:35.724371+0000","last_fullsized":"2024-09-10T22:02:35.724371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:54.682292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.848412+0000","last_change":"2024-09-10T22:00:34.514348+0000","last_active":"2024-09-10T22:03:03.848412+0000","last_peered":"2024-09-10T22:03:03.848412+0000","last_clean":"2024-09-10T22:03:03.848412+0000","last_became_active":"2024-09-10T22:00:34.513797+0000","last_became_peered":"2024-09-10T22:00:34.513797+0000","last_unstale":"2024-09-10T22:03:03.848412+0000","last_undegraded":"2024-09-10T22:03:03.848412+0000","last_fullsized":"2024-09-10T22:03:03.848412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:12.797643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728093+0000","last_change":"2024-09-10T22:00:30.826486+0000","last_active":"2024-09-10T22:02:35.728093+0000","last_peered":"2024-09-10T22:02:35.728093+0000","last_clean":"2024-09-10T22:02:35.728093+0000","last_became_active":"2024-09-10T22:00:30.826111+0000","last_became_peered":"2024-09-10T22:00:30.826111+0000","last_unstale":"2024-09-10T22:02:35.728093+0000","last_undegraded":"2024-09-10T22:02:35.728093+0000","last_fullsized":"2024-09-10T22:02:35.728093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:45.609925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948482+0000","last_change":"2024-09-10T22:00:43.709432+0000","last_active":"2024-09-10T22:02:35.948482+0000","last_peered":"2024-09-10T22:02:35.948482+0000","last_clean":"2024-09-10T22:02:35.948482+0000","last_became_active":"2024-09-10T22:00:43.708627+0000","last_became_peered":"2024-09-10T22:00:43.708627+0000","last_unstale":"2024-09-10T22:02:35.948482+0000","last_undegraded":"2024-09-10T22:02:35.948482+0000","last_fullsized":"2024-09-10T22:02:35.948482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:12.799976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.848267+0000","last_change":"2024-09-10T22:00:34.499203+0000","last_active":"2024-09-10T22:03:03.848267+0000","last_peered":"2024-09-10T22:03:03.848267+0000","last_clean":"2024-09-10T22:03:03.848267+0000","last_became_active":"2024-09-10T22:00:34.497877+0000","last_became_peered":"2024-09-10T22:00:34.497877+0000","last_unstale":"2024-09-10T22:03:03.848267+0000","last_undegraded":"2024-09-10T22:03:03.848267+0000","last_fullsized":"2024-09-10T22:03:03.848267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:41.897603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723973+0000","last_change":"2024-09-10T22:00:30.820852+0000","last_active":"2024-09-10T22:02:35.723973+0000","last_peered":"2024-09-10T22:02:35.723973+0000","last_clean":"2024-09-10T22:02:35.723973+0000","last_became_active":"2024-09-10T22:00:30.820023+0000","last_became_peered":"2024-09-10T22:00:30.820023+0000","last_unstale":"2024-09-10T22:02:35.723973+0000","last_undegraded":"2024-09-10T22:02:35.723973+0000","last_fullsized":"2024-09-10T22:02:35.723973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:54.245142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727813+0000","last_change":"2024-09-10T22:00:43.716323+0000","last_active":"2024-09-10T22:02:35.727813+0000","last_peered":"2024-09-10T22:02:35.727813+0000","last_clean":"2024-09-10T22:02:35.727813+0000","last_became_active":"2024-09-10T22:00:43.715827+0000","last_became_peered":"2024-09-10T22:00:43.715827+0000","last_unstale":"2024-09-10T22:02:35.727813+0000","last_undegraded":"2024-09-10T22:02:35.727813+0000","last_fullsized":"2024-09-10T22:02:35.727813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:10.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.839342+0000","last_change":"2024-09-10T22:00:34.500114+0000","last_active":"2024-09-10T22:03:03.839342+0000","last_peered":"2024-09-10T22:03:03.839342+0000","last_clean":"2024-09-10T22:03:03.839342+0000","last_became_active":"2024-09-10T22:00:34.499334+0000","last_became_peered":"2024-09-10T22:00:34.499334+0000","last_unstale":"2024-09-10T22:03:03.839342+0000","last_undegraded":"2024-09-10T22:03:03.839342+0000","last_fullsized":"2024-09-10T22:03:03.839342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:11.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":369,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:49.006270+0000","last_change":"2024-09-10T22:00:30.822561+0000","last_active":"2024-09-10T22:03:49.006270+0000","last_peered":"2024-09-10T22:03:49.006270+0000","last_clean":"2024-09-10T22:03:49.006270+0000","last_became_active":"2024-09-10T22:00:30.822201+0000","last_became_peered":"2024-09-10T22:00:30.822201+0000","last_unstale":"2024-09-10T22:03:49.006270+0000","last_undegraded":"2024-09-10T22:03:49.006270+0000","last_fullsized":"2024-09-10T22:03:49.006270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:41.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67713,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":209,"num_write":42,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871420+0000","last_change":"2024-09-10T22:01:35.444988+0000","last_active":"2024-09-10T22:02:35.871420+0000","last_peered":"2024-09-10T22:02:35.871420+0000","last_clean":"2024-09-10T22:02:35.871420+0000","last_became_active":"2024-09-10T22:01:35.444498+0000","last_became_peered":"2024-09-10T22:01:35.444498+0000","last_unstale":"2024-09-10T22:02:35.871420+0000","last_undegraded":"2024-09-10T22:02:35.871420+0000","last_fullsized":"2024-09-10T22:02:35.871420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:35.017660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.852407+0000","last_change":"2024-09-10T22:00:34.503234+0000","last_active":"2024-09-10T22:03:03.852407+0000","last_peered":"2024-09-10T22:03:03.852407+0000","last_clean":"2024-09-10T22:03:03.852407+0000","last_became_active":"2024-09-10T22:00:34.501501+0000","last_became_peered":"2024-09-10T22:00:34.501501+0000","last_unstale":"2024-09-10T22:03:03.852407+0000","last_undegraded":"2024-09-10T22:03:03.852407+0000","last_fullsized":"2024-09-10T22:03:03.852407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:23.772491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728102+0000","last_change":"2024-09-10T22:00:31.082131+0000","last_active":"2024-09-10T22:02:35.728102+0000","last_peered":"2024-09-10T22:02:35.728102+0000","last_clean":"2024-09-10T22:02:35.728102+0000","last_became_active":"2024-09-10T22:00:31.081405+0000","last_became_peered":"2024-09-10T22:00:31.081405+0000","last_unstale":"2024-09-10T22:02:35.728102+0000","last_undegraded":"2024-09-10T22:02:35.728102+0000","last_fullsized":"2024-09-10T22:02:35.728102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:02.400004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897415+0000","last_change":"2024-09-10T22:00:43.705732+0000","last_active":"2024-09-10T22:02:35.897415+0000","last_peered":"2024-09-10T22:02:35.897415+0000","last_clean":"2024-09-10T22:02:35.897415+0000","last_became_active":"2024-09-10T22:00:43.705274+0000","last_became_peered":"2024-09-10T22:00:43.705274+0000","last_unstale":"2024-09-10T22:02:35.897415+0000","last_undegraded":"2024-09-10T22:02:35.897415+0000","last_fullsized":"2024-09-10T22:02:35.897415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:54.707636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.845781+0000","last_change":"2024-09-10T22:00:34.499585+0000","last_active":"2024-09-10T22:03:03.845781+0000","last_peered":"2024-09-10T22:03:03.845781+0000","last_clean":"2024-09-10T22:03:03.845781+0000","last_became_active":"2024-09-10T22:00:34.499261+0000","last_became_peered":"2024-09-10T22:00:34.499261+0000","last_unstale":"2024-09-10T22:03:03.845781+0000","last_undegraded":"2024-09-10T22:03:03.845781+0000","last_fullsized":"2024-09-10T22:03:03.845781+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:41.156093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863703+0000","last_change":"2024-09-10T22:00:30.823322+0000","last_active":"2024-09-10T22:02:35.863703+0000","last_peered":"2024-09-10T22:02:35.863703+0000","last_clean":"2024-09-10T22:02:35.863703+0000","last_became_active":"2024-09-10T22:00:30.822987+0000","last_became_peered":"2024-09-10T22:00:30.822987+0000","last_unstale":"2024-09-10T22:02:35.863703+0000","last_undegraded":"2024-09-10T22:02:35.863703+0000","last_fullsized":"2024-09-10T22:02:35.863703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:33.605627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723255+0000","last_change":"2024-09-10T22:00:43.708746+0000","last_active":"2024-09-10T22:02:35.723255+0000","last_peered":"2024-09-10T22:02:35.723255+0000","last_clean":"2024-09-10T22:02:35.723255+0000","last_became_active":"2024-09-10T22:00:43.708283+0000","last_became_peered":"2024-09-10T22:00:43.708283+0000","last_unstale":"2024-09-10T22:02:35.723255+0000","last_undegraded":"2024-09-10T22:02:35.723255+0000","last_fullsized":"2024-09-10T22:02:35.723255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:30.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.854268+0000","last_change":"2024-09-10T22:00:34.501472+0000","last_active":"2024-09-10T22:03:03.854268+0000","last_peered":"2024-09-10T22:03:03.854268+0000","last_clean":"2024-09-10T22:03:03.854268+0000","last_became_active":"2024-09-10T22:00:34.501131+0000","last_became_peered":"2024-09-10T22:00:34.501131+0000","last_unstale":"2024-09-10T22:03:03.854268+0000","last_undegraded":"2024-09-10T22:03:03.854268+0000","last_fullsized":"2024-09-10T22:03:03.854268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:11.281413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'8","reported_seq":326,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:49.004930+0000","last_change":"2024-09-10T22:00:30.828335+0000","last_active":"2024-09-10T22:03:49.004930+0000","last_peered":"2024-09-10T22:03:49.004930+0000","last_clean":"2024-09-10T22:03:49.004930+0000","last_became_active":"2024-09-10T22:00:30.828028+0000","last_became_peered":"2024-09-10T22:00:30.828028+0000","last_unstale":"2024-09-10T22:03:49.004930+0000","last_undegraded":"2024-09-10T22:03:49.004930+0000","last_fullsized":"2024-09-10T22:03:49.004930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:56.399002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":274,"num_read_kb":274,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722469+0000","last_change":"2024-09-10T22:00:43.710086+0000","last_active":"2024-09-10T22:02:35.722469+0000","last_peered":"2024-09-10T22:02:35.722469+0000","last_clean":"2024-09-10T22:02:35.722469+0000","last_became_active":"2024-09-10T22:00:43.709518+0000","last_became_peered":"2024-09-10T22:00:43.709518+0000","last_unstale":"2024-09-10T22:02:35.722469+0000","last_undegraded":"2024-09-10T22:02:35.722469+0000","last_fullsized":"2024-09-10T22:02:35.722469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:07.636887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947965+0000","last_change":"2024-09-10T22:00:43.711620+0000","last_active":"2024-09-10T22:02:35.947965+0000","last_peered":"2024-09-10T22:02:35.947965+0000","last_clean":"2024-09-10T22:02:35.947965+0000","last_became_active":"2024-09-10T22:00:43.711224+0000","last_became_peered":"2024-09-10T22:00:43.711224+0000","last_unstale":"2024-09-10T22:02:35.947965+0000","last_undegraded":"2024-09-10T22:02:35.947965+0000","last_fullsized":"2024-09-10T22:02:35.947965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:43.860286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841267+0000","last_change":"2024-09-10T22:00:34.494408+0000","last_active":"2024-09-10T22:03:03.841267+0000","last_peered":"2024-09-10T22:03:03.841267+0000","last_clean":"2024-09-10T22:03:03.841267+0000","last_became_active":"2024-09-10T22:00:34.494066+0000","last_became_peered":"2024-09-10T22:00:34.494066+0000","last_unstale":"2024-09-10T22:03:03.841267+0000","last_undegraded":"2024-09-10T22:03:03.841267+0000","last_fullsized":"2024-09-10T22:03:03.841267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:49.565755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727709+0000","last_change":"2024-09-10T22:00:30.821842+0000","last_active":"2024-09-10T22:02:35.727709+0000","last_peered":"2024-09-10T22:02:35.727709+0000","last_clean":"2024-09-10T22:02:35.727709+0000","last_became_active":"2024-09-10T22:00:30.821499+0000","last_became_peered":"2024-09-10T22:00:30.821499+0000","last_unstale":"2024-09-10T22:02:35.727709+0000","last_undegraded":"2024-09-10T22:02:35.727709+0000","last_fullsized":"2024-09-10T22:02:35.727709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:19.125052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948928+0000","last_change":"2024-09-10T22:00:43.706689+0000","last_active":"2024-09-10T22:02:35.948928+0000","last_peered":"2024-09-10T22:02:35.948928+0000","last_clean":"2024-09-10T22:02:35.948928+0000","last_became_active":"2024-09-10T22:00:43.705954+0000","last_became_peered":"2024-09-10T22:00:43.705954+0000","last_unstale":"2024-09-10T22:02:35.948928+0000","last_undegraded":"2024-09-10T22:02:35.948928+0000","last_fullsized":"2024-09-10T22:02:35.948928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:30.800710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.831092+0000","last_change":"2024-09-10T22:00:34.490853+0000","last_active":"2024-09-10T22:03:03.831092+0000","last_peered":"2024-09-10T22:03:03.831092+0000","last_clean":"2024-09-10T22:03:03.831092+0000","last_became_active":"2024-09-10T22:00:34.489908+0000","last_became_peered":"2024-09-10T22:00:34.489908+0000","last_unstale":"2024-09-10T22:03:03.831092+0000","last_undegraded":"2024-09-10T22:03:03.831092+0000","last_fullsized":"2024-09-10T22:03:03.831092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:57.105004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893936+0000","last_change":"2024-09-10T22:00:30.816497+0000","last_active":"2024-09-10T22:02:35.893936+0000","last_peered":"2024-09-10T22:02:35.893936+0000","last_clean":"2024-09-10T22:02:35.893936+0000","last_became_active":"2024-09-10T22:00:30.816181+0000","last_became_peered":"2024-09-10T22:00:30.816181+0000","last_unstale":"2024-09-10T22:02:35.893936+0000","last_undegraded":"2024-09-10T22:02:35.893936+0000","last_fullsized":"2024-09-10T22:02:35.893936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:12.779113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871850+0000","last_change":"2024-09-10T22:00:30.821402+0000","last_active":"2024-09-10T22:02:35.871850+0000","last_peered":"2024-09-10T22:02:35.871850+0000","last_clean":"2024-09-10T22:02:35.871850+0000","last_became_active":"2024-09-10T22:00:30.821105+0000","last_became_peered":"2024-09-10T22:00:30.821105+0000","last_unstale":"2024-09-10T22:02:35.871850+0000","last_undegraded":"2024-09-10T22:02:35.871850+0000","last_fullsized":"2024-09-10T22:02:35.871850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:36.090579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840274+0000","last_change":"2024-09-10T22:00:34.511911+0000","last_active":"2024-09-10T22:03:03.840274+0000","last_peered":"2024-09-10T22:03:03.840274+0000","last_clean":"2024-09-10T22:03:03.840274+0000","last_became_active":"2024-09-10T22:00:34.511545+0000","last_became_peered":"2024-09-10T22:00:34.511545+0000","last_unstale":"2024-09-10T22:03:03.840274+0000","last_undegraded":"2024-09-10T22:03:03.840274+0000","last_fullsized":"2024-09-10T22:03:03.840274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:12.484733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723568+0000","last_change":"2024-09-10T22:01:35.750458+0000","last_active":"2024-09-10T22:02:35.723568+0000","last_peered":"2024-09-10T22:02:35.723568+0000","last_clean":"2024-09-10T22:02:35.723568+0000","last_became_active":"2024-09-10T22:01:35.749946+0000","last_became_peered":"2024-09-10T22:01:35.749946+0000","last_unstale":"2024-09-10T22:02:35.723568+0000","last_undegraded":"2024-09-10T22:02:35.723568+0000","last_fullsized":"2024-09-10T22:02:35.723568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.362221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":193,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:46.000213+0000","last_change":"2024-09-10T22:00:30.826427+0000","last_active":"2024-09-10T22:03:46.000213+0000","last_peered":"2024-09-10T22:03:46.000213+0000","last_clean":"2024-09-10T22:03:46.000213+0000","last_became_active":"2024-09-10T22:00:30.826038+0000","last_became_peered":"2024-09-10T22:00:30.826038+0000","last_unstale":"2024-09-10T22:03:46.000213+0000","last_undegraded":"2024-09-10T22:03:46.000213+0000","last_fullsized":"2024-09-10T22:03:46.000213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:45.075208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":140,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.833092+0000","last_change":"2024-09-10T22:00:34.494479+0000","last_active":"2024-09-10T22:03:03.833092+0000","last_peered":"2024-09-10T22:03:03.833092+0000","last_clean":"2024-09-10T22:03:03.833092+0000","last_became_active":"2024-09-10T22:00:34.494087+0000","last_became_peered":"2024-09-10T22:00:34.494087+0000","last_unstale":"2024-09-10T22:03:03.833092+0000","last_undegraded":"2024-09-10T22:03:03.833092+0000","last_fullsized":"2024-09-10T22:03:03.833092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:28.548999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723643+0000","last_change":"2024-09-10T22:00:43.714568+0000","last_active":"2024-09-10T22:02:35.723643+0000","last_peered":"2024-09-10T22:02:35.723643+0000","last_clean":"2024-09-10T22:02:35.723643+0000","last_became_active":"2024-09-10T22:00:43.712923+0000","last_became_peered":"2024-09-10T22:00:43.712923+0000","last_unstale":"2024-09-10T22:02:35.723643+0000","last_undegraded":"2024-09-10T22:02:35.723643+0000","last_fullsized":"2024-09-10T22:02:35.723643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:27.945799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.833182+0000","last_change":"2024-09-10T22:00:34.496942+0000","last_active":"2024-09-10T22:03:03.833182+0000","last_peered":"2024-09-10T22:03:03.833182+0000","last_clean":"2024-09-10T22:03:03.833182+0000","last_became_active":"2024-09-10T22:00:34.496475+0000","last_became_peered":"2024-09-10T22:00:34.496475+0000","last_unstale":"2024-09-10T22:03:03.833182+0000","last_undegraded":"2024-09-10T22:03:03.833182+0000","last_fullsized":"2024-09-10T22:03:03.833182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:33.277346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837137+0000","last_change":"2024-09-10T22:00:34.506571+0000","last_active":"2024-09-10T22:03:03.837137+0000","last_peered":"2024-09-10T22:03:03.837137+0000","last_clean":"2024-09-10T22:03:03.837137+0000","last_became_active":"2024-09-10T22:00:34.505718+0000","last_became_peered":"2024-09-10T22:00:34.505718+0000","last_unstale":"2024-09-10T22:03:03.837137+0000","last_undegraded":"2024-09-10T22:03:03.837137+0000","last_fullsized":"2024-09-10T22:03:03.837137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:10.111415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837418+0000","last_change":"2024-09-10T22:00:34.502030+0000","last_active":"2024-09-10T22:03:03.837418+0000","last_peered":"2024-09-10T22:03:03.837418+0000","last_clean":"2024-09-10T22:03:03.837418+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:03:03.837418+0000","last_undegraded":"2024-09-10T22:03:03.837418+0000","last_fullsized":"2024-09-10T22:03:03.837418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:09.532523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.846513+0000","last_change":"2024-09-10T22:00:34.513630+0000","last_active":"2024-09-10T22:03:03.846513+0000","last_peered":"2024-09-10T22:03:03.846513+0000","last_clean":"2024-09-10T22:03:03.846513+0000","last_became_active":"2024-09-10T22:00:34.513348+0000","last_became_peered":"2024-09-10T22:00:34.513348+0000","last_unstale":"2024-09-10T22:03:03.846513+0000","last_undegraded":"2024-09-10T22:03:03.846513+0000","last_fullsized":"2024-09-10T22:03:03.846513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:53.942004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.844287+0000","last_change":"2024-09-10T22:00:34.508697+0000","last_active":"2024-09-10T22:03:03.844287+0000","last_peered":"2024-09-10T22:03:03.844287+0000","last_clean":"2024-09-10T22:03:03.844287+0000","last_became_active":"2024-09-10T22:00:34.508371+0000","last_became_peered":"2024-09-10T22:00:34.508371+0000","last_unstale":"2024-09-10T22:03:03.844287+0000","last_undegraded":"2024-09-10T22:03:03.844287+0000","last_fullsized":"2024-09-10T22:03:03.844287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:32.843517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.832359+0000","last_change":"2024-09-10T22:00:34.496021+0000","last_active":"2024-09-10T22:03:03.832359+0000","last_peered":"2024-09-10T22:03:03.832359+0000","last_clean":"2024-09-10T22:03:03.832359+0000","last_became_active":"2024-09-10T22:00:34.495663+0000","last_became_peered":"2024-09-10T22:00:34.495663+0000","last_unstale":"2024-09-10T22:03:03.832359+0000","last_undegraded":"2024-09-10T22:03:03.832359+0000","last_fullsized":"2024-09-10T22:03:03.832359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:16.512182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851192+0000","last_change":"2024-09-10T22:00:34.506589+0000","last_active":"2024-09-10T22:03:03.851192+0000","last_peered":"2024-09-10T22:03:03.851192+0000","last_clean":"2024-09-10T22:03:03.851192+0000","last_became_active":"2024-09-10T22:00:34.506214+0000","last_became_peered":"2024-09-10T22:00:34.506214+0000","last_unstale":"2024-09-10T22:03:03.851192+0000","last_undegraded":"2024-09-10T22:03:03.851192+0000","last_fullsized":"2024-09-10T22:03:03.851192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:47.072458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.825874+0000","last_change":"2024-09-10T22:00:34.503471+0000","last_active":"2024-09-10T22:03:03.825874+0000","last_peered":"2024-09-10T22:03:03.825874+0000","last_clean":"2024-09-10T22:03:03.825874+0000","last_became_active":"2024-09-10T22:00:34.501145+0000","last_became_peered":"2024-09-10T22:00:34.501145+0000","last_unstale":"2024-09-10T22:03:03.825874+0000","last_undegraded":"2024-09-10T22:03:03.825874+0000","last_fullsized":"2024-09-10T22:03:03.825874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:24.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840484+0000","last_change":"2024-09-10T22:00:34.490177+0000","last_active":"2024-09-10T22:03:03.840484+0000","last_peered":"2024-09-10T22:03:03.840484+0000","last_clean":"2024-09-10T22:03:03.840484+0000","last_became_active":"2024-09-10T22:00:34.488428+0000","last_became_peered":"2024-09-10T22:00:34.488428+0000","last_unstale":"2024-09-10T22:03:03.840484+0000","last_undegraded":"2024-09-10T22:03:03.840484+0000","last_fullsized":"2024-09-10T22:03:03.840484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:21.111826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841640+0000","last_change":"2024-09-10T22:00:34.499174+0000","last_active":"2024-09-10T22:03:03.841640+0000","last_peered":"2024-09-10T22:03:03.841640+0000","last_clean":"2024-09-10T22:03:03.841640+0000","last_became_active":"2024-09-10T22:00:34.498824+0000","last_became_peered":"2024-09-10T22:00:34.498824+0000","last_unstale":"2024-09-10T22:03:03.841640+0000","last_undegraded":"2024-09-10T22:03:03.841640+0000","last_fullsized":"2024-09-10T22:03:03.841640+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:15.946270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853015+0000","last_change":"2024-09-10T22:00:34.505748+0000","last_active":"2024-09-10T22:03:03.853015+0000","last_peered":"2024-09-10T22:03:03.853015+0000","last_clean":"2024-09-10T22:03:03.853015+0000","last_became_active":"2024-09-10T22:00:34.500441+0000","last_became_peered":"2024-09-10T22:00:34.500441+0000","last_unstale":"2024-09-10T22:03:03.853015+0000","last_undegraded":"2024-09-10T22:03:03.853015+0000","last_fullsized":"2024-09-10T22:03:03.853015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:22.177566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.850314+0000","last_change":"2024-09-10T22:00:34.502319+0000","last_active":"2024-09-10T22:03:03.850314+0000","last_peered":"2024-09-10T22:03:03.850314+0000","last_clean":"2024-09-10T22:03:03.850314+0000","last_became_active":"2024-09-10T22:00:34.501427+0000","last_became_peered":"2024-09-10T22:00:34.501427+0000","last_unstale":"2024-09-10T22:03:03.850314+0000","last_undegraded":"2024-09-10T22:03:03.850314+0000","last_fullsized":"2024-09-10T22:03:03.850314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:18.463319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.842219+0000","last_change":"2024-09-10T22:00:34.497571+0000","last_active":"2024-09-10T22:03:03.842219+0000","last_peered":"2024-09-10T22:03:03.842219+0000","last_clean":"2024-09-10T22:03:03.842219+0000","last_became_active":"2024-09-10T22:00:34.497205+0000","last_became_peered":"2024-09-10T22:00:34.497205+0000","last_unstale":"2024-09-10T22:03:03.842219+0000","last_undegraded":"2024-09-10T22:03:03.842219+0000","last_fullsized":"2024-09-10T22:03:03.842219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:49.924518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837147+0000","last_change":"2024-09-10T22:00:34.504641+0000","last_active":"2024-09-10T22:03:03.837147+0000","last_peered":"2024-09-10T22:03:03.837147+0000","last_clean":"2024-09-10T22:03:03.837147+0000","last_became_active":"2024-09-10T22:00:34.504183+0000","last_became_peered":"2024-09-10T22:00:34.504183+0000","last_unstale":"2024-09-10T22:03:03.837147+0000","last_undegraded":"2024-09-10T22:03:03.837147+0000","last_fullsized":"2024-09-10T22:03:03.837147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:06.704972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851050+0000","last_change":"2024-09-10T22:00:34.503884+0000","last_active":"2024-09-10T22:03:03.851050+0000","last_peered":"2024-09-10T22:03:03.851050+0000","last_clean":"2024-09-10T22:03:03.851050+0000","last_became_active":"2024-09-10T22:00:34.502950+0000","last_became_peered":"2024-09-10T22:00:34.502950+0000","last_unstale":"2024-09-10T22:03:03.851050+0000","last_undegraded":"2024-09-10T22:03:03.851050+0000","last_fullsized":"2024-09-10T22:03:03.851050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:45.560958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.850635+0000","last_change":"2024-09-10T22:00:34.504981+0000","last_active":"2024-09-10T22:03:03.850635+0000","last_peered":"2024-09-10T22:03:03.850635+0000","last_clean":"2024-09-10T22:03:03.850635+0000","last_became_active":"2024-09-10T22:00:34.500474+0000","last_became_peered":"2024-09-10T22:00:34.500474+0000","last_unstale":"2024-09-10T22:03:03.850635+0000","last_undegraded":"2024-09-10T22:03:03.850635+0000","last_fullsized":"2024-09-10T22:03:03.850635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:20.892605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853045+0000","last_change":"2024-09-10T22:00:34.506650+0000","last_active":"2024-09-10T22:03:03.853045+0000","last_peered":"2024-09-10T22:03:03.853045+0000","last_clean":"2024-09-10T22:03:03.853045+0000","last_became_active":"2024-09-10T22:00:34.505698+0000","last_became_peered":"2024-09-10T22:00:34.505698+0000","last_unstale":"2024-09-10T22:03:03.853045+0000","last_undegraded":"2024-09-10T22:03:03.853045+0000","last_fullsized":"2024-09-10T22:03:03.853045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:29.221693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841202+0000","last_change":"2024-09-10T22:00:34.502238+0000","last_active":"2024-09-10T22:03:03.841202+0000","last_peered":"2024-09-10T22:03:03.841202+0000","last_clean":"2024-09-10T22:03:03.841202+0000","last_became_active":"2024-09-10T22:00:34.500049+0000","last_became_peered":"2024-09-10T22:00:34.500049+0000","last_unstale":"2024-09-10T22:03:03.841202+0000","last_undegraded":"2024-09-10T22:03:03.841202+0000","last_fullsized":"2024-09-10T22:03:03.841202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:05.669784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.839971+0000","last_change":"2024-09-10T22:00:34.499340+0000","last_active":"2024-09-10T22:03:03.839971+0000","last_peered":"2024-09-10T22:03:03.839971+0000","last_clean":"2024-09-10T22:03:03.839971+0000","last_became_active":"2024-09-10T22:00:34.497655+0000","last_became_peered":"2024-09-10T22:00:34.497655+0000","last_unstale":"2024-09-10T22:03:03.839971+0000","last_undegraded":"2024-09-10T22:03:03.839971+0000","last_fullsized":"2024-09-10T22:03:03.839971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:34.115829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838094+0000","last_change":"2024-09-10T22:00:34.500705+0000","last_active":"2024-09-10T22:03:03.838094+0000","last_peered":"2024-09-10T22:03:03.838094+0000","last_clean":"2024-09-10T22:03:03.838094+0000","last_became_active":"2024-09-10T22:00:34.500071+0000","last_became_peered":"2024-09-10T22:00:34.500071+0000","last_unstale":"2024-09-10T22:03:03.838094+0000","last_undegraded":"2024-09-10T22:03:03.838094+0000","last_fullsized":"2024-09-10T22:03:03.838094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:40.519976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838120+0000","last_change":"2024-09-10T22:00:34.510570+0000","last_active":"2024-09-10T22:03:03.838120+0000","last_peered":"2024-09-10T22:03:03.838120+0000","last_clean":"2024-09-10T22:03:03.838120+0000","last_became_active":"2024-09-10T22:00:34.506922+0000","last_became_peered":"2024-09-10T22:00:34.506922+0000","last_unstale":"2024-09-10T22:03:03.838120+0000","last_undegraded":"2024-09-10T22:03:03.838120+0000","last_fullsized":"2024-09-10T22:03:03.838120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.489031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853043+0000","last_change":"2024-09-10T22:00:34.509985+0000","last_active":"2024-09-10T22:03:03.853043+0000","last_peered":"2024-09-10T22:03:03.853043+0000","last_clean":"2024-09-10T22:03:03.853043+0000","last_became_active":"2024-09-10T22:00:34.509527+0000","last_became_peered":"2024-09-10T22:00:34.509527+0000","last_unstale":"2024-09-10T22:03:03.853043+0000","last_undegraded":"2024-09-10T22:03:03.853043+0000","last_fullsized":"2024-09-10T22:03:03.853043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:49.463334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.839704+0000","last_change":"2024-09-10T22:00:34.497986+0000","last_active":"2024-09-10T22:03:03.839704+0000","last_peered":"2024-09-10T22:03:03.839704+0000","last_clean":"2024-09-10T22:03:03.839704+0000","last_became_active":"2024-09-10T22:00:34.497612+0000","last_became_peered":"2024-09-10T22:00:34.497612+0000","last_unstale":"2024-09-10T22:03:03.839704+0000","last_undegraded":"2024-09-10T22:03:03.839704+0000","last_fullsized":"2024-09-10T22:03:03.839704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:15.696988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851846+0000","last_change":"2024-09-10T22:00:34.498482+0000","last_active":"2024-09-10T22:03:03.851846+0000","last_peered":"2024-09-10T22:03:03.851846+0000","last_clean":"2024-09-10T22:03:03.851846+0000","last_became_active":"2024-09-10T22:00:34.498139+0000","last_became_peered":"2024-09-10T22:00:34.498139+0000","last_unstale":"2024-09-10T22:03:03.851846+0000","last_undegraded":"2024-09-10T22:03:03.851846+0000","last_fullsized":"2024-09-10T22:03:03.851846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:23.427253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841157+0000","last_change":"2024-09-10T22:00:34.491325+0000","last_active":"2024-09-10T22:03:03.841157+0000","last_peered":"2024-09-10T22:03:03.841157+0000","last_clean":"2024-09-10T22:03:03.841157+0000","last_became_active":"2024-09-10T22:00:34.491026+0000","last_became_peered":"2024-09-10T22:00:34.491026+0000","last_unstale":"2024-09-10T22:03:03.841157+0000","last_undegraded":"2024-09-10T22:03:03.841157+0000","last_fullsized":"2024-09-10T22:03:03.841157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:07.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.843338+0000","last_change":"2024-09-10T22:00:34.503975+0000","last_active":"2024-09-10T22:03:03.843338+0000","last_peered":"2024-09-10T22:03:03.843338+0000","last_clean":"2024-09-10T22:03:03.843338+0000","last_became_active":"2024-09-10T22:00:34.503624+0000","last_became_peered":"2024-09-10T22:00:34.503624+0000","last_unstale":"2024-09-10T22:03:03.843338+0000","last_undegraded":"2024-09-10T22:03:03.843338+0000","last_fullsized":"2024-09-10T22:03:03.843338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:17.678811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837903+0000","last_change":"2024-09-10T22:01:35.596824+0000","last_active":"2024-09-10T22:03:03.837903+0000","last_peered":"2024-09-10T22:03:03.837903+0000","last_clean":"2024-09-10T22:03:03.837903+0000","last_became_active":"2024-09-10T22:01:35.596550+0000","last_became_peered":"2024-09-10T22:01:35.596550+0000","last_unstale":"2024-09-10T22:03:03.837903+0000","last_undegraded":"2024-09-10T22:03:03.837903+0000","last_fullsized":"2024-09-10T22:03:03.837903+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:19.617649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.832879+0000","last_change":"2024-09-10T22:02:35.895282+0000","last_active":"2024-09-10T22:03:03.832879+0000","last_peered":"2024-09-10T22:03:03.832879+0000","last_clean":"2024-09-10T22:03:03.832879+0000","last_became_active":"2024-09-10T22:02:35.894850+0000","last_became_peered":"2024-09-10T22:02:35.894850+0000","last_unstale":"2024-09-10T22:03:03.832879+0000","last_undegraded":"2024-09-10T22:03:03.832879+0000","last_fullsized":"2024-09-10T22:03:03.832879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:50.109012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840009+0000","last_change":"2024-09-10T22:00:34.513134+0000","last_active":"2024-09-10T22:03:03.840009+0000","last_peered":"2024-09-10T22:03:03.840009+0000","last_clean":"2024-09-10T22:03:03.840009+0000","last_became_active":"2024-09-10T22:00:34.507578+0000","last_became_peered":"2024-09-10T22:00:34.507578+0000","last_unstale":"2024-09-10T22:03:03.840009+0000","last_undegraded":"2024-09-10T22:03:03.840009+0000","last_fullsized":"2024-09-10T22:03:03.840009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:33.387337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851019+0000","last_change":"2024-09-10T22:00:34.507645+0000","last_active":"2024-09-10T22:03:03.851019+0000","last_peered":"2024-09-10T22:03:03.851019+0000","last_clean":"2024-09-10T22:03:03.851019+0000","last_became_active":"2024-09-10T22:00:34.503614+0000","last_became_peered":"2024-09-10T22:00:34.503614+0000","last_unstale":"2024-09-10T22:03:03.851019+0000","last_undegraded":"2024-09-10T22:03:03.851019+0000","last_fullsized":"2024-09-10T22:03:03.851019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:53.033523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841836+0000","last_change":"2024-09-10T22:00:34.501891+0000","last_active":"2024-09-10T22:03:03.841836+0000","last_peered":"2024-09-10T22:03:03.841836+0000","last_clean":"2024-09-10T22:03:03.841836+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:03:03.841836+0000","last_undegraded":"2024-09-10T22:03:03.841836+0000","last_fullsized":"2024-09-10T22:03:03.841836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:30.068461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.842854+0000","last_change":"2024-09-10T22:00:34.501475+0000","last_active":"2024-09-10T22:03:03.842854+0000","last_peered":"2024-09-10T22:03:03.842854+0000","last_clean":"2024-09-10T22:03:03.842854+0000","last_became_active":"2024-09-10T22:00:34.500909+0000","last_became_peered":"2024-09-10T22:00:34.500909+0000","last_unstale":"2024-09-10T22:03:03.842854+0000","last_undegraded":"2024-09-10T22:03:03.842854+0000","last_fullsized":"2024-09-10T22:03:03.842854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:39.616773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":471451,"num_objects":99,"num_object_clones":0,"num_object_copies":297,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":1584,"num_read_kb":944,"num_write":379,"num_write_kb":1261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":770048,"data_stored":2201905,"data_compressed":94032,"data_compressed_allocated":122880,"data_compressed_original":1916928,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246272,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678071,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":536890,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874300,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1212416,"data_stored":1585083,"data_compressed":86530,"data_compressed_allocated":114688,"data_compressed_original":1048576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037826,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":539779,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234058,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":929165,"data_compressed":51846,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430288,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":1135242,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593815,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":532212,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790045,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":1128774,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953571,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":536286,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149801,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1093632,"data_stored":1317887,"data_compressed":69937,"data_compressed_allocated":94208,"data_compressed_original":851968,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378735,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":597774,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542262,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":601067,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705787,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":533668,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":787422,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19272,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":464250,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:03:54.177 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T22:03:54.178 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T22:03:54.550 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:54.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:54 smithi090 ceph-mon[30350]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:03:54.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:54 smithi090 ceph-mon[30350]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-10T22:03:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:54 smithi171 ceph-mon[34138]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:03:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:54 smithi171 ceph-mon[34138]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-10T22:03:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:54 smithi154 ceph-mon[34495]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:03:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:54 smithi154 ceph-mon[34495]: pgmap v418: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 57 op/s 2024-09-10T22:03:55.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.197+0000 7f01a6a00640 1 -- 172.21.15.90:0/3131668227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 msgr2=0x7f01a010ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:55.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.197+0000 7f01a6a00640 1 --2- 172.21.15.90:0/3131668227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 0x7f01a010ac40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f018c009900 tx=0x7f018c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.198+0000 7f01a6a00640 1 -- 172.21.15.90:0/3131668227 shutdown_connections 2024-09-10T22:03:55.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.198+0000 7f01a6a00640 1 --2- 172.21.15.90:0/3131668227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01a010b180 0x7f01a0111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.198+0000 7f01a6a00640 1 --2- 172.21.15.90:0/3131668227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 0x7f01a010ac40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.198+0000 7f01a6a00640 1 --2- 172.21.15.90:0/3131668227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a0104df0 0x7f01a01051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.198+0000 7f01a6a00640 1 -- 172.21.15.90:0/3131668227 >> 172.21.15.90:0/3131668227 conn(0x7f01a0100840 msgr2=0x7f01a0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.198+0000 7f01a6a00640 1 -- 172.21.15.90:0/3131668227 shutdown_connections 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.198+0000 7f01a6a00640 1 -- 172.21.15.90:0/3131668227 wait complete. 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.198+0000 7f01a6a00640 1 Processor -- start 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.199+0000 7f01a6a00640 1 -- start start 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.199+0000 7f01a6a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01a0104df0 0x7f01a019b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.199+0000 7f01a6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 0x7f01a019b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.199+0000 7f01a6a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a010b180 0x7f01a01a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.199+0000 7f01a6a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0114cf0 con 0x7f01a010a800 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.199+0000 7f01a6a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0114b70 con 0x7f01a0104df0 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.199+0000 7f01a6a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0114e70 con 0x7f01a010b180 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.199+0000 7f01a4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 0x7f01a019b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:55.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.199+0000 7f01a4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 0x7f01a019b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60762/0 (socket says 172.21.15.90:60762) 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.200+0000 7f01a4c00640 1 -- 172.21.15.90:0/293456914 learned_addr learned my addr 172.21.15.90:0/293456914 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.200+0000 7f01a6000640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a010b180 0x7f01a01a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.200+0000 7f01a5600640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01a0104df0 0x7f01a019b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.200+0000 7f01a4c00640 1 -- 172.21.15.90:0/293456914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a010b180 msgr2=0x7f01a01a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.200+0000 7f01a4c00640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a010b180 0x7f01a01a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.200+0000 7f01a4c00640 1 -- 172.21.15.90:0/293456914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01a0104df0 msgr2=0x7f01a019b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.200+0000 7f01a4c00640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01a0104df0 0x7f01a019b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.200+0000 7f01a4c00640 1 -- 172.21.15.90:0/293456914 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a01a10d0 con 0x7f01a010a800 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.200+0000 7f01a4c00640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 0x7f01a019b560 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f018c009a20 tx=0x7f018c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.201+0000 7f019e000640 1 -- 172.21.15.90:0/293456914 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c03f030 con 0x7f01a010a800 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.201+0000 7f019e000640 1 -- 172.21.15.90:0/293456914 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f018c0035f0 con 0x7f01a010a800 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.201+0000 7f019e000640 1 -- 172.21.15.90:0/293456914 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c035070 con 0x7f01a010a800 2024-09-10T22:03:55.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.201+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a01a1360 con 0x7f01a010a800 2024-09-10T22:03:55.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.201+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a01a7a80 con 0x7f01a010a800 2024-09-10T22:03:55.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.202+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a01051d0 con 0x7f01a010a800 2024-09-10T22:03:55.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.204+0000 7f019e000640 1 -- 172.21.15.90:0/293456914 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f018c031a70 con 0x7f01a010a800 2024-09-10T22:03:55.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.206+0000 7f019e000640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01880b54e0 0x7f01880b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:55.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.206+0000 7f019e000640 1 -- 172.21.15.90:0/293456914 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f018c106d30 con 0x7f01a010a800 2024-09-10T22:03:55.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.206+0000 7f01a5600640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01880b54e0 0x7f01880b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:55.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.207+0000 7f01a5600640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01880b54e0 0x7f01880b79a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f019400abe0 tx=0x7f019400a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:55.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.209+0000 7f019e000640 1 -- 172.21.15.90:0/293456914 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f018c0cd5a0 con 0x7f01a010a800 2024-09-10T22:03:55.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.392+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f01a0107dc0 con 0x7f01880b54e0 2024-09-10T22:03:55.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.398+0000 7f019e000640 1 -- 172.21.15.90:0/293456914 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340937 (secure 0 0 0) 0x7f01a0107dc0 con 0x7f01880b54e0 2024-09-10T22:03:55.402 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:55.404 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-09-10T22:03:55.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01880b54e0 msgr2=0x7f01880b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:55.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01880b54e0 0x7f01880b79a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f019400abe0 tx=0x7f019400a2d0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 msgr2=0x7f01a019b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:55.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 0x7f01a019b560 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f018c009a20 tx=0x7f018c004270 comp rx=0 tx=0).stop 2024-09-10T22:03:55.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 shutdown_connections 2024-09-10T22:03:55.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01880b54e0 0x7f01880b79a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01a010b180 0x7f01a01a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a010a800 0x7f01a019b560 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 --2- 172.21.15.90:0/293456914 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01a0104df0 0x7f01a019b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:55.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 >> 172.21.15.90:0/293456914 conn(0x7f01a0100840 msgr2=0x7f01a0103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:55.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 shutdown_connections 2024-09-10T22:03:55.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:55.403+0000 7f01a6a00640 1 -- 172.21.15.90:0/293456914 wait complete. 2024-09-10T22:03:56.333 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-10T22:03:53.634255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1455829,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1842,"num_read_kb":1166,"num_write":628,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2951952,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":977,"ondisk_log_size":977,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":349324,"kb_used_data":12544,"kb_used_omap":18,"kb_used_meta":336557,"kb_avail":1124592500,"statfs":{"total":1151940427776,"available":1151582720000,"internally_reserved":0,"allocated":12845056,"data_stored":10760819,"data_compressed":610470,"data_compressed_allocated":823296,"data_compressed_original":5136384,"omap_allocated":19143,"internal_metadata":344634681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":793771,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":455,"num_read_kb":364,"num_write":241,"num_write_kb":2106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003058"},"pg_stats":[{"pgid":"2.1f","version":"90'15","reported_seq":202,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:51.987213+0000","last_change":"2024-09-10T22:00:30.824265+0000","last_active":"2024-09-10T22:03:51.987213+0000","last_peered":"2024-09-10T22:03:51.987213+0000","last_clean":"2024-09-10T22:03:51.987213+0000","last_became_active":"2024-09-10T22:00:30.823905+0000","last_became_peered":"2024-09-10T22:00:30.823905+0000","last_unstale":"2024-09-10T22:03:51.987213+0000","last_undegraded":"2024-09-10T22:03:51.987213+0000","last_fullsized":"2024-09-10T22:03:51.987213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:45:16.340817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":142,"num_read_kb":142,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837001+0000","last_change":"2024-09-10T22:00:34.497719+0000","last_active":"2024-09-10T22:03:03.837001+0000","last_peered":"2024-09-10T22:03:03.837001+0000","last_clean":"2024-09-10T22:03:03.837001+0000","last_became_active":"2024-09-10T22:00:34.497380+0000","last_became_peered":"2024-09-10T22:00:34.497380+0000","last_unstale":"2024-09-10T22:03:03.837001+0000","last_undegraded":"2024-09-10T22:03:03.837001+0000","last_fullsized":"2024-09-10T22:03:03.837001+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:26.336299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723331+0000","last_change":"2024-09-10T22:00:43.710835+0000","last_active":"2024-09-10T22:02:35.723331+0000","last_peered":"2024-09-10T22:02:35.723331+0000","last_clean":"2024-09-10T22:02:35.723331+0000","last_became_active":"2024-09-10T22:00:43.710363+0000","last_became_peered":"2024-09-10T22:00:43.710363+0000","last_unstale":"2024-09-10T22:02:35.723331+0000","last_undegraded":"2024-09-10T22:02:35.723331+0000","last_fullsized":"2024-09-10T22:02:35.723331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:28.173482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723977+0000","last_change":"2024-09-10T22:00:31.083228+0000","last_active":"2024-09-10T22:02:35.723977+0000","last_peered":"2024-09-10T22:02:35.723977+0000","last_clean":"2024-09-10T22:02:35.723977+0000","last_became_active":"2024-09-10T22:00:31.082911+0000","last_became_peered":"2024-09-10T22:00:31.082911+0000","last_unstale":"2024-09-10T22:02:35.723977+0000","last_undegraded":"2024-09-10T22:02:35.723977+0000","last_fullsized":"2024-09-10T22:02:35.723977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:28.598628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853513+0000","last_change":"2024-09-10T22:00:34.509939+0000","last_active":"2024-09-10T22:03:03.853513+0000","last_peered":"2024-09-10T22:03:03.853513+0000","last_clean":"2024-09-10T22:03:03.853513+0000","last_became_active":"2024-09-10T22:00:34.509536+0000","last_became_peered":"2024-09-10T22:00:34.509536+0000","last_unstale":"2024-09-10T22:03:03.853513+0000","last_undegraded":"2024-09-10T22:03:03.853513+0000","last_fullsized":"2024-09-10T22:03:03.853513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:27.602378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897125+0000","last_change":"2024-09-10T22:00:43.714662+0000","last_active":"2024-09-10T22:02:35.897125+0000","last_peered":"2024-09-10T22:02:35.897125+0000","last_clean":"2024-09-10T22:02:35.897125+0000","last_became_active":"2024-09-10T22:00:43.714074+0000","last_became_peered":"2024-09-10T22:00:43.714074+0000","last_unstale":"2024-09-10T22:02:35.897125+0000","last_undegraded":"2024-09-10T22:02:35.897125+0000","last_fullsized":"2024-09-10T22:02:35.897125+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:00:41.204519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948479+0000","last_change":"2024-09-10T22:01:35.209979+0000","last_active":"2024-09-10T22:02:35.948479+0000","last_peered":"2024-09-10T22:02:35.948479+0000","last_clean":"2024-09-10T22:02:35.948479+0000","last_became_active":"2024-09-10T22:01:35.197192+0000","last_became_peered":"2024-09-10T22:01:35.197192+0000","last_unstale":"2024-09-10T22:02:35.948479+0000","last_undegraded":"2024-09-10T22:02:35.948479+0000","last_fullsized":"2024-09-10T22:02:35.948479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:47.707679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853796+0000","last_change":"2024-09-10T22:00:34.502795+0000","last_active":"2024-09-10T22:03:03.853796+0000","last_peered":"2024-09-10T22:03:03.853796+0000","last_clean":"2024-09-10T22:03:03.853796+0000","last_became_active":"2024-09-10T22:00:34.501112+0000","last_became_peered":"2024-09-10T22:00:34.501112+0000","last_unstale":"2024-09-10T22:03:03.853796+0000","last_undegraded":"2024-09-10T22:03:03.853796+0000","last_fullsized":"2024-09-10T22:03:03.853796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:45.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897363+0000","last_change":"2024-09-10T22:01:35.619765+0000","last_active":"2024-09-10T22:02:35.897363+0000","last_peered":"2024-09-10T22:02:35.897363+0000","last_clean":"2024-09-10T22:02:35.897363+0000","last_became_active":"2024-09-10T22:01:35.619407+0000","last_became_peered":"2024-09-10T22:01:35.619407+0000","last_unstale":"2024-09-10T22:02:35.897363+0000","last_undegraded":"2024-09-10T22:02:35.897363+0000","last_fullsized":"2024-09-10T22:02:35.897363+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:17.064585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726660+0000","last_change":"2024-09-10T22:00:30.818224+0000","last_active":"2024-09-10T22:02:35.726660+0000","last_peered":"2024-09-10T22:02:35.726660+0000","last_clean":"2024-09-10T22:02:35.726660+0000","last_became_active":"2024-09-10T22:00:30.817781+0000","last_became_peered":"2024-09-10T22:00:30.817781+0000","last_unstale":"2024-09-10T22:02:35.726660+0000","last_undegraded":"2024-09-10T22:02:35.726660+0000","last_fullsized":"2024-09-10T22:02:35.726660+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:47.097642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851346+0000","last_change":"2024-09-10T22:02:35.949974+0000","last_active":"2024-09-10T22:03:03.851346+0000","last_peered":"2024-09-10T22:03:03.851346+0000","last_clean":"2024-09-10T22:03:03.851346+0000","last_became_active":"2024-09-10T22:02:35.949618+0000","last_became_peered":"2024-09-10T22:02:35.949618+0000","last_unstale":"2024-09-10T22:03:03.851346+0000","last_undegraded":"2024-09-10T22:03:03.851346+0000","last_fullsized":"2024-09-10T22:03:03.851346+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:02.877650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897288+0000","last_change":"2024-09-10T22:00:43.712066+0000","last_active":"2024-09-10T22:02:35.897288+0000","last_peered":"2024-09-10T22:02:35.897288+0000","last_clean":"2024-09-10T22:02:35.897288+0000","last_became_active":"2024-09-10T22:00:43.711614+0000","last_became_peered":"2024-09-10T22:00:43.711614+0000","last_unstale":"2024-09-10T22:02:35.897288+0000","last_undegraded":"2024-09-10T22:02:35.897288+0000","last_fullsized":"2024-09-10T22:02:35.897288+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:42.403367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.854262+0000","last_change":"2024-09-10T22:00:34.501943+0000","last_active":"2024-09-10T22:03:03.854262+0000","last_peered":"2024-09-10T22:03:03.854262+0000","last_clean":"2024-09-10T22:03:03.854262+0000","last_became_active":"2024-09-10T22:00:34.501602+0000","last_became_peered":"2024-09-10T22:00:34.501602+0000","last_unstale":"2024-09-10T22:03:03.854262+0000","last_undegraded":"2024-09-10T22:03:03.854262+0000","last_fullsized":"2024-09-10T22:03:03.854262+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:53.200974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726593+0000","last_change":"2024-09-10T22:01:35.200791+0000","last_active":"2024-09-10T22:02:35.726593+0000","last_peered":"2024-09-10T22:02:35.726593+0000","last_clean":"2024-09-10T22:02:35.726593+0000","last_became_active":"2024-09-10T22:01:35.196059+0000","last_became_peered":"2024-09-10T22:01:35.196059+0000","last_unstale":"2024-09-10T22:02:35.726593+0000","last_undegraded":"2024-09-10T22:02:35.726593+0000","last_fullsized":"2024-09-10T22:02:35.726593+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:51.349216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897010+0000","last_change":"2024-09-10T22:00:43.760567+0000","last_active":"2024-09-10T22:02:35.897010+0000","last_peered":"2024-09-10T22:02:35.897010+0000","last_clean":"2024-09-10T22:02:35.897010+0000","last_became_active":"2024-09-10T22:00:43.760064+0000","last_became_peered":"2024-09-10T22:00:43.760064+0000","last_unstale":"2024-09-10T22:02:35.897010+0000","last_undegraded":"2024-09-10T22:02:35.897010+0000","last_fullsized":"2024-09-10T22:02:35.897010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:55.469525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.843933+0000","last_change":"2024-09-10T22:00:34.486225+0000","last_active":"2024-09-10T22:03:03.843933+0000","last_peered":"2024-09-10T22:03:03.843933+0000","last_clean":"2024-09-10T22:03:03.843933+0000","last_became_active":"2024-09-10T22:00:34.485896+0000","last_became_peered":"2024-09-10T22:00:34.485896+0000","last_unstale":"2024-09-10T22:03:03.843933+0000","last_undegraded":"2024-09-10T22:03:03.843933+0000","last_fullsized":"2024-09-10T22:03:03.843933+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:39:00.151927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'4","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:43.663404+0000","last_change":"2024-09-10T22:00:30.817958+0000","last_active":"2024-09-10T22:02:43.663404+0000","last_peered":"2024-09-10T22:02:43.663404+0000","last_clean":"2024-09-10T22:02:43.663404+0000","last_became_active":"2024-09-10T22:00:30.817646+0000","last_became_peered":"2024-09-10T22:00:30.817646+0000","last_unstale":"2024-09-10T22:02:43.663404+0000","last_undegraded":"2024-09-10T22:02:43.663404+0000","last_fullsized":"2024-09-10T22:02:43.663404+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:27.722184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897658+0000","last_change":"2024-09-10T22:00:43.716465+0000","last_active":"2024-09-10T22:02:35.897658+0000","last_peered":"2024-09-10T22:02:35.897658+0000","last_clean":"2024-09-10T22:02:35.897658+0000","last_became_active":"2024-09-10T22:00:43.716165+0000","last_became_peered":"2024-09-10T22:00:43.716165+0000","last_unstale":"2024-09-10T22:02:35.897658+0000","last_undegraded":"2024-09-10T22:02:35.897658+0000","last_fullsized":"2024-09-10T22:02:35.897658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:18.761842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841664+0000","last_change":"2024-09-10T22:00:34.493992+0000","last_active":"2024-09-10T22:03:03.841664+0000","last_peered":"2024-09-10T22:03:03.841664+0000","last_clean":"2024-09-10T22:03:03.841664+0000","last_became_active":"2024-09-10T22:00:34.493670+0000","last_became_peered":"2024-09-10T22:00:34.493670+0000","last_unstale":"2024-09-10T22:03:03.841664+0000","last_undegraded":"2024-09-10T22:03:03.841664+0000","last_fullsized":"2024-09-10T22:03:03.841664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:04.991819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722649+0000","last_change":"2024-09-10T22:00:30.819830+0000","last_active":"2024-09-10T22:02:35.722649+0000","last_peered":"2024-09-10T22:02:35.722649+0000","last_clean":"2024-09-10T22:02:35.722649+0000","last_became_active":"2024-09-10T22:00:30.819401+0000","last_became_peered":"2024-09-10T22:00:30.819401+0000","last_unstale":"2024-09-10T22:02:35.722649+0000","last_undegraded":"2024-09-10T22:02:35.722649+0000","last_fullsized":"2024-09-10T22:02:35.722649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:37.823107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724263+0000","last_change":"2024-09-10T22:00:43.718500+0000","last_active":"2024-09-10T22:02:35.724263+0000","last_peered":"2024-09-10T22:02:35.724263+0000","last_clean":"2024-09-10T22:02:35.724263+0000","last_became_active":"2024-09-10T22:00:43.717667+0000","last_became_peered":"2024-09-10T22:00:43.717667+0000","last_unstale":"2024-09-10T22:02:35.724263+0000","last_undegraded":"2024-09-10T22:02:35.724263+0000","last_fullsized":"2024-09-10T22:02:35.724263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:23.328314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.792059+0000","last_change":"2024-09-10T22:00:34.494162+0000","last_active":"2024-09-10T22:03:03.792059+0000","last_peered":"2024-09-10T22:03:03.792059+0000","last_clean":"2024-09-10T22:03:03.792059+0000","last_became_active":"2024-09-10T22:00:34.493819+0000","last_became_peered":"2024-09-10T22:00:34.493819+0000","last_unstale":"2024-09-10T22:03:03.792059+0000","last_undegraded":"2024-09-10T22:03:03.792059+0000","last_fullsized":"2024-09-10T22:03:03.792059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:43.264363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948452+0000","last_change":"2024-09-10T22:00:31.081559+0000","last_active":"2024-09-10T22:02:35.948452+0000","last_peered":"2024-09-10T22:02:35.948452+0000","last_clean":"2024-09-10T22:02:35.948452+0000","last_became_active":"2024-09-10T22:00:31.081257+0000","last_became_peered":"2024-09-10T22:00:31.081257+0000","last_unstale":"2024-09-10T22:02:35.948452+0000","last_undegraded":"2024-09-10T22:02:35.948452+0000","last_fullsized":"2024-09-10T22:02:35.948452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:53.235765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897682+0000","last_change":"2024-09-10T22:01:35.445171+0000","last_active":"2024-09-10T22:02:35.897682+0000","last_peered":"2024-09-10T22:02:35.897682+0000","last_clean":"2024-09-10T22:02:35.897682+0000","last_became_active":"2024-09-10T22:01:35.444651+0000","last_became_peered":"2024-09-10T22:01:35.444651+0000","last_unstale":"2024-09-10T22:02:35.897682+0000","last_undegraded":"2024-09-10T22:02:35.897682+0000","last_fullsized":"2024-09-10T22:02:35.897682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:36.457151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.847341+0000","last_change":"2024-09-10T22:01:35.750982+0000","last_active":"2024-09-10T22:03:03.847341+0000","last_peered":"2024-09-10T22:03:03.847341+0000","last_clean":"2024-09-10T22:03:03.847341+0000","last_became_active":"2024-09-10T22:01:35.750572+0000","last_became_peered":"2024-09-10T22:01:35.750572+0000","last_unstale":"2024-09-10T22:03:03.847341+0000","last_undegraded":"2024-09-10T22:03:03.847341+0000","last_fullsized":"2024-09-10T22:03:03.847341+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:36.596313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897938+0000","last_change":"2024-09-10T22:00:31.311864+0000","last_active":"2024-09-10T22:02:35.897938+0000","last_peered":"2024-09-10T22:02:35.897938+0000","last_clean":"2024-09-10T22:02:35.897938+0000","last_became_active":"2024-09-10T22:00:31.311611+0000","last_became_peered":"2024-09-10T22:00:31.311611+0000","last_unstale":"2024-09-10T22:02:35.897938+0000","last_undegraded":"2024-09-10T22:02:35.897938+0000","last_fullsized":"2024-09-10T22:02:35.897938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:27.969999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723479+0000","last_change":"2024-09-10T22:00:43.708023+0000","last_active":"2024-09-10T22:02:35.723479+0000","last_peered":"2024-09-10T22:02:35.723479+0000","last_clean":"2024-09-10T22:02:35.723479+0000","last_became_active":"2024-09-10T22:00:43.707086+0000","last_became_peered":"2024-09-10T22:00:43.707086+0000","last_unstale":"2024-09-10T22:02:35.723479+0000","last_undegraded":"2024-09-10T22:02:35.723479+0000","last_fullsized":"2024-09-10T22:02:35.723479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:15.585104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.846819+0000","last_change":"2024-09-10T22:00:34.494104+0000","last_active":"2024-09-10T22:03:03.846819+0000","last_peered":"2024-09-10T22:03:03.846819+0000","last_clean":"2024-09-10T22:03:03.846819+0000","last_became_active":"2024-09-10T22:00:34.493404+0000","last_became_peered":"2024-09-10T22:00:34.493404+0000","last_unstale":"2024-09-10T22:03:03.846819+0000","last_undegraded":"2024-09-10T22:03:03.846819+0000","last_fullsized":"2024-09-10T22:03:03.846819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:47.444100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'100","reported_seq":234,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:49.679223+0000","last_change":"2024-09-10T22:00:30.819427+0000","last_active":"2024-09-10T22:03:49.679223+0000","last_peered":"2024-09-10T22:03:49.679223+0000","last_clean":"2024-09-10T22:03:49.679223+0000","last_became_active":"2024-09-10T22:00:30.819098+0000","last_became_peered":"2024-09-10T22:00:30.819098+0000","last_unstale":"2024-09-10T22:03:49.679223+0000","last_undegraded":"2024-09-10T22:03:49.679223+0000","last_fullsized":"2024-09-10T22:03:49.679223+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:23.068155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":302,"num_read_kb":268,"num_write":184,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722303+0000","last_change":"2024-09-10T22:00:43.760602+0000","last_active":"2024-09-10T22:02:35.722303+0000","last_peered":"2024-09-10T22:02:35.722303+0000","last_clean":"2024-09-10T22:02:35.722303+0000","last_became_active":"2024-09-10T22:00:43.760172+0000","last_became_peered":"2024-09-10T22:00:43.760172+0000","last_unstale":"2024-09-10T22:02:35.722303+0000","last_undegraded":"2024-09-10T22:02:35.722303+0000","last_fullsized":"2024-09-10T22:02:35.722303+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:40.332101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.836073+0000","last_change":"2024-09-10T22:00:34.501632+0000","last_active":"2024-09-10T22:03:03.836073+0000","last_peered":"2024-09-10T22:03:03.836073+0000","last_clean":"2024-09-10T22:03:03.836073+0000","last_became_active":"2024-09-10T22:00:34.500924+0000","last_became_peered":"2024-09-10T22:00:34.500924+0000","last_unstale":"2024-09-10T22:03:03.836073+0000","last_undegraded":"2024-09-10T22:03:03.836073+0000","last_fullsized":"2024-09-10T22:03:03.836073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:28.052570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726206+0000","last_change":"2024-09-10T22:00:31.081391+0000","last_active":"2024-09-10T22:02:35.726206+0000","last_peered":"2024-09-10T22:02:35.726206+0000","last_clean":"2024-09-10T22:02:35.726206+0000","last_became_active":"2024-09-10T22:00:31.081018+0000","last_became_peered":"2024-09-10T22:00:31.081018+0000","last_unstale":"2024-09-10T22:02:35.726206+0000","last_undegraded":"2024-09-10T22:02:35.726206+0000","last_fullsized":"2024-09-10T22:02:35.726206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:09.450785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948812+0000","last_change":"2024-09-10T22:00:43.711592+0000","last_active":"2024-09-10T22:02:35.948812+0000","last_peered":"2024-09-10T22:02:35.948812+0000","last_clean":"2024-09-10T22:02:35.948812+0000","last_became_active":"2024-09-10T22:00:43.711144+0000","last_became_peered":"2024-09-10T22:00:43.711144+0000","last_unstale":"2024-09-10T22:02:35.948812+0000","last_undegraded":"2024-09-10T22:02:35.948812+0000","last_fullsized":"2024-09-10T22:02:35.948812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:01.818907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840894+0000","last_change":"2024-09-10T22:00:34.494070+0000","last_active":"2024-09-10T22:03:03.840894+0000","last_peered":"2024-09-10T22:03:03.840894+0000","last_clean":"2024-09-10T22:03:03.840894+0000","last_became_active":"2024-09-10T22:00:34.493324+0000","last_became_peered":"2024-09-10T22:00:34.493324+0000","last_unstale":"2024-09-10T22:03:03.840894+0000","last_undegraded":"2024-09-10T22:03:03.840894+0000","last_fullsized":"2024-09-10T22:03:03.840894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:29.969743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723888+0000","last_change":"2024-09-10T22:00:31.082439+0000","last_active":"2024-09-10T22:02:35.723888+0000","last_peered":"2024-09-10T22:02:35.723888+0000","last_clean":"2024-09-10T22:02:35.723888+0000","last_became_active":"2024-09-10T22:00:31.082025+0000","last_became_peered":"2024-09-10T22:00:31.082025+0000","last_unstale":"2024-09-10T22:02:35.723888+0000","last_undegraded":"2024-09-10T22:02:35.723888+0000","last_fullsized":"2024-09-10T22:02:35.723888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:21.933419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.726116+0000","last_change":"2024-09-10T22:00:43.760997+0000","last_active":"2024-09-10T22:02:35.726116+0000","last_peered":"2024-09-10T22:02:35.726116+0000","last_clean":"2024-09-10T22:02:35.726116+0000","last_became_active":"2024-09-10T22:00:43.760124+0000","last_became_peered":"2024-09-10T22:00:43.760124+0000","last_unstale":"2024-09-10T22:02:35.726116+0000","last_undegraded":"2024-09-10T22:02:35.726116+0000","last_fullsized":"2024-09-10T22:02:35.726116+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:29.940602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.843650+0000","last_change":"2024-09-10T22:00:34.511730+0000","last_active":"2024-09-10T22:03:03.843650+0000","last_peered":"2024-09-10T22:03:03.843650+0000","last_clean":"2024-09-10T22:03:03.843650+0000","last_became_active":"2024-09-10T22:00:34.511440+0000","last_became_peered":"2024-09-10T22:00:34.511440+0000","last_unstale":"2024-09-10T22:03:03.843650+0000","last_undegraded":"2024-09-10T22:03:03.843650+0000","last_fullsized":"2024-09-10T22:03:03.843650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:09.053835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'8","reported_seq":194,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:47.981333+0000","last_change":"2024-09-10T22:00:30.825999+0000","last_active":"2024-09-10T22:03:47.981333+0000","last_peered":"2024-09-10T22:03:47.981333+0000","last_clean":"2024-09-10T22:03:47.981333+0000","last_became_active":"2024-09-10T22:00:30.825334+0000","last_became_peered":"2024-09-10T22:00:30.825334+0000","last_unstale":"2024-09-10T22:03:47.981333+0000","last_undegraded":"2024-09-10T22:03:47.981333+0000","last_fullsized":"2024-09-10T22:03:47.981333+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:07:14.684222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":141,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894213+0000","last_change":"2024-09-10T22:00:43.710517+0000","last_active":"2024-09-10T22:02:35.894213+0000","last_peered":"2024-09-10T22:02:35.894213+0000","last_clean":"2024-09-10T22:02:35.894213+0000","last_became_active":"2024-09-10T22:00:43.709252+0000","last_became_peered":"2024-09-10T22:00:43.709252+0000","last_unstale":"2024-09-10T22:02:35.894213+0000","last_undegraded":"2024-09-10T22:02:35.894213+0000","last_fullsized":"2024-09-10T22:02:35.894213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:21.355152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838763+0000","last_change":"2024-09-10T22:02:35.898878+0000","last_active":"2024-09-10T22:03:03.838763+0000","last_peered":"2024-09-10T22:03:03.838763+0000","last_clean":"2024-09-10T22:03:03.838763+0000","last_became_active":"2024-09-10T22:02:35.898531+0000","last_became_peered":"2024-09-10T22:02:35.898531+0000","last_unstale":"2024-09-10T22:03:03.838763+0000","last_undegraded":"2024-09-10T22:03:03.838763+0000","last_fullsized":"2024-09-10T22:03:03.838763+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:02.735208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":188,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:48.037068+0000","last_change":"2024-09-10T22:00:31.081590+0000","last_active":"2024-09-10T22:03:48.037068+0000","last_peered":"2024-09-10T22:03:48.037068+0000","last_clean":"2024-09-10T22:03:48.037068+0000","last_became_active":"2024-09-10T22:00:31.081250+0000","last_became_peered":"2024-09-10T22:00:31.081250+0000","last_unstale":"2024-09-10T22:03:48.037068+0000","last_undegraded":"2024-09-10T22:03:48.037068+0000","last_fullsized":"2024-09-10T22:03:48.037068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:12.872832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893781+0000","last_change":"2024-09-10T22:00:43.761351+0000","last_active":"2024-09-10T22:02:35.893781+0000","last_peered":"2024-09-10T22:02:35.893781+0000","last_clean":"2024-09-10T22:02:35.893781+0000","last_became_active":"2024-09-10T22:00:43.760903+0000","last_became_peered":"2024-09-10T22:00:43.760903+0000","last_unstale":"2024-09-10T22:02:35.893781+0000","last_undegraded":"2024-09-10T22:02:35.893781+0000","last_fullsized":"2024-09-10T22:02:35.893781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:14.113974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838302+0000","last_change":"2024-09-10T22:00:34.504671+0000","last_active":"2024-09-10T22:03:03.838302+0000","last_peered":"2024-09-10T22:03:03.838302+0000","last_clean":"2024-09-10T22:03:03.838302+0000","last_became_active":"2024-09-10T22:00:34.504368+0000","last_became_peered":"2024-09-10T22:00:34.504368+0000","last_unstale":"2024-09-10T22:03:03.838302+0000","last_undegraded":"2024-09-10T22:03:03.838302+0000","last_fullsized":"2024-09-10T22:03:03.838302+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:59.771589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863056+0000","last_change":"2024-09-10T22:00:30.820401+0000","last_active":"2024-09-10T22:02:35.863056+0000","last_peered":"2024-09-10T22:02:35.863056+0000","last_clean":"2024-09-10T22:02:35.863056+0000","last_became_active":"2024-09-10T22:00:30.820194+0000","last_became_peered":"2024-09-10T22:00:30.820194+0000","last_unstale":"2024-09-10T22:02:35.863056+0000","last_undegraded":"2024-09-10T22:02:35.863056+0000","last_fullsized":"2024-09-10T22:02:35.863056+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:47.767102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727473+0000","last_change":"2024-09-10T22:00:43.762001+0000","last_active":"2024-09-10T22:02:35.727473+0000","last_peered":"2024-09-10T22:02:35.727473+0000","last_clean":"2024-09-10T22:02:35.727473+0000","last_became_active":"2024-09-10T22:00:43.761323+0000","last_became_peered":"2024-09-10T22:00:43.761323+0000","last_unstale":"2024-09-10T22:02:35.727473+0000","last_undegraded":"2024-09-10T22:02:35.727473+0000","last_fullsized":"2024-09-10T22:02:35.727473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:50.113545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.847877+0000","last_change":"2024-09-10T22:00:34.498591+0000","last_active":"2024-09-10T22:03:03.847877+0000","last_peered":"2024-09-10T22:03:03.847877+0000","last_clean":"2024-09-10T22:03:03.847877+0000","last_became_active":"2024-09-10T22:00:34.496449+0000","last_became_peered":"2024-09-10T22:00:34.496449+0000","last_unstale":"2024-09-10T22:03:03.847877+0000","last_undegraded":"2024-09-10T22:03:03.847877+0000","last_fullsized":"2024-09-10T22:03:03.847877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:54.815268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897905+0000","last_change":"2024-09-10T22:00:30.817201+0000","last_active":"2024-09-10T22:02:35.897905+0000","last_peered":"2024-09-10T22:02:35.897905+0000","last_clean":"2024-09-10T22:02:35.897905+0000","last_became_active":"2024-09-10T22:00:30.816792+0000","last_became_peered":"2024-09-10T22:00:30.816792+0000","last_unstale":"2024-09-10T22:02:35.897905+0000","last_undegraded":"2024-09-10T22:02:35.897905+0000","last_fullsized":"2024-09-10T22:02:35.897905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:48.239811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":121,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863189+0000","last_change":"2024-09-10T21:59:01.830452+0000","last_active":"2024-09-10T22:02:35.863189+0000","last_peered":"2024-09-10T22:02:35.863189+0000","last_clean":"2024-09-10T22:02:35.863189+0000","last_became_active":"2024-09-10T21:59:01.823744+0000","last_became_peered":"2024-09-10T21:59:01.823744+0000","last_unstale":"2024-09-10T22:02:35.863189+0000","last_undegraded":"2024-09-10T22:02:35.863189+0000","last_fullsized":"2024-09-10T22:02:35.863189+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:54:48.621299+0000","last_clean_scrub_stamp":"2024-09-10T21:54:48.621299+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:39.573257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863562+0000","last_change":"2024-09-10T22:00:43.713888+0000","last_active":"2024-09-10T22:02:35.863562+0000","last_peered":"2024-09-10T22:02:35.863562+0000","last_clean":"2024-09-10T22:02:35.863562+0000","last_became_active":"2024-09-10T22:00:43.713329+0000","last_became_peered":"2024-09-10T22:00:43.713329+0000","last_unstale":"2024-09-10T22:02:35.863562+0000","last_undegraded":"2024-09-10T22:02:35.863562+0000","last_fullsized":"2024-09-10T22:02:35.863562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:01.062089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.852867+0000","last_change":"2024-09-10T22:00:34.502589+0000","last_active":"2024-09-10T22:03:03.852867+0000","last_peered":"2024-09-10T22:03:03.852867+0000","last_clean":"2024-09-10T22:03:03.852867+0000","last_became_active":"2024-09-10T22:00:34.502216+0000","last_became_peered":"2024-09-10T22:00:34.502216+0000","last_unstale":"2024-09-10T22:03:03.852867+0000","last_undegraded":"2024-09-10T22:03:03.852867+0000","last_fullsized":"2024-09-10T22:03:03.852867+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:44.959375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":193,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:47.001644+0000","last_change":"2024-09-10T22:00:31.311807+0000","last_active":"2024-09-10T22:03:47.001644+0000","last_peered":"2024-09-10T22:03:47.001644+0000","last_clean":"2024-09-10T22:03:47.001644+0000","last_became_active":"2024-09-10T22:00:31.311424+0000","last_became_peered":"2024-09-10T22:00:31.311424+0000","last_unstale":"2024-09-10T22:03:47.001644+0000","last_undegraded":"2024-09-10T22:03:47.001644+0000","last_fullsized":"2024-09-10T22:03:47.001644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:09.754777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2195,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":141,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948273+0000","last_change":"2024-09-10T22:00:43.711882+0000","last_active":"2024-09-10T22:02:35.948273+0000","last_peered":"2024-09-10T22:02:35.948273+0000","last_clean":"2024-09-10T22:02:35.948273+0000","last_became_active":"2024-09-10T22:00:43.711408+0000","last_became_peered":"2024-09-10T22:00:43.711408+0000","last_unstale":"2024-09-10T22:02:35.948273+0000","last_undegraded":"2024-09-10T22:02:35.948273+0000","last_fullsized":"2024-09-10T22:02:35.948273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:31.155723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841900+0000","last_change":"2024-09-10T22:00:34.505653+0000","last_active":"2024-09-10T22:03:03.841900+0000","last_peered":"2024-09-10T22:03:03.841900+0000","last_clean":"2024-09-10T22:03:03.841900+0000","last_became_active":"2024-09-10T22:00:34.505317+0000","last_became_peered":"2024-09-10T22:00:34.505317+0000","last_unstale":"2024-09-10T22:03:03.841900+0000","last_undegraded":"2024-09-10T22:03:03.841900+0000","last_fullsized":"2024-09-10T22:03:03.841900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:35.050892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'62","reported_seq":137,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:49.286712+0000","last_change":"2024-09-10T22:00:30.817431+0000","last_active":"2024-09-10T22:03:49.286712+0000","last_peered":"2024-09-10T22:03:49.286712+0000","last_clean":"2024-09-10T22:03:49.286712+0000","last_became_active":"2024-09-10T22:00:30.816952+0000","last_became_peered":"2024-09-10T22:00:30.816952+0000","last_unstale":"2024-09-10T22:03:49.286712+0000","last_undegraded":"2024-09-10T22:03:49.286712+0000","last_fullsized":"2024-09-10T22:03:49.286712+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":62,"log_dups_size":0,"ondisk_log_size":62,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:37.881641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947502+0000","last_change":"2024-09-10T22:00:43.720627+0000","last_active":"2024-09-10T22:02:35.947502+0000","last_peered":"2024-09-10T22:02:35.947502+0000","last_clean":"2024-09-10T22:02:35.947502+0000","last_became_active":"2024-09-10T22:00:43.720235+0000","last_became_peered":"2024-09-10T22:00:43.720235+0000","last_unstale":"2024-09-10T22:02:35.947502+0000","last_undegraded":"2024-09-10T22:02:35.947502+0000","last_fullsized":"2024-09-10T22:02:35.947502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:02:50.549536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841647+0000","last_change":"2024-09-10T22:00:34.501649+0000","last_active":"2024-09-10T22:03:03.841647+0000","last_peered":"2024-09-10T22:03:03.841647+0000","last_clean":"2024-09-10T22:03:03.841647+0000","last_became_active":"2024-09-10T22:00:34.498882+0000","last_became_peered":"2024-09-10T22:00:34.498882+0000","last_unstale":"2024-09-10T22:03:03.841647+0000","last_undegraded":"2024-09-10T22:03:03.841647+0000","last_fullsized":"2024-09-10T22:03:03.841647+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:13.722639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.894508+0000","last_change":"2024-09-10T22:00:30.821943+0000","last_active":"2024-09-10T22:02:35.894508+0000","last_peered":"2024-09-10T22:02:35.894508+0000","last_clean":"2024-09-10T22:02:35.894508+0000","last_became_active":"2024-09-10T22:00:30.821372+0000","last_became_peered":"2024-09-10T22:00:30.821372+0000","last_unstale":"2024-09-10T22:02:35.894508+0000","last_undegraded":"2024-09-10T22:02:35.894508+0000","last_fullsized":"2024-09-10T22:02:35.894508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:04.360383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727340+0000","last_change":"2024-09-10T22:00:43.710045+0000","last_active":"2024-09-10T22:02:35.727340+0000","last_peered":"2024-09-10T22:02:35.727340+0000","last_clean":"2024-09-10T22:02:35.727340+0000","last_became_active":"2024-09-10T22:00:43.709416+0000","last_became_peered":"2024-09-10T22:00:43.709416+0000","last_unstale":"2024-09-10T22:02:35.727340+0000","last_undegraded":"2024-09-10T22:02:35.727340+0000","last_fullsized":"2024-09-10T22:02:35.727340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:56.005549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.834717+0000","last_change":"2024-09-10T22:00:34.496163+0000","last_active":"2024-09-10T22:03:03.834717+0000","last_peered":"2024-09-10T22:03:03.834717+0000","last_clean":"2024-09-10T22:03:03.834717+0000","last_became_active":"2024-09-10T22:00:34.495814+0000","last_became_peered":"2024-09-10T22:00:34.495814+0000","last_unstale":"2024-09-10T22:03:03.834717+0000","last_undegraded":"2024-09-10T22:03:03.834717+0000","last_fullsized":"2024-09-10T22:03:03.834717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:05.398154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:45.682374+0000","last_change":"2024-09-10T22:00:30.812702+0000","last_active":"2024-09-10T22:02:45.682374+0000","last_peered":"2024-09-10T22:02:45.682374+0000","last_clean":"2024-09-10T22:02:45.682374+0000","last_became_active":"2024-09-10T22:00:30.812369+0000","last_became_peered":"2024-09-10T22:00:30.812369+0000","last_unstale":"2024-09-10T22:02:45.682374+0000","last_undegraded":"2024-09-10T22:02:45.682374+0000","last_fullsized":"2024-09-10T22:02:45.682374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:52.332876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727291+0000","last_change":"2024-09-10T22:00:43.715573+0000","last_active":"2024-09-10T22:02:35.727291+0000","last_peered":"2024-09-10T22:02:35.727291+0000","last_clean":"2024-09-10T22:02:35.727291+0000","last_became_active":"2024-09-10T22:00:43.714475+0000","last_became_peered":"2024-09-10T22:00:43.714475+0000","last_unstale":"2024-09-10T22:02:35.727291+0000","last_undegraded":"2024-09-10T22:02:35.727291+0000","last_fullsized":"2024-09-10T22:02:35.727291+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:11:55.051042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.844868+0000","last_change":"2024-09-10T22:02:35.949769+0000","last_active":"2024-09-10T22:03:03.844868+0000","last_peered":"2024-09-10T22:03:03.844868+0000","last_clean":"2024-09-10T22:03:03.844868+0000","last_became_active":"2024-09-10T22:02:35.949440+0000","last_became_peered":"2024-09-10T22:02:35.949440+0000","last_unstale":"2024-09-10T22:03:03.844868+0000","last_undegraded":"2024-09-10T22:03:03.844868+0000","last_fullsized":"2024-09-10T22:03:03.844868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:48:03.639412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948309+0000","last_change":"2024-09-10T22:01:35.903381+0000","last_active":"2024-09-10T22:02:35.948309+0000","last_peered":"2024-09-10T22:02:35.948309+0000","last_clean":"2024-09-10T22:02:35.948309+0000","last_became_active":"2024-09-10T22:01:35.878262+0000","last_became_peered":"2024-09-10T22:01:35.878262+0000","last_unstale":"2024-09-10T22:02:35.948309+0000","last_undegraded":"2024-09-10T22:02:35.948309+0000","last_fullsized":"2024-09-10T22:02:35.948309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:45.419449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,1],"acting":[11,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893771+0000","last_change":"2024-09-10T22:00:43.720728+0000","last_active":"2024-09-10T22:02:35.893771+0000","last_peered":"2024-09-10T22:02:35.893771+0000","last_clean":"2024-09-10T22:02:35.893771+0000","last_became_active":"2024-09-10T22:00:43.720305+0000","last_became_peered":"2024-09-10T22:00:43.720305+0000","last_unstale":"2024-09-10T22:02:35.893771+0000","last_undegraded":"2024-09-10T22:02:35.893771+0000","last_fullsized":"2024-09-10T22:02:35.893771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:54.986822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841861+0000","last_change":"2024-09-10T22:00:34.501571+0000","last_active":"2024-09-10T22:03:03.841861+0000","last_peered":"2024-09-10T22:03:03.841861+0000","last_clean":"2024-09-10T22:03:03.841861+0000","last_became_active":"2024-09-10T22:00:34.501211+0000","last_became_peered":"2024-09-10T22:00:34.501211+0000","last_unstale":"2024-09-10T22:03:03.841861+0000","last_undegraded":"2024-09-10T22:03:03.841861+0000","last_fullsized":"2024-09-10T22:03:03.841861+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:05.018759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'8","reported_seq":199,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:51.043862+0000","last_change":"2024-09-10T22:00:30.824296+0000","last_active":"2024-09-10T22:03:51.043862+0000","last_peered":"2024-09-10T22:03:51.043862+0000","last_clean":"2024-09-10T22:03:51.043862+0000","last_became_active":"2024-09-10T22:00:30.823886+0000","last_became_peered":"2024-09-10T22:00:30.823886+0000","last_unstale":"2024-09-10T22:03:51.043862+0000","last_undegraded":"2024-09-10T22:03:51.043862+0000","last_fullsized":"2024-09-10T22:03:51.043862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:26.071453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1613,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.724371+0000","last_change":"2024-09-10T22:00:43.717580+0000","last_active":"2024-09-10T22:02:35.724371+0000","last_peered":"2024-09-10T22:02:35.724371+0000","last_clean":"2024-09-10T22:02:35.724371+0000","last_became_active":"2024-09-10T22:00:43.708377+0000","last_became_peered":"2024-09-10T22:00:43.708377+0000","last_unstale":"2024-09-10T22:02:35.724371+0000","last_undegraded":"2024-09-10T22:02:35.724371+0000","last_fullsized":"2024-09-10T22:02:35.724371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:54.682292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.848412+0000","last_change":"2024-09-10T22:00:34.514348+0000","last_active":"2024-09-10T22:03:03.848412+0000","last_peered":"2024-09-10T22:03:03.848412+0000","last_clean":"2024-09-10T22:03:03.848412+0000","last_became_active":"2024-09-10T22:00:34.513797+0000","last_became_peered":"2024-09-10T22:00:34.513797+0000","last_unstale":"2024-09-10T22:03:03.848412+0000","last_undegraded":"2024-09-10T22:03:03.848412+0000","last_fullsized":"2024-09-10T22:03:03.848412+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:12.797643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728093+0000","last_change":"2024-09-10T22:00:30.826486+0000","last_active":"2024-09-10T22:02:35.728093+0000","last_peered":"2024-09-10T22:02:35.728093+0000","last_clean":"2024-09-10T22:02:35.728093+0000","last_became_active":"2024-09-10T22:00:30.826111+0000","last_became_peered":"2024-09-10T22:00:30.826111+0000","last_unstale":"2024-09-10T22:02:35.728093+0000","last_undegraded":"2024-09-10T22:02:35.728093+0000","last_fullsized":"2024-09-10T22:02:35.728093+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:45.609925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948482+0000","last_change":"2024-09-10T22:00:43.709432+0000","last_active":"2024-09-10T22:02:35.948482+0000","last_peered":"2024-09-10T22:02:35.948482+0000","last_clean":"2024-09-10T22:02:35.948482+0000","last_became_active":"2024-09-10T22:00:43.708627+0000","last_became_peered":"2024-09-10T22:00:43.708627+0000","last_unstale":"2024-09-10T22:02:35.948482+0000","last_undegraded":"2024-09-10T22:02:35.948482+0000","last_fullsized":"2024-09-10T22:02:35.948482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:12.799976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.848267+0000","last_change":"2024-09-10T22:00:34.499203+0000","last_active":"2024-09-10T22:03:03.848267+0000","last_peered":"2024-09-10T22:03:03.848267+0000","last_clean":"2024-09-10T22:03:03.848267+0000","last_became_active":"2024-09-10T22:00:34.497877+0000","last_became_peered":"2024-09-10T22:00:34.497877+0000","last_unstale":"2024-09-10T22:03:03.848267+0000","last_undegraded":"2024-09-10T22:03:03.848267+0000","last_fullsized":"2024-09-10T22:03:03.848267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:41.897603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723973+0000","last_change":"2024-09-10T22:00:30.820852+0000","last_active":"2024-09-10T22:02:35.723973+0000","last_peered":"2024-09-10T22:02:35.723973+0000","last_clean":"2024-09-10T22:02:35.723973+0000","last_became_active":"2024-09-10T22:00:30.820023+0000","last_became_peered":"2024-09-10T22:00:30.820023+0000","last_unstale":"2024-09-10T22:02:35.723973+0000","last_undegraded":"2024-09-10T22:02:35.723973+0000","last_fullsized":"2024-09-10T22:02:35.723973+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:54.245142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727813+0000","last_change":"2024-09-10T22:00:43.716323+0000","last_active":"2024-09-10T22:02:35.727813+0000","last_peered":"2024-09-10T22:02:35.727813+0000","last_clean":"2024-09-10T22:02:35.727813+0000","last_became_active":"2024-09-10T22:00:43.715827+0000","last_became_peered":"2024-09-10T22:00:43.715827+0000","last_unstale":"2024-09-10T22:02:35.727813+0000","last_undegraded":"2024-09-10T22:02:35.727813+0000","last_fullsized":"2024-09-10T22:02:35.727813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:58:10.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.839342+0000","last_change":"2024-09-10T22:00:34.500114+0000","last_active":"2024-09-10T22:03:03.839342+0000","last_peered":"2024-09-10T22:03:03.839342+0000","last_clean":"2024-09-10T22:03:03.839342+0000","last_became_active":"2024-09-10T22:00:34.499334+0000","last_became_peered":"2024-09-10T22:00:34.499334+0000","last_unstale":"2024-09-10T22:03:03.839342+0000","last_undegraded":"2024-09-10T22:03:03.839342+0000","last_fullsized":"2024-09-10T22:03:03.839342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:11.203134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'40","reported_seq":369,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:49.006270+0000","last_change":"2024-09-10T22:00:30.822561+0000","last_active":"2024-09-10T22:03:49.006270+0000","last_peered":"2024-09-10T22:03:49.006270+0000","last_clean":"2024-09-10T22:03:49.006270+0000","last_became_active":"2024-09-10T22:00:30.822201+0000","last_became_peered":"2024-09-10T22:00:30.822201+0000","last_unstale":"2024-09-10T22:03:49.006270+0000","last_undegraded":"2024-09-10T22:03:49.006270+0000","last_fullsized":"2024-09-10T22:03:49.006270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:41.476349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67713,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":285,"num_read_kb":209,"num_write":42,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871420+0000","last_change":"2024-09-10T22:01:35.444988+0000","last_active":"2024-09-10T22:02:35.871420+0000","last_peered":"2024-09-10T22:02:35.871420+0000","last_clean":"2024-09-10T22:02:35.871420+0000","last_became_active":"2024-09-10T22:01:35.444498+0000","last_became_peered":"2024-09-10T22:01:35.444498+0000","last_unstale":"2024-09-10T22:02:35.871420+0000","last_undegraded":"2024-09-10T22:02:35.871420+0000","last_fullsized":"2024-09-10T22:02:35.871420+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:35.017660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.852407+0000","last_change":"2024-09-10T22:00:34.503234+0000","last_active":"2024-09-10T22:03:03.852407+0000","last_peered":"2024-09-10T22:03:03.852407+0000","last_clean":"2024-09-10T22:03:03.852407+0000","last_became_active":"2024-09-10T22:00:34.501501+0000","last_became_peered":"2024-09-10T22:00:34.501501+0000","last_unstale":"2024-09-10T22:03:03.852407+0000","last_undegraded":"2024-09-10T22:03:03.852407+0000","last_fullsized":"2024-09-10T22:03:03.852407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:23.772491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.728102+0000","last_change":"2024-09-10T22:00:31.082131+0000","last_active":"2024-09-10T22:02:35.728102+0000","last_peered":"2024-09-10T22:02:35.728102+0000","last_clean":"2024-09-10T22:02:35.728102+0000","last_became_active":"2024-09-10T22:00:31.081405+0000","last_became_peered":"2024-09-10T22:00:31.081405+0000","last_unstale":"2024-09-10T22:02:35.728102+0000","last_undegraded":"2024-09-10T22:02:35.728102+0000","last_fullsized":"2024-09-10T22:02:35.728102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:01:02.400004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.897415+0000","last_change":"2024-09-10T22:00:43.705732+0000","last_active":"2024-09-10T22:02:35.897415+0000","last_peered":"2024-09-10T22:02:35.897415+0000","last_clean":"2024-09-10T22:02:35.897415+0000","last_became_active":"2024-09-10T22:00:43.705274+0000","last_became_peered":"2024-09-10T22:00:43.705274+0000","last_unstale":"2024-09-10T22:02:35.897415+0000","last_undegraded":"2024-09-10T22:02:35.897415+0000","last_fullsized":"2024-09-10T22:02:35.897415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:54.707636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.845781+0000","last_change":"2024-09-10T22:00:34.499585+0000","last_active":"2024-09-10T22:03:03.845781+0000","last_peered":"2024-09-10T22:03:03.845781+0000","last_clean":"2024-09-10T22:03:03.845781+0000","last_became_active":"2024-09-10T22:00:34.499261+0000","last_became_peered":"2024-09-10T22:00:34.499261+0000","last_unstale":"2024-09-10T22:03:03.845781+0000","last_undegraded":"2024-09-10T22:03:03.845781+0000","last_fullsized":"2024-09-10T22:03:03.845781+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:41.156093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.863703+0000","last_change":"2024-09-10T22:00:30.823322+0000","last_active":"2024-09-10T22:02:35.863703+0000","last_peered":"2024-09-10T22:02:35.863703+0000","last_clean":"2024-09-10T22:02:35.863703+0000","last_became_active":"2024-09-10T22:00:30.822987+0000","last_became_peered":"2024-09-10T22:00:30.822987+0000","last_unstale":"2024-09-10T22:02:35.863703+0000","last_undegraded":"2024-09-10T22:02:35.863703+0000","last_fullsized":"2024-09-10T22:02:35.863703+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:33.605627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723255+0000","last_change":"2024-09-10T22:00:43.708746+0000","last_active":"2024-09-10T22:02:35.723255+0000","last_peered":"2024-09-10T22:02:35.723255+0000","last_clean":"2024-09-10T22:02:35.723255+0000","last_became_active":"2024-09-10T22:00:43.708283+0000","last_became_peered":"2024-09-10T22:00:43.708283+0000","last_unstale":"2024-09-10T22:02:35.723255+0000","last_undegraded":"2024-09-10T22:02:35.723255+0000","last_fullsized":"2024-09-10T22:02:35.723255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:30.015617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.854268+0000","last_change":"2024-09-10T22:00:34.501472+0000","last_active":"2024-09-10T22:03:03.854268+0000","last_peered":"2024-09-10T22:03:03.854268+0000","last_clean":"2024-09-10T22:03:03.854268+0000","last_became_active":"2024-09-10T22:00:34.501131+0000","last_became_peered":"2024-09-10T22:00:34.501131+0000","last_unstale":"2024-09-10T22:03:03.854268+0000","last_undegraded":"2024-09-10T22:03:03.854268+0000","last_fullsized":"2024-09-10T22:03:03.854268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:11.281413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'8","reported_seq":326,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:49.004930+0000","last_change":"2024-09-10T22:00:30.828335+0000","last_active":"2024-09-10T22:03:49.004930+0000","last_peered":"2024-09-10T22:03:49.004930+0000","last_clean":"2024-09-10T22:03:49.004930+0000","last_became_active":"2024-09-10T22:00:30.828028+0000","last_became_peered":"2024-09-10T22:00:30.828028+0000","last_unstale":"2024-09-10T22:03:49.004930+0000","last_undegraded":"2024-09-10T22:03:49.004930+0000","last_fullsized":"2024-09-10T22:03:49.004930+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:56.399002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":274,"num_read_kb":274,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.722469+0000","last_change":"2024-09-10T22:00:43.710086+0000","last_active":"2024-09-10T22:02:35.722469+0000","last_peered":"2024-09-10T22:02:35.722469+0000","last_clean":"2024-09-10T22:02:35.722469+0000","last_became_active":"2024-09-10T22:00:43.709518+0000","last_became_peered":"2024-09-10T22:00:43.709518+0000","last_unstale":"2024-09-10T22:02:35.722469+0000","last_undegraded":"2024-09-10T22:02:35.722469+0000","last_fullsized":"2024-09-10T22:02:35.722469+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:07.636887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.947965+0000","last_change":"2024-09-10T22:00:43.711620+0000","last_active":"2024-09-10T22:02:35.947965+0000","last_peered":"2024-09-10T22:02:35.947965+0000","last_clean":"2024-09-10T22:02:35.947965+0000","last_became_active":"2024-09-10T22:00:43.711224+0000","last_became_peered":"2024-09-10T22:00:43.711224+0000","last_unstale":"2024-09-10T22:02:35.947965+0000","last_undegraded":"2024-09-10T22:02:35.947965+0000","last_fullsized":"2024-09-10T22:02:35.947965+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:43.860286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841267+0000","last_change":"2024-09-10T22:00:34.494408+0000","last_active":"2024-09-10T22:03:03.841267+0000","last_peered":"2024-09-10T22:03:03.841267+0000","last_clean":"2024-09-10T22:03:03.841267+0000","last_became_active":"2024-09-10T22:00:34.494066+0000","last_became_peered":"2024-09-10T22:00:34.494066+0000","last_unstale":"2024-09-10T22:03:03.841267+0000","last_undegraded":"2024-09-10T22:03:03.841267+0000","last_fullsized":"2024-09-10T22:03:03.841267+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:49.565755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.727709+0000","last_change":"2024-09-10T22:00:30.821842+0000","last_active":"2024-09-10T22:02:35.727709+0000","last_peered":"2024-09-10T22:02:35.727709+0000","last_clean":"2024-09-10T22:02:35.727709+0000","last_became_active":"2024-09-10T22:00:30.821499+0000","last_became_peered":"2024-09-10T22:00:30.821499+0000","last_unstale":"2024-09-10T22:02:35.727709+0000","last_undegraded":"2024-09-10T22:02:35.727709+0000","last_fullsized":"2024-09-10T22:02:35.727709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:19.125052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.948928+0000","last_change":"2024-09-10T22:00:43.706689+0000","last_active":"2024-09-10T22:02:35.948928+0000","last_peered":"2024-09-10T22:02:35.948928+0000","last_clean":"2024-09-10T22:02:35.948928+0000","last_became_active":"2024-09-10T22:00:43.705954+0000","last_became_peered":"2024-09-10T22:00:43.705954+0000","last_unstale":"2024-09-10T22:02:35.948928+0000","last_undegraded":"2024-09-10T22:02:35.948928+0000","last_fullsized":"2024-09-10T22:02:35.948928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:33:30.800710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.831092+0000","last_change":"2024-09-10T22:00:34.490853+0000","last_active":"2024-09-10T22:03:03.831092+0000","last_peered":"2024-09-10T22:03:03.831092+0000","last_clean":"2024-09-10T22:03:03.831092+0000","last_became_active":"2024-09-10T22:00:34.489908+0000","last_became_peered":"2024-09-10T22:00:34.489908+0000","last_unstale":"2024-09-10T22:03:03.831092+0000","last_undegraded":"2024-09-10T22:03:03.831092+0000","last_fullsized":"2024-09-10T22:03:03.831092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:57.105004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.893936+0000","last_change":"2024-09-10T22:00:30.816497+0000","last_active":"2024-09-10T22:02:35.893936+0000","last_peered":"2024-09-10T22:02:35.893936+0000","last_clean":"2024-09-10T22:02:35.893936+0000","last_became_active":"2024-09-10T22:00:30.816181+0000","last_became_peered":"2024-09-10T22:00:30.816181+0000","last_unstale":"2024-09-10T22:02:35.893936+0000","last_undegraded":"2024-09-10T22:02:35.893936+0000","last_fullsized":"2024-09-10T22:02:35.893936+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:06:12.779113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.871850+0000","last_change":"2024-09-10T22:00:30.821402+0000","last_active":"2024-09-10T22:02:35.871850+0000","last_peered":"2024-09-10T22:02:35.871850+0000","last_clean":"2024-09-10T22:02:35.871850+0000","last_became_active":"2024-09-10T22:00:30.821105+0000","last_became_peered":"2024-09-10T22:00:30.821105+0000","last_unstale":"2024-09-10T22:02:35.871850+0000","last_undegraded":"2024-09-10T22:02:35.871850+0000","last_fullsized":"2024-09-10T22:02:35.871850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:36.090579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840274+0000","last_change":"2024-09-10T22:00:34.511911+0000","last_active":"2024-09-10T22:03:03.840274+0000","last_peered":"2024-09-10T22:03:03.840274+0000","last_clean":"2024-09-10T22:03:03.840274+0000","last_became_active":"2024-09-10T22:00:34.511545+0000","last_became_peered":"2024-09-10T22:00:34.511545+0000","last_unstale":"2024-09-10T22:03:03.840274+0000","last_undegraded":"2024-09-10T22:03:03.840274+0000","last_fullsized":"2024-09-10T22:03:03.840274+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:12.484733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723568+0000","last_change":"2024-09-10T22:01:35.750458+0000","last_active":"2024-09-10T22:02:35.723568+0000","last_peered":"2024-09-10T22:02:35.723568+0000","last_clean":"2024-09-10T22:02:35.723568+0000","last_became_active":"2024-09-10T22:01:35.749946+0000","last_became_peered":"2024-09-10T22:01:35.749946+0000","last_unstale":"2024-09-10T22:02:35.723568+0000","last_undegraded":"2024-09-10T22:02:35.723568+0000","last_fullsized":"2024-09-10T22:02:35.723568+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:48.362221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":198,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:51.009495+0000","last_change":"2024-09-10T22:00:30.826427+0000","last_active":"2024-09-10T22:03:51.009495+0000","last_peered":"2024-09-10T22:03:51.009495+0000","last_clean":"2024-09-10T22:03:51.009495+0000","last_became_active":"2024-09-10T22:00:30.826038+0000","last_became_peered":"2024-09-10T22:00:30.826038+0000","last_unstale":"2024-09-10T22:03:51.009495+0000","last_undegraded":"2024-09-10T22:03:51.009495+0000","last_fullsized":"2024-09-10T22:03:51.009495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:29.794374+0000","last_clean_scrub_stamp":"2024-09-10T22:00:29.794374+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:45.075208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1647,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.833092+0000","last_change":"2024-09-10T22:00:34.494479+0000","last_active":"2024-09-10T22:03:03.833092+0000","last_peered":"2024-09-10T22:03:03.833092+0000","last_clean":"2024-09-10T22:03:03.833092+0000","last_became_active":"2024-09-10T22:00:34.494087+0000","last_became_peered":"2024-09-10T22:00:34.494087+0000","last_unstale":"2024-09-10T22:03:03.833092+0000","last_undegraded":"2024-09-10T22:03:03.833092+0000","last_fullsized":"2024-09-10T22:03:03.833092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:28.548999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:02:35.723643+0000","last_change":"2024-09-10T22:00:43.714568+0000","last_active":"2024-09-10T22:02:35.723643+0000","last_peered":"2024-09-10T22:02:35.723643+0000","last_clean":"2024-09-10T22:02:35.723643+0000","last_became_active":"2024-09-10T22:00:43.712923+0000","last_became_peered":"2024-09-10T22:00:43.712923+0000","last_unstale":"2024-09-10T22:02:35.723643+0000","last_undegraded":"2024-09-10T22:02:35.723643+0000","last_fullsized":"2024-09-10T22:02:35.723643+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:42.684205+0000","last_clean_scrub_stamp":"2024-09-10T22:00:42.684205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:04:27.945799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.833182+0000","last_change":"2024-09-10T22:00:34.496942+0000","last_active":"2024-09-10T22:03:03.833182+0000","last_peered":"2024-09-10T22:03:03.833182+0000","last_clean":"2024-09-10T22:03:03.833182+0000","last_became_active":"2024-09-10T22:00:34.496475+0000","last_became_peered":"2024-09-10T22:00:34.496475+0000","last_unstale":"2024-09-10T22:03:03.833182+0000","last_undegraded":"2024-09-10T22:03:03.833182+0000","last_fullsized":"2024-09-10T22:03:03.833182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:33.277346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837137+0000","last_change":"2024-09-10T22:00:34.506571+0000","last_active":"2024-09-10T22:03:03.837137+0000","last_peered":"2024-09-10T22:03:03.837137+0000","last_clean":"2024-09-10T22:03:03.837137+0000","last_became_active":"2024-09-10T22:00:34.505718+0000","last_became_peered":"2024-09-10T22:00:34.505718+0000","last_unstale":"2024-09-10T22:03:03.837137+0000","last_undegraded":"2024-09-10T22:03:03.837137+0000","last_fullsized":"2024-09-10T22:03:03.837137+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:10.111415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837418+0000","last_change":"2024-09-10T22:00:34.502030+0000","last_active":"2024-09-10T22:03:03.837418+0000","last_peered":"2024-09-10T22:03:03.837418+0000","last_clean":"2024-09-10T22:03:03.837418+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:03:03.837418+0000","last_undegraded":"2024-09-10T22:03:03.837418+0000","last_fullsized":"2024-09-10T22:03:03.837418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:09.532523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.846513+0000","last_change":"2024-09-10T22:00:34.513630+0000","last_active":"2024-09-10T22:03:03.846513+0000","last_peered":"2024-09-10T22:03:03.846513+0000","last_clean":"2024-09-10T22:03:03.846513+0000","last_became_active":"2024-09-10T22:00:34.513348+0000","last_became_peered":"2024-09-10T22:00:34.513348+0000","last_unstale":"2024-09-10T22:03:03.846513+0000","last_undegraded":"2024-09-10T22:03:03.846513+0000","last_fullsized":"2024-09-10T22:03:03.846513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:53.942004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.844287+0000","last_change":"2024-09-10T22:00:34.508697+0000","last_active":"2024-09-10T22:03:03.844287+0000","last_peered":"2024-09-10T22:03:03.844287+0000","last_clean":"2024-09-10T22:03:03.844287+0000","last_became_active":"2024-09-10T22:00:34.508371+0000","last_became_peered":"2024-09-10T22:00:34.508371+0000","last_unstale":"2024-09-10T22:03:03.844287+0000","last_undegraded":"2024-09-10T22:03:03.844287+0000","last_fullsized":"2024-09-10T22:03:03.844287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:32.843517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.832359+0000","last_change":"2024-09-10T22:00:34.496021+0000","last_active":"2024-09-10T22:03:03.832359+0000","last_peered":"2024-09-10T22:03:03.832359+0000","last_clean":"2024-09-10T22:03:03.832359+0000","last_became_active":"2024-09-10T22:00:34.495663+0000","last_became_peered":"2024-09-10T22:00:34.495663+0000","last_unstale":"2024-09-10T22:03:03.832359+0000","last_undegraded":"2024-09-10T22:03:03.832359+0000","last_fullsized":"2024-09-10T22:03:03.832359+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:16.512182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851192+0000","last_change":"2024-09-10T22:00:34.506589+0000","last_active":"2024-09-10T22:03:03.851192+0000","last_peered":"2024-09-10T22:03:03.851192+0000","last_clean":"2024-09-10T22:03:03.851192+0000","last_became_active":"2024-09-10T22:00:34.506214+0000","last_became_peered":"2024-09-10T22:00:34.506214+0000","last_unstale":"2024-09-10T22:03:03.851192+0000","last_undegraded":"2024-09-10T22:03:03.851192+0000","last_fullsized":"2024-09-10T22:03:03.851192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:47.072458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.825874+0000","last_change":"2024-09-10T22:00:34.503471+0000","last_active":"2024-09-10T22:03:03.825874+0000","last_peered":"2024-09-10T22:03:03.825874+0000","last_clean":"2024-09-10T22:03:03.825874+0000","last_became_active":"2024-09-10T22:00:34.501145+0000","last_became_peered":"2024-09-10T22:00:34.501145+0000","last_unstale":"2024-09-10T22:03:03.825874+0000","last_undegraded":"2024-09-10T22:03:03.825874+0000","last_fullsized":"2024-09-10T22:03:03.825874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:24.020056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840484+0000","last_change":"2024-09-10T22:00:34.490177+0000","last_active":"2024-09-10T22:03:03.840484+0000","last_peered":"2024-09-10T22:03:03.840484+0000","last_clean":"2024-09-10T22:03:03.840484+0000","last_became_active":"2024-09-10T22:00:34.488428+0000","last_became_peered":"2024-09-10T22:00:34.488428+0000","last_unstale":"2024-09-10T22:03:03.840484+0000","last_undegraded":"2024-09-10T22:03:03.840484+0000","last_fullsized":"2024-09-10T22:03:03.840484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:21.111826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841640+0000","last_change":"2024-09-10T22:00:34.499174+0000","last_active":"2024-09-10T22:03:03.841640+0000","last_peered":"2024-09-10T22:03:03.841640+0000","last_clean":"2024-09-10T22:03:03.841640+0000","last_became_active":"2024-09-10T22:00:34.498824+0000","last_became_peered":"2024-09-10T22:00:34.498824+0000","last_unstale":"2024-09-10T22:03:03.841640+0000","last_undegraded":"2024-09-10T22:03:03.841640+0000","last_fullsized":"2024-09-10T22:03:03.841640+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:15.946270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853015+0000","last_change":"2024-09-10T22:00:34.505748+0000","last_active":"2024-09-10T22:03:03.853015+0000","last_peered":"2024-09-10T22:03:03.853015+0000","last_clean":"2024-09-10T22:03:03.853015+0000","last_became_active":"2024-09-10T22:00:34.500441+0000","last_became_peered":"2024-09-10T22:00:34.500441+0000","last_unstale":"2024-09-10T22:03:03.853015+0000","last_undegraded":"2024-09-10T22:03:03.853015+0000","last_fullsized":"2024-09-10T22:03:03.853015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:22.177566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.850314+0000","last_change":"2024-09-10T22:00:34.502319+0000","last_active":"2024-09-10T22:03:03.850314+0000","last_peered":"2024-09-10T22:03:03.850314+0000","last_clean":"2024-09-10T22:03:03.850314+0000","last_became_active":"2024-09-10T22:00:34.501427+0000","last_became_peered":"2024-09-10T22:00:34.501427+0000","last_unstale":"2024-09-10T22:03:03.850314+0000","last_undegraded":"2024-09-10T22:03:03.850314+0000","last_fullsized":"2024-09-10T22:03:03.850314+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:18.463319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.842219+0000","last_change":"2024-09-10T22:00:34.497571+0000","last_active":"2024-09-10T22:03:03.842219+0000","last_peered":"2024-09-10T22:03:03.842219+0000","last_clean":"2024-09-10T22:03:03.842219+0000","last_became_active":"2024-09-10T22:00:34.497205+0000","last_became_peered":"2024-09-10T22:00:34.497205+0000","last_unstale":"2024-09-10T22:03:03.842219+0000","last_undegraded":"2024-09-10T22:03:03.842219+0000","last_fullsized":"2024-09-10T22:03:03.842219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:56:49.924518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837147+0000","last_change":"2024-09-10T22:00:34.504641+0000","last_active":"2024-09-10T22:03:03.837147+0000","last_peered":"2024-09-10T22:03:03.837147+0000","last_clean":"2024-09-10T22:03:03.837147+0000","last_became_active":"2024-09-10T22:00:34.504183+0000","last_became_peered":"2024-09-10T22:00:34.504183+0000","last_unstale":"2024-09-10T22:03:03.837147+0000","last_undegraded":"2024-09-10T22:03:03.837147+0000","last_fullsized":"2024-09-10T22:03:03.837147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:06.704972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851050+0000","last_change":"2024-09-10T22:00:34.503884+0000","last_active":"2024-09-10T22:03:03.851050+0000","last_peered":"2024-09-10T22:03:03.851050+0000","last_clean":"2024-09-10T22:03:03.851050+0000","last_became_active":"2024-09-10T22:00:34.502950+0000","last_became_peered":"2024-09-10T22:00:34.502950+0000","last_unstale":"2024-09-10T22:03:03.851050+0000","last_undegraded":"2024-09-10T22:03:03.851050+0000","last_fullsized":"2024-09-10T22:03:03.851050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:09:45.560958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.850635+0000","last_change":"2024-09-10T22:00:34.504981+0000","last_active":"2024-09-10T22:03:03.850635+0000","last_peered":"2024-09-10T22:03:03.850635+0000","last_clean":"2024-09-10T22:03:03.850635+0000","last_became_active":"2024-09-10T22:00:34.500474+0000","last_became_peered":"2024-09-10T22:00:34.500474+0000","last_unstale":"2024-09-10T22:03:03.850635+0000","last_undegraded":"2024-09-10T22:03:03.850635+0000","last_fullsized":"2024-09-10T22:03:03.850635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:20.892605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853045+0000","last_change":"2024-09-10T22:00:34.506650+0000","last_active":"2024-09-10T22:03:03.853045+0000","last_peered":"2024-09-10T22:03:03.853045+0000","last_clean":"2024-09-10T22:03:03.853045+0000","last_became_active":"2024-09-10T22:00:34.505698+0000","last_became_peered":"2024-09-10T22:00:34.505698+0000","last_unstale":"2024-09-10T22:03:03.853045+0000","last_undegraded":"2024-09-10T22:03:03.853045+0000","last_fullsized":"2024-09-10T22:03:03.853045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:29.221693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841202+0000","last_change":"2024-09-10T22:00:34.502238+0000","last_active":"2024-09-10T22:03:03.841202+0000","last_peered":"2024-09-10T22:03:03.841202+0000","last_clean":"2024-09-10T22:03:03.841202+0000","last_became_active":"2024-09-10T22:00:34.500049+0000","last_became_peered":"2024-09-10T22:00:34.500049+0000","last_unstale":"2024-09-10T22:03:03.841202+0000","last_undegraded":"2024-09-10T22:03:03.841202+0000","last_fullsized":"2024-09-10T22:03:03.841202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:05.669784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.839971+0000","last_change":"2024-09-10T22:00:34.499340+0000","last_active":"2024-09-10T22:03:03.839971+0000","last_peered":"2024-09-10T22:03:03.839971+0000","last_clean":"2024-09-10T22:03:03.839971+0000","last_became_active":"2024-09-10T22:00:34.497655+0000","last_became_peered":"2024-09-10T22:00:34.497655+0000","last_unstale":"2024-09-10T22:03:03.839971+0000","last_undegraded":"2024-09-10T22:03:03.839971+0000","last_fullsized":"2024-09-10T22:03:03.839971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:21:34.115829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838094+0000","last_change":"2024-09-10T22:00:34.500705+0000","last_active":"2024-09-10T22:03:03.838094+0000","last_peered":"2024-09-10T22:03:03.838094+0000","last_clean":"2024-09-10T22:03:03.838094+0000","last_became_active":"2024-09-10T22:00:34.500071+0000","last_became_peered":"2024-09-10T22:00:34.500071+0000","last_unstale":"2024-09-10T22:03:03.838094+0000","last_undegraded":"2024-09-10T22:03:03.838094+0000","last_fullsized":"2024-09-10T22:03:03.838094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:40.519976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.838120+0000","last_change":"2024-09-10T22:00:34.510570+0000","last_active":"2024-09-10T22:03:03.838120+0000","last_peered":"2024-09-10T22:03:03.838120+0000","last_clean":"2024-09-10T22:03:03.838120+0000","last_became_active":"2024-09-10T22:00:34.506922+0000","last_became_peered":"2024-09-10T22:00:34.506922+0000","last_unstale":"2024-09-10T22:03:03.838120+0000","last_undegraded":"2024-09-10T22:03:03.838120+0000","last_fullsized":"2024-09-10T22:03:03.838120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:36.489031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.853043+0000","last_change":"2024-09-10T22:00:34.509985+0000","last_active":"2024-09-10T22:03:03.853043+0000","last_peered":"2024-09-10T22:03:03.853043+0000","last_clean":"2024-09-10T22:03:03.853043+0000","last_became_active":"2024-09-10T22:00:34.509527+0000","last_became_peered":"2024-09-10T22:00:34.509527+0000","last_unstale":"2024-09-10T22:03:03.853043+0000","last_undegraded":"2024-09-10T22:03:03.853043+0000","last_fullsized":"2024-09-10T22:03:03.853043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:20:49.463334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.839704+0000","last_change":"2024-09-10T22:00:34.497986+0000","last_active":"2024-09-10T22:03:03.839704+0000","last_peered":"2024-09-10T22:03:03.839704+0000","last_clean":"2024-09-10T22:03:03.839704+0000","last_became_active":"2024-09-10T22:00:34.497612+0000","last_became_peered":"2024-09-10T22:00:34.497612+0000","last_unstale":"2024-09-10T22:03:03.839704+0000","last_undegraded":"2024-09-10T22:03:03.839704+0000","last_fullsized":"2024-09-10T22:03:03.839704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:15.696988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851846+0000","last_change":"2024-09-10T22:00:34.498482+0000","last_active":"2024-09-10T22:03:03.851846+0000","last_peered":"2024-09-10T22:03:03.851846+0000","last_clean":"2024-09-10T22:03:03.851846+0000","last_became_active":"2024-09-10T22:00:34.498139+0000","last_became_peered":"2024-09-10T22:00:34.498139+0000","last_unstale":"2024-09-10T22:03:03.851846+0000","last_undegraded":"2024-09-10T22:03:03.851846+0000","last_fullsized":"2024-09-10T22:03:03.851846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:23.427253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841157+0000","last_change":"2024-09-10T22:00:34.491325+0000","last_active":"2024-09-10T22:03:03.841157+0000","last_peered":"2024-09-10T22:03:03.841157+0000","last_clean":"2024-09-10T22:03:03.841157+0000","last_became_active":"2024-09-10T22:00:34.491026+0000","last_became_peered":"2024-09-10T22:00:34.491026+0000","last_unstale":"2024-09-10T22:03:03.841157+0000","last_undegraded":"2024-09-10T22:03:03.841157+0000","last_fullsized":"2024-09-10T22:03:03.841157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:07.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.843338+0000","last_change":"2024-09-10T22:00:34.503975+0000","last_active":"2024-09-10T22:03:03.843338+0000","last_peered":"2024-09-10T22:03:03.843338+0000","last_clean":"2024-09-10T22:03:03.843338+0000","last_became_active":"2024-09-10T22:00:34.503624+0000","last_became_peered":"2024-09-10T22:00:34.503624+0000","last_unstale":"2024-09-10T22:03:03.843338+0000","last_undegraded":"2024-09-10T22:03:03.843338+0000","last_fullsized":"2024-09-10T22:03:03.843338+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:17.678811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.837903+0000","last_change":"2024-09-10T22:01:35.596824+0000","last_active":"2024-09-10T22:03:03.837903+0000","last_peered":"2024-09-10T22:03:03.837903+0000","last_clean":"2024-09-10T22:03:03.837903+0000","last_became_active":"2024-09-10T22:01:35.596550+0000","last_became_peered":"2024-09-10T22:01:35.596550+0000","last_unstale":"2024-09-10T22:03:03.837903+0000","last_undegraded":"2024-09-10T22:03:03.837903+0000","last_fullsized":"2024-09-10T22:03:03.837903+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:19.617649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.832879+0000","last_change":"2024-09-10T22:02:35.895282+0000","last_active":"2024-09-10T22:03:03.832879+0000","last_peered":"2024-09-10T22:03:03.832879+0000","last_clean":"2024-09-10T22:03:03.832879+0000","last_became_active":"2024-09-10T22:02:35.894850+0000","last_became_peered":"2024-09-10T22:02:35.894850+0000","last_unstale":"2024-09-10T22:03:03.832879+0000","last_undegraded":"2024-09-10T22:03:03.832879+0000","last_fullsized":"2024-09-10T22:03:03.832879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:50.109012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.840009+0000","last_change":"2024-09-10T22:00:34.513134+0000","last_active":"2024-09-10T22:03:03.840009+0000","last_peered":"2024-09-10T22:03:03.840009+0000","last_clean":"2024-09-10T22:03:03.840009+0000","last_became_active":"2024-09-10T22:00:34.507578+0000","last_became_peered":"2024-09-10T22:00:34.507578+0000","last_unstale":"2024-09-10T22:03:03.840009+0000","last_undegraded":"2024-09-10T22:03:03.840009+0000","last_fullsized":"2024-09-10T22:03:03.840009+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:25:33.387337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.851019+0000","last_change":"2024-09-10T22:00:34.507645+0000","last_active":"2024-09-10T22:03:03.851019+0000","last_peered":"2024-09-10T22:03:03.851019+0000","last_clean":"2024-09-10T22:03:03.851019+0000","last_became_active":"2024-09-10T22:00:34.503614+0000","last_became_peered":"2024-09-10T22:00:34.503614+0000","last_unstale":"2024-09-10T22:03:03.851019+0000","last_undegraded":"2024-09-10T22:03:03.851019+0000","last_fullsized":"2024-09-10T22:03:03.851019+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:53.033523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.841836+0000","last_change":"2024-09-10T22:00:34.501891+0000","last_active":"2024-09-10T22:03:03.841836+0000","last_peered":"2024-09-10T22:03:03.841836+0000","last_clean":"2024-09-10T22:03:03.841836+0000","last_became_active":"2024-09-10T22:00:34.501134+0000","last_became_peered":"2024-09-10T22:00:34.501134+0000","last_unstale":"2024-09-10T22:03:03.841836+0000","last_undegraded":"2024-09-10T22:03:03.841836+0000","last_fullsized":"2024-09-10T22:03:03.841836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:30.068461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-10T22:03:03.842854+0000","last_change":"2024-09-10T22:00:34.501475+0000","last_active":"2024-09-10T22:03:03.842854+0000","last_peered":"2024-09-10T22:03:03.842854+0000","last_clean":"2024-09-10T22:03:03.842854+0000","last_became_active":"2024-09-10T22:00:34.500909+0000","last_became_peered":"2024-09-10T22:00:34.500909+0000","last_unstale":"2024-09-10T22:03:03.842854+0000","last_undegraded":"2024-09-10T22:03:03.842854+0000","last_fullsized":"2024-09-10T22:03:03.842854+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T22:00:33.463455+0000","last_clean_scrub_stamp":"2024-09-10T22:00:33.463455+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:39.616773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":865227,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1761,"num_read_kb":1097,"num_write":503,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":112,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":823296,"data_stored":2595681,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":383,"ondisk_log_size":383,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":246272,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678071,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":536890,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874300,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1212416,"data_stored":1585083,"data_compressed":86530,"data_compressed_allocated":114688,"data_compressed_original":1048576,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037827,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":932995,"data_compressed":51846,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234058,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":929165,"data_compressed":51846,"data_compressed_allocated":69632,"data_compressed_original":483328,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430288,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":1135242,"data_compressed":68439,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593815,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1003520,"data_stored":532212,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790046,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1138688,"data_stored":1522550,"data_compressed":86530,"data_compressed_allocated":114688,"data_compressed_original":1048576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953572,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":536286,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149801,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1093632,"data_stored":1317887,"data_compressed":69937,"data_compressed_allocated":94208,"data_compressed_original":851968,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378735,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":597774,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542262,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":601067,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705787,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":533668,"data_compressed":32557,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34248,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35882,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":787422,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":397621,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":464250,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T22:03:56.338 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:03:56.722 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:56.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:56 smithi090 ceph-mon[30350]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:03:56.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:56 smithi090 ceph-mon[30350]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 59 op/s 2024-09-10T22:03:56.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:56 smithi171 ceph-mon[34138]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:03:56.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:56 smithi171 ceph-mon[34138]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 59 op/s 2024-09-10T22:03:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:56 smithi154 ceph-mon[34495]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T22:03:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:56 smithi154 ceph-mon[34495]: pgmap v419: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 175 KiB/s wr, 59 op/s 2024-09-10T22:03:57.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.362+0000 7f37b9800640 1 -- 172.21.15.90:0/2650059573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108620 msgr2=0x7f37b4108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:57.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.362+0000 7f37b9800640 1 --2- 172.21.15.90:0/2650059573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108620 0x7f37b4108a60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f379c009900 tx=0x7f379c02a400 comp rx=0 tx=0).stop 2024-09-10T22:03:57.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.362+0000 7f37b9800640 1 -- 172.21.15.90:0/2650059573 shutdown_connections 2024-09-10T22:03:57.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.362+0000 7f37b9800640 1 --2- 172.21.15.90:0/2650059573 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37b4108fa0 0x7f37b410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.362+0000 7f37b9800640 1 --2- 172.21.15.90:0/2650059573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108620 0x7f37b4108a60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.362+0000 7f37b9800640 1 --2- 172.21.15.90:0/2650059573 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b4102c10 0x7f37b4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.362+0000 7f37b9800640 1 -- 172.21.15.90:0/2650059573 >> 172.21.15.90:0/2650059573 conn(0x7f37b40fe660 msgr2=0x7f37b4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.362+0000 7f37b9800640 1 -- 172.21.15.90:0/2650059573 shutdown_connections 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.362+0000 7f37b9800640 1 -- 172.21.15.90:0/2650059573 wait complete. 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.363+0000 7f37b9800640 1 Processor -- start 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.363+0000 7f37b9800640 1 -- start start 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.363+0000 7f37b9800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b4102c10 0x7f37b40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37b9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108fa0 0x7f37b40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37b9800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37b4075f20 0x7f37b41a7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37b9800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f37b4112b60 con 0x7f37b4108fa0 2024-09-10T22:03:57.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37b9800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f37b41129e0 con 0x7f37b4075f20 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37b9800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f37b4112ce0 con 0x7f37b4102c10 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37abe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108fa0 0x7f37b40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37abe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108fa0 0x7f37b40759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60780/0 (socket says 172.21.15.90:60780) 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37abe00640 1 -- 172.21.15.90:0/2632559867 learned_addr learned my addr 172.21.15.90:0/2632559867 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37b8e00640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37b4075f20 0x7f37b41a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37b3e00640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b4102c10 0x7f37b40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37abe00640 1 -- 172.21.15.90:0/2632559867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b4102c10 msgr2=0x7f37b40754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37abe00640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b4102c10 0x7f37b40754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37abe00640 1 -- 172.21.15.90:0/2632559867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37b4075f20 msgr2=0x7f37b41a7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37abe00640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37b4075f20 0x7f37b41a7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.364+0000 7f37abe00640 1 -- 172.21.15.90:0/2632559867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b41a79d0 con 0x7f37b4108fa0 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.365+0000 7f37abe00640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108fa0 0x7f37b40759e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f379c0057c0 tx=0x7f379c003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.365+0000 7f37b1600640 1 -- 172.21.15.90:0/2632559867 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c03f020 con 0x7f37b4108fa0 2024-09-10T22:03:57.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.365+0000 7f37b1600640 1 -- 172.21.15.90:0/2632559867 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f379c03bd00 con 0x7f37b4108fa0 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.365+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b41a7c60 con 0x7f37b4108fa0 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.365+0000 7f37b1600640 1 -- 172.21.15.90:0/2632559867 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c035070 con 0x7f37b4108fa0 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.365+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b41a8110 con 0x7f37b4108fa0 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.366+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b4108a60 con 0x7f37b4108fa0 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.368+0000 7f37b1600640 1 -- 172.21.15.90:0/2632559867 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f379c03b870 con 0x7f37b4108fa0 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.370+0000 7f37b1600640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37980b5520 0x7f37980b79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.370+0000 7f37b1600640 1 -- 172.21.15.90:0/2632559867 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f379c106d10 con 0x7f37b4108fa0 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.370+0000 7f37b3e00640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37980b5520 0x7f37980b79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.371+0000 7f37b3e00640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37980b5520 0x7f37980b79e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f37a4005fd0 tx=0x7f37a400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:57.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.374+0000 7f37b1600640 1 -- 172.21.15.90:0/2632559867 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f379c10c020 con 0x7f37b4108fa0 2024-09-10T22:03:57.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.595+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37b402e5c0 con 0x7f37b4108fa0 2024-09-10T22:03:57.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.596+0000 7f37b1600640 1 -- 172.21.15.90:0/2632559867 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11986 (secure 0 0 0) 0x7f379c049020 con 0x7f37b4108fa0 2024-09-10T22:03:57.597 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:57.597 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"btime":"2024-09-10T22:03:47:815790+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:03:47.057424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:03:57.597 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 30 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37980b5520 msgr2=0x7f37980b79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37980b5520 0x7f37980b79e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f37a4005fd0 tx=0x7f37a400b390 comp rx=0 tx=0).stop 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108fa0 msgr2=0x7f37b40759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108fa0 0x7f37b40759e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f379c0057c0 tx=0x7f379c003d50 comp rx=0 tx=0).stop 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 shutdown_connections 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37980b5520 0x7f37980b79e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37b4075f20 0x7f37b41a7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b4108fa0 0x7f37b40759e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 --2- 172.21.15.90:0/2632559867 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37b4102c10 0x7f37b40754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 >> 172.21.15.90:0/2632559867 conn(0x7f37b40fe660 msgr2=0x7f37b4100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:57.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 shutdown_connections 2024-09-10T22:03:57.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:57.600+0000 7f37b9800640 1 -- 172.21.15.90:0/2632559867 wait complete. 2024-09-10T22:03:57.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:57 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2632559867' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:03:57.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:57 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2632559867' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:03:58.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:57 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2632559867' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:03:58.368 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:03:58.740 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:03:58.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:58 smithi171 ceph-mon[34138]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-10T22:03:59.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:58 smithi090 ceph-mon[30350]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-10T22:03:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:58 smithi154 ceph-mon[34495]: pgmap v420: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 56 op/s 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.446+0000 7f2faa000640 1 -- 172.21.15.90:0/2729921647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410f760 msgr2=0x7f2fa4115f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.446+0000 7f2faa000640 1 --2- 172.21.15.90:0/2729921647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410f760 0x7f2fa4115f60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009900 tx=0x7f2f9c02a400 comp rx=0 tx=0).stop 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.446+0000 7f2faa000640 1 -- 172.21.15.90:0/2729921647 shutdown_connections 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.446+0000 7f2faa000640 1 --2- 172.21.15.90:0/2729921647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410f760 0x7f2fa4115f60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.446+0000 7f2faa000640 1 --2- 172.21.15.90:0/2729921647 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa410ede0 0x7f2fa410f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.446+0000 7f2faa000640 1 --2- 172.21.15.90:0/2729921647 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa41093d0 0x7f2fa41097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.446+0000 7f2faa000640 1 -- 172.21.15.90:0/2729921647 >> 172.21.15.90:0/2729921647 conn(0x7f2fa4078a40 msgr2=0x7f2fa407ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.447+0000 7f2faa000640 1 -- 172.21.15.90:0/2729921647 shutdown_connections 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.447+0000 7f2faa000640 1 -- 172.21.15.90:0/2729921647 wait complete. 2024-09-10T22:03:59.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.448+0000 7f2faa000640 1 Processor -- start 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.448+0000 7f2faa000640 1 -- start start 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.448+0000 7f2faa000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa41093d0 0x7f2fa419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.448+0000 7f2faa000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410ede0 0x7f2fa419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.448+0000 7f2faa000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa410f760 0x7f2fa41a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.448+0000 7f2faa000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa4119240 con 0x7f2fa410ede0 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.448+0000 7f2faa000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa41190c0 con 0x7f2fa410f760 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.448+0000 7f2faa000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa41193c0 con 0x7f2fa41093d0 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.448+0000 7f2fa2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410ede0 0x7f2fa419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410ede0 0x7f2fa419fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48952/0 (socket says 172.21.15.90:48952) 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa2a00640 1 -- 172.21.15.90:0/246958120 learned_addr learned my addr 172.21.15.90:0/246958120 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa3400640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa41093d0 0x7f2fa419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa3e00640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa410f760 0x7f2fa41a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:59.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa2a00640 1 -- 172.21.15.90:0/246958120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa41093d0 msgr2=0x7f2fa419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:59.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa2a00640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa41093d0 0x7f2fa419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa2a00640 1 -- 172.21.15.90:0/246958120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa410f760 msgr2=0x7f2fa41a4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:59.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa2a00640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa410f760 0x7f2fa41a4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa2a00640 1 -- 172.21.15.90:0/246958120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa41a5760 con 0x7f2fa410ede0 2024-09-10T22:03:59.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.449+0000 7f2fa2a00640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410ede0 0x7f2fa419fbf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000d950 tx=0x7f2f9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:59.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.450+0000 7f2f9be00640 1 -- 172.21.15.90:0/246958120 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90017030 con 0x7f2fa410ede0 2024-09-10T22:03:59.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.450+0000 7f2f9be00640 1 -- 172.21.15.90:0/246958120 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f90004020 con 0x7f2fa410ede0 2024-09-10T22:03:59.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.450+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa41a5a50 con 0x7f2fa410ede0 2024-09-10T22:03:59.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.450+0000 7f2f9be00640 1 -- 172.21.15.90:0/246958120 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f900055d0 con 0x7f2fa410ede0 2024-09-10T22:03:59.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.450+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa41ac110 con 0x7f2fa410ede0 2024-09-10T22:03:59.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.451+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa41097b0 con 0x7f2fa410ede0 2024-09-10T22:03:59.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.453+0000 7f2f9be00640 1 -- 172.21.15.90:0/246958120 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f2f90009a30 con 0x7f2fa410ede0 2024-09-10T22:03:59.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.454+0000 7f2f9be00640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f880b53c0 0x7f2f880b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:03:59.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.455+0000 7f2f9be00640 1 -- 172.21.15.90:0/246958120 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f2f900e5c10 con 0x7f2fa410ede0 2024-09-10T22:03:59.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.455+0000 7f2fa3400640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f880b53c0 0x7f2f880b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:03:59.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.455+0000 7f2fa3400640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f880b53c0 0x7f2f880b7880 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f94007ec0 tx=0x7f2f940075b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:03:59.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.458+0000 7f2f9be00640 1 -- 172.21.15.90:0/246958120 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2f900ac480 con 0x7f2fa410ede0 2024-09-10T22:03:59.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.679+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fa41ac4e0 con 0x7f2fa410ede0 2024-09-10T22:03:59.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.680+0000 7f2f9be00640 1 -- 172.21.15.90:0/246958120 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11986 (secure 0 0 0) 0x7f2f900abd80 con 0x7f2fa410ede0 2024-09-10T22:03:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:03:59.682 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"btime":"2024-09-10T22:03:47:815790+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:03:47.057424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:03:59.683 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 30 2024-09-10T22:03:59.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.682+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f880b53c0 msgr2=0x7f2f880b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:59.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.682+0000 7f2faa000640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f880b53c0 0x7f2f880b7880 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2f94007ec0 tx=0x7f2f940075b0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.682+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410ede0 msgr2=0x7f2fa419fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:03:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.682+0000 7f2faa000640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410ede0 0x7f2fa419fbf0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000d950 tx=0x7f2f9000de10 comp rx=0 tx=0).stop 2024-09-10T22:03:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.683+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 shutdown_connections 2024-09-10T22:03:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.683+0000 7f2faa000640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f880b53c0 0x7f2f880b7880 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.683+0000 7f2faa000640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2fa410f760 0x7f2fa41a4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.683+0000 7f2faa000640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2fa410ede0 0x7f2fa419fbf0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.683+0000 7f2faa000640 1 --2- 172.21.15.90:0/246958120 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2fa41093d0 0x7f2fa419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:03:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.683+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 >> 172.21.15.90:0/246958120 conn(0x7f2fa4078a40 msgr2=0x7f2fa407ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:03:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.683+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 shutdown_connections 2024-09-10T22:03:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:03:59.683+0000 7f2faa000640 1 -- 172.21.15.90:0/246958120 wait complete. 2024-09-10T22:03:59.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:03:59 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/246958120' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:00.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:03:59 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/246958120' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:03:59 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/246958120' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:00.482 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:04:00.850 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:00.989 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:00 smithi090 ceph-mon[30350]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-10T22:04:00.989 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:04:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:04:00.907+0000 7fb78c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:00.989 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:04:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:04:00.907+0000 7fb78c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:00.989 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:04:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:04:00.908+0000 7fb78c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:00.989 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:04:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:04:00.908+0000 7fb78c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:00.989 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:04:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:04:00.908+0000 7fb78c800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T22:04:00.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:00 smithi171 ceph-mon[34138]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-10T22:04:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:00 smithi154 ceph-mon[34495]: pgmap v421: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 175 KiB/s wr, 58 op/s 2024-09-10T22:04:01.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.785+0000 7f2561000640 1 -- 172.21.15.90:0/4189468356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c10a820 msgr2=0x7f255c10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:01.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.785+0000 7f2561000640 1 --2- 172.21.15.90:0/4189468356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c10a820 0x7f255c10ac60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2544009900 tx=0x7f254402a410 comp rx=0 tx=0).stop 2024-09-10T22:04:01.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.785+0000 7f2561000640 1 -- 172.21.15.90:0/4189468356 shutdown_connections 2024-09-10T22:04:01.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.785+0000 7f2561000640 1 --2- 172.21.15.90:0/4189468356 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f255c10b1a0 0x7f255c1119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.785+0000 7f2561000640 1 --2- 172.21.15.90:0/4189468356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c10a820 0x7f255c10ac60 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.785+0000 7f2561000640 1 --2- 172.21.15.90:0/4189468356 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f255c104e10 0x7f255c1051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.785+0000 7f2561000640 1 -- 172.21.15.90:0/4189468356 >> 172.21.15.90:0/4189468356 conn(0x7f255c100840 msgr2=0x7f255c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.785+0000 7f2561000640 1 -- 172.21.15.90:0/4189468356 shutdown_connections 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.785+0000 7f2561000640 1 -- 172.21.15.90:0/4189468356 wait complete. 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.786+0000 7f2561000640 1 Processor -- start 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.786+0000 7f2561000640 1 -- start start 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f2561000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c104e10 0x7f255c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f2561000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f255c10a820 0x7f255c19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f2561000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f255c10b1a0 0x7f255c1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f2561000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f255c114d60 con 0x7f255c104e10 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f2561000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f255c114be0 con 0x7f255c10b1a0 2024-09-10T22:04:01.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f2561000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f255c114ee0 con 0x7f255c10a820 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f255b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c104e10 0x7f255c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f255b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c104e10 0x7f255c19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48988/0 (socket says 172.21.15.90:48988) 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f255b400640 1 -- 172.21.15.90:0/2969543948 learned_addr learned my addr 172.21.15.90:0/2969543948 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f2553e00640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f255c10a820 0x7f255c19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f255be00640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f255c10b1a0 0x7f255c1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f255b400640 1 -- 172.21.15.90:0/2969543948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f255c10a820 msgr2=0x7f255c19b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f255b400640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f255c10a820 0x7f255c19b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f255b400640 1 -- 172.21.15.90:0/2969543948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f255c10b1a0 msgr2=0x7f255c1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.787+0000 7f255b400640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f255c10b1a0 0x7f255c1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.788+0000 7f255b400640 1 -- 172.21.15.90:0/2969543948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f255c1a1240 con 0x7f255c104e10 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.788+0000 7f2553e00640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f255c10a820 0x7f255c19b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:01.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.788+0000 7f255b400640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c104e10 0x7f255c19b190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f254c00d770 tx=0x7f254c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.788+0000 7f2558c00640 1 -- 172.21.15.90:0/2969543948 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c017070 con 0x7f255c104e10 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.788+0000 7f2558c00640 1 -- 172.21.15.90:0/2969543948 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f254c0054b0 con 0x7f255c104e10 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.788+0000 7f2558c00640 1 -- 172.21.15.90:0/2969543948 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c004c60 con 0x7f255c104e10 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.788+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f255c1a1530 con 0x7f255c104e10 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.788+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f255c1a7bf0 con 0x7f255c104e10 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.789+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f255c1051f0 con 0x7f255c104e10 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.791+0000 7f2558c00640 1 -- 172.21.15.90:0/2969543948 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 153324+0+0 (secure 0 0 0) 0x7f254c004e00 con 0x7f255c104e10 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.794+0000 7f2558c00640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f25400b54e0 0x7f25400b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.794+0000 7f2558c00640 1 -- 172.21.15.90:0/2969543948 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f254c0e5d70 con 0x7f255c104e10 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.794+0000 7f2553e00640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f25400b54e0 0x7f25400b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.795+0000 7f2553e00640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f25400b54e0 0x7f25400b79a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2544006010 tx=0x7f2544005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.798+0000 7f2558c00640 1 -- 172.21.15.90:0/2969543948 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f254c0ac5e0 con 0x7f255c104e10 2024-09-10T22:04:01.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.987+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f255c02e5c0 con 0x7f255c104e10 2024-09-10T22:04:01.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.990+0000 7f2558c00640 1 -- 172.21.15.90:0/2969543948 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25227 (secure 0 0 0) 0x7f254c0abee0 con 0x7f255c104e10 2024-09-10T22:04:01.992 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:04:01.992 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:02:35.713572+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:04:01.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f25400b54e0 msgr2=0x7f25400b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:01.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f25400b54e0 0x7f25400b79a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2544006010 tx=0x7f2544005a20 comp rx=0 tx=0).stop 2024-09-10T22:04:01.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c104e10 msgr2=0x7f255c19b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:01.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c104e10 0x7f255c19b190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f254c00d770 tx=0x7f254c00dc30 comp rx=0 tx=0).stop 2024-09-10T22:04:01.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 shutdown_connections 2024-09-10T22:04:01.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f25400b54e0 0x7f25400b79a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:01.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f255c10b1a0 0x7f255c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:01.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f255c10a820 0x7f255c19b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:01.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 --2- 172.21.15.90:0/2969543948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f255c104e10 0x7f255c19b190 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:01.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 >> 172.21.15.90:0/2969543948 conn(0x7f255c100840 msgr2=0x7f255c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:01.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 shutdown_connections 2024-09-10T22:04:01.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:01.992+0000 7f2561000640 1 -- 172.21.15.90:0/2969543948 wait complete. 2024-09-10T22:04:02.793 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-10T22:04:02.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:02 smithi171 ceph-mon[34138]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 84 KiB/s wr, 35 op/s 2024-09-10T22:04:02.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:02 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2969543948' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:04:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:02 smithi090 ceph-mon[30350]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 84 KiB/s wr, 35 op/s 2024-09-10T22:04:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:02 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2969543948' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:04:03.165 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:02 smithi154 ceph-mon[34495]: pgmap v422: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 84 KiB/s wr, 35 op/s 2024-09-10T22:04:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:02 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2969543948' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:04:03.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:03 smithi171 ceph-mon[34138]: mgrmap e24: x(active, since 11m), standbys: z, y 2024-09-10T22:04:04.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:03 smithi090 ceph-mon[30350]: mgrmap e24: x(active, since 11m), standbys: z, y 2024-09-10T22:04:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:03 smithi154 ceph-mon[34495]: mgrmap e24: x(active, since 11m), standbys: z, y 2024-09-10T22:04:04.571 INFO:teuthology.orchestra.run.smithi090.stdout:{"run_dir":"/var/run/ceph"} 2024-09-10T22:04:04.572 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-10T22:04:04.572 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 6190 2024-09-10T22:04:04.572 DEBUG:tasks.quiescer:Yielding 2024-09-10T22:04:04.572 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T22:04:04.591 INFO:tasks.workunit:Pulling workunits from ref 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:04:04.591 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T22:04:04.591 INFO:tasks.workunit:timeout=3h 2024-09-10T22:04:04.591 INFO:tasks.workunit:cleanup=True 2024-09-10T22:04:04.592 DEBUG:teuthology.orchestra.run.smithi090:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T22:04:04.592 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:04:04.629 INFO:teuthology.orchestra.run.smithi090.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T22:04:04.629 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-10T22:04:04.629 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 60h/96d Inode: 1 Links: 2 2024-09-10T22:04:04.629 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T22:04:04.629 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-10T22:04:04.629 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-09-10 22:01:39.734846395 +0000 2024-09-10T22:04:04.629 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-09-10 22:01:39.734846395 +0000 2024-09-10T22:04:04.629 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-09-10 22:03:46.655362053 +0000 2024-09-10T22:04:04.629 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: - 2024-09-10T22:04:04.630 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T22:04:04.630 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T22:04:04.738 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T22:04:04.776 INFO:tasks.workunit.client.0.smithi090.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T22:04:04.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:04 smithi090 ceph-mon[30350]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 105 KiB/s wr, 48 op/s 2024-09-10T22:04:04.975 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:04 smithi171 ceph-mon[34138]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 105 KiB/s wr, 48 op/s 2024-09-10T22:04:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:04 smithi154 ceph-mon[34495]: pgmap v423: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 105 KiB/s wr, 48 op/s 2024-09-10T22:04:05.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.659+0000 7f37d6400640 1 -- 172.21.15.90:0/4280541277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0102da0 msgr2=0x7f37d0103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.659+0000 7f37d6400640 1 --2- 172.21.15.90:0/4280541277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0102da0 0x7f37d0103180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009ca0 tx=0x7f37c002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.660+0000 7f37d6400640 1 -- 172.21.15.90:0/4280541277 shutdown_connections 2024-09-10T22:04:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.660+0000 7f37d6400640 1 --2- 172.21.15.90:0/4280541277 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37d0109130 0x7f37d010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.660+0000 7f37d6400640 1 --2- 172.21.15.90:0/4280541277 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37d01087b0 0x7f37d0108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.660+0000 7f37d6400640 1 --2- 172.21.15.90:0/4280541277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0102da0 0x7f37d0103180 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.660+0000 7f37d6400640 1 -- 172.21.15.90:0/4280541277 >> 172.21.15.90:0/4280541277 conn(0x7f37d00fe810 msgr2=0x7f37d0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.660+0000 7f37d6400640 1 -- 172.21.15.90:0/4280541277 shutdown_connections 2024-09-10T22:04:05.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.660+0000 7f37d6400640 1 -- 172.21.15.90:0/4280541277 wait complete. 2024-09-10T22:04:05.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37d6400640 1 Processor -- start 2024-09-10T22:04:05.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37d6400640 1 -- start start 2024-09-10T22:04:05.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37d6400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37d0102da0 0x7f37d0199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37d6400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37d01087b0 0x7f37d0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37d6400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0109130 0x7f37d019ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37d6400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0112cb0 con 0x7f37d0109130 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37d6400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0112b30 con 0x7f37d0102da0 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37d6400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f37d0112e30 con 0x7f37d01087b0 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0109130 0x7f37d019ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0109130 0x7f37d019ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49020/0 (socket says 172.21.15.90:49020) 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.661+0000 7f37cfe00640 1 -- 172.21.15.90:0/950571066 learned_addr learned my addr 172.21.15.90:0/950571066 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37cf400640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37d0102da0 0x7f37d0199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37cea00640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37d01087b0 0x7f37d0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37cfe00640 1 -- 172.21.15.90:0/950571066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37d01087b0 msgr2=0x7f37d0199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:05.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37cfe00640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37d01087b0 0x7f37d0199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37cfe00640 1 -- 172.21.15.90:0/950571066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37d0102da0 msgr2=0x7f37d0199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:05.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37cfe00640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37d0102da0 0x7f37d0199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37cfe00640 1 -- 172.21.15.90:0/950571066 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d019f1d0 con 0x7f37d0109130 2024-09-10T22:04:05.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37cea00640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37d01087b0 0x7f37d0199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:05.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37cfe00640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0109130 0x7f37d019ea00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f37c800d750 tx=0x7f37c800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:05.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37c7e00640 1 -- 172.21.15.90:0/950571066 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8017070 con 0x7f37d0109130 2024-09-10T22:04:05.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37c7e00640 1 -- 172.21.15.90:0/950571066 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37c80054b0 con 0x7f37d0109130 2024-09-10T22:04:05.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.662+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d019f4c0 con 0x7f37d0109130 2024-09-10T22:04:05.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.663+0000 7f37c7e00640 1 -- 172.21.15.90:0/950571066 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8004b90 con 0x7f37d0109130 2024-09-10T22:04:05.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.663+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d01a5b80 con 0x7f37d0109130 2024-09-10T22:04:05.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.664+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d0103180 con 0x7f37d0109130 2024-09-10T22:04:05.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.666+0000 7f37c7e00640 1 -- 172.21.15.90:0/950571066 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f37c8004d30 con 0x7f37d0109130 2024-09-10T22:04:05.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.667+0000 7f37c7e00640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37b40b54d0 0x7f37b40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:05.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.667+0000 7f37c7e00640 1 -- 172.21.15.90:0/950571066 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f37c80e5e70 con 0x7f37d0109130 2024-09-10T22:04:05.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.667+0000 7f37cf400640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37b40b54d0 0x7f37b40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:05.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.668+0000 7f37cf400640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37b40b54d0 0x7f37b40b7990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f37c0005fa0 tx=0x7f37c0009360 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:05.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.671+0000 7f37c7e00640 1 -- 172.21.15.90:0/950571066 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f37c80ac620 con 0x7f37d0109130 2024-09-10T22:04:05.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.897+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37d002e5c0 con 0x7f37d0109130 2024-09-10T22:04:05.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.897+0000 7f37c7e00640 1 -- 172.21.15.90:0/950571066 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11986 (secure 0 0 0) 0x7f37c80abf20 con 0x7f37d0109130 2024-09-10T22:04:05.899 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:04:05.900 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"btime":"2024-09-10T22:03:47:815790+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:03:47.057424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:04:05.900 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 30 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37b40b54d0 msgr2=0x7f37b40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37b40b54d0 0x7f37b40b7990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f37c0005fa0 tx=0x7f37c0009360 comp rx=0 tx=0).stop 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0109130 msgr2=0x7f37d019ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0109130 0x7f37d019ea00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f37c800d750 tx=0x7f37c800dc10 comp rx=0 tx=0).stop 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 shutdown_connections 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f37b40b54d0 0x7f37b40b7990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37d0109130 0x7f37d019ea00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f37d01087b0 0x7f37d0199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 --2- 172.21.15.90:0/950571066 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f37d0102da0 0x7f37d0199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 >> 172.21.15.90:0/950571066 conn(0x7f37d00fe810 msgr2=0x7f37d00ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 shutdown_connections 2024-09-10T22:04:05.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:05.900+0000 7f37d6400640 1 -- 172.21.15.90:0/950571066 wait complete. 2024-09-10T22:04:06.637 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T22:00:35.595646+0000', 'modified': '2024-09-10T22:03:47.057424+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 34703, 'mds_1': 34670, 'mds_2': 34646, 'mds_3': 34640, 'mds_4': 34607}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14841': {'gid': 14841, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6839/3596805819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6838', 'nonce': 3596805819}, {'type': 'v1', 'addr': '172.21.15.90:6839', 'nonce': 3596805819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14853': {'gid': 14853, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.90:6841/3169114114', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6840', 'nonce': 3169114114}, {'type': 'v1', 'addr': '172.21.15.90:6841', 'nonce': 3169114114}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24598': {'gid': 24598, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6833/3906453887', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6832', 'nonce': 3906453887}, {'type': 'v1', 'addr': '172.21.15.171:6833', 'nonce': 3906453887}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24655': {'gid': 24655, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6837/2889350064', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6836', 'nonce': 2889350064}, {'type': 'v1', 'addr': '172.21.15.171:6837', 'nonce': 2889350064}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24676': {'gid': 24676, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.171:6839/3372214298', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6838', 'nonce': 3372214298}, {'type': 'v1', 'addr': '172.21.15.171:6839', 'nonce': 3372214298}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34607': {'gid': 34607, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 29, 'addr': '172.21.15.154:6833/2781539957', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6832', 'nonce': 2781539957}, {'type': 'v1', 'addr': '172.21.15.154:6833', 'nonce': 2781539957}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34640': {'gid': 34640, 'name': 'e', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 25, 'addr': '172.21.15.154:6835/622088003', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6834', 'nonce': 622088003}, {'type': 'v1', 'addr': '172.21.15.154:6835', 'nonce': 622088003}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34646': {'gid': 34646, 'name': 'f', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.171:6835/2969925217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.171:6834', 'nonce': 2969925217}, {'type': 'v1', 'addr': '172.21.15.171:6835', 'nonce': 2969925217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34670': {'gid': 34670, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.154:6837/3907253955', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6836', 'nonce': 3907253955}, {'type': 'v1', 'addr': '172.21.15.154:6837', 'nonce': 3907253955}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_34703': {'gid': 34703, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.154:6839/703390433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6838', 'nonce': 703390433}, {'type': 'v1', 'addr': '172.21.15.154:6839', 'nonce': 703390433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 34703, 'qdb_cluster': [34703, 34670, 34640, 34646, 34607]} 2024-09-10T22:04:06.637 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T22:04:06.637 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T22:04:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:06 smithi090 ceph-mon[30350]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-10T22:04:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:06 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/950571066' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:06.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:06 smithi171 ceph-mon[34138]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-10T22:04:06.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:06 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/950571066' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:07.048 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:06 smithi154 ceph-mon[34495]: pgmap v424: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-10T22:04:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:06 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/950571066' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:07.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.593+0000 7fd5a9200640 1 -- 172.21.15.90:0/857278163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 msgr2=0x7fd5a41051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:07.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.593+0000 7fd5a9200640 1 --2- 172.21.15.90:0/857278163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 0x7fd5a41051d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd594009ca0 tx=0x7fd59402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.593+0000 7fd5a9200640 1 -- 172.21.15.90:0/857278163 shutdown_connections 2024-09-10T22:04:07.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.593+0000 7fd5a9200640 1 --2- 172.21.15.90:0/857278163 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5a410b180 0x7fd5a4111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.593+0000 7fd5a9200640 1 --2- 172.21.15.90:0/857278163 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5a410a800 0x7fd5a410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.593+0000 7fd5a9200640 1 --2- 172.21.15.90:0/857278163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 0x7fd5a41051d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.593+0000 7fd5a9200640 1 -- 172.21.15.90:0/857278163 >> 172.21.15.90:0/857278163 conn(0x7fd5a4100840 msgr2=0x7fd5a4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.593+0000 7fd5a9200640 1 -- 172.21.15.90:0/857278163 shutdown_connections 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.593+0000 7fd5a9200640 1 -- 172.21.15.90:0/857278163 wait complete. 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.594+0000 7fd5a9200640 1 Processor -- start 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.594+0000 7fd5a9200640 1 -- start start 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 0x7fd5a419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a9200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5a410a800 0x7fd5a419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a9200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5a410b180 0x7fd5a41a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a9200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4114d40 con 0x7fd5a4104df0 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a9200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4114bc0 con 0x7fd5a410b180 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a9200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a4114ec0 con 0x7fd5a410a800 2024-09-10T22:04:07.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 0x7fd5a419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 0x7fd5a419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49046/0 (socket says 172.21.15.90:49046) 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a3400640 1 -- 172.21.15.90:0/3739868609 learned_addr learned my addr 172.21.15.90:0/3739868609 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a2a00640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5a410a800 0x7fd5a419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.595+0000 7fd5a3400640 1 -- 172.21.15.90:0/3739868609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5a410a800 msgr2=0x7fd5a419b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5a410b180 0x7fd5a41a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a3400640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5a410a800 0x7fd5a419b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a3400640 1 -- 172.21.15.90:0/3739868609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5a410b180 msgr2=0x7fd5a41a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a3400640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5a410b180 0x7fd5a41a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a3400640 1 -- 172.21.15.90:0/3739868609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a41a11c0 con 0x7fd5a4104df0 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a2a00640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5a410a800 0x7fd5a419b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5a410b180 0x7fd5a41a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a3400640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 0x7fd5a419b110 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd594031040 tx=0x7fd594032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:07.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd59be00640 1 -- 172.21.15.90:0/3739868609 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594004040 con 0x7fd5a4104df0 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd59be00640 1 -- 172.21.15.90:0/3739868609 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd594004670 con 0x7fd5a4104df0 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a9200640 1 -- 172.21.15.90:0/3739868609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a41a1450 con 0x7fd5a4104df0 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd59be00640 1 -- 172.21.15.90:0/3739868609 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594030d50 con 0x7fd5a4104df0 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.596+0000 7fd5a9200640 1 -- 172.21.15.90:0/3739868609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a41a7b70 con 0x7fd5a4104df0 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.599+0000 7fd5ad272740 1 Processor -- start 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.599+0000 7fd5ad272740 1 -- start start 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.599+0000 7fd59be00640 1 -- 172.21.15.90:0/3739868609 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd5940041e0 con 0x7fd5a4104df0 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 0x555b224ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ffd20 0x555b2250b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 0x555b2250e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5ad272740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555b22476d50 con 0x555b224ffd20 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5ad272740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555b22466180 con 0x555b2250bcb0 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5ad272740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555b22469f90 con 0x555b224ff400 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ffd20 0x555b2250b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5a3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 0x555b224ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ffd20 0x555b2250b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49050/0 (socket says 172.21.15.90:49050) 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5a3400640 1 -- 172.21.15.90:0/2017535425 learned_addr learned my addr 172.21.15.90:0/2017535425 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.600+0000 7fd5a2a00640 1 --2- 172.21.15.90:0/2017535425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 0x555b2250e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.601+0000 7fd5a3400640 1 -- 172.21.15.90:0/2017535425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 msgr2=0x555b224ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:07.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.601+0000 7fd5a3400640 1 --2- 172.21.15.90:0/2017535425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 0x555b224ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.601+0000 7fd5a3400640 1 -- 172.21.15.90:0/2017535425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 msgr2=0x555b2250e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.601+0000 7fd5a3400640 1 --2- 172.21.15.90:0/2017535425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 0x555b2250e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.601+0000 7fd5a3400640 1 -- 172.21.15.90:0/2017535425 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b22430a70 con 0x555b224ffd20 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.601+0000 7fd5a3400640 1 --2- 172.21.15.90:0/2017535425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ffd20 0x555b2250b770 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd59410da60 tx=0x7fd59410db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275e168286b7f4e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.601+0000 7fd598c00640 1 -- 172.21.15.90:0/2017535425 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594107ce0 con 0x555b224ffd20 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.602+0000 7fd598c00640 1 -- 172.21.15.90:0/2017535425 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd594117020 con 0x555b224ffd20 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.602+0000 7fd59be00640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd5880b5580 0x7fd5880b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.602+0000 7fd598c00640 1 -- 172.21.15.90:0/2017535425 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594112800 con 0x555b224ffd20 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.602+0000 7fd59be00640 1 -- 172.21.15.90:0/3739868609 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd5941076e0 con 0x7fd5a4104df0 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.602+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd5880b5580 0x7fd5880b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.602+0000 7fd5ad272740 1 -- 172.21.15.90:0/2017535425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ffd20 msgr2=0x555b2250b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.602+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2017535425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ffd20 0x555b2250b770 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd59410da60 tx=0x7fd59410db50 comp rx=0 tx=0).stop 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5ad272740 1 -- 172.21.15.90:0/2017535425 shutdown_connections 2024-09-10T22:04:07.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2017535425 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 0x555b2250e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2017535425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ffd20 0x555b2250b770 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2017535425 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 0x555b224ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5ad272740 1 -- 172.21.15.90:0/2017535425 >> 172.21.15.90:0/2017535425 conn(0x555b224fd230 msgr2=0x555b224fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd5880b5580 0x7fd5880b7a40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd59c005fd0 tx=0x7fd59c009420 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5ad272740 1 -- 172.21.15.90:0/2017535425 shutdown_connections 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5ad272740 1 -- 172.21.15.90:0/2017535425 wait complete. 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5ad272740 1 -- start start 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.603+0000 7fd5ad272740 1 Processor -- start 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 0x555b22508c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 0x555b225091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b2250bcb0 0x555b22509700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5ad272740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555b222c75f0 con 0x555b2250bcb0 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5ad272740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555b22401460 con 0x555b224ffd20 2024-09-10T22:04:07.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5ad272740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555b22476d50 con 0x555b224ff400 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5ad272740 10 client.0 _ll_register_callbacks cb 0x555b22411210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b2250bcb0 0x555b22509700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 0x555b22508c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b2250bcb0 0x555b22509700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49052/0 (socket says 172.21.15.90:49052) 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a3e00640 1 -- 172.21.15.90:0/2820475740 learned_addr learned my addr 172.21.15.90:0/2820475740 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a2a00640 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 0x555b225091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a3e00640 1 -- 172.21.15.90:0/2820475740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 msgr2=0x555b22508c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 0x555b22508c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a3e00640 1 -- 172.21.15.90:0/2820475740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 msgr2=0x555b225091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:07.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 0x555b225091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.604+0000 7fd5a3e00640 1 -- 172.21.15.90:0/2820475740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b221c4540 con 0x555b2250bcb0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.605+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b2250bcb0 0x555b22509700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009b00 tx=0x7fd59c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b939d7b8241e9005 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.605+0000 7fd58d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.605+0000 7fd58d600640 1 -- 172.21.15.90:0/2820475740 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c01f8f0 con 0x555b2250bcb0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.605+0000 7fd58d600640 1 -- 172.21.15.90:0/2820475740 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd59c01fa90 con 0x555b2250bcb0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.605+0000 7fd5ad272740 1 -- 172.21.15.90:0/2820475740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b22403a70 con 0x555b2250bcb0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.605+0000 7fd58d600640 1 -- 172.21.15.90:0/2820475740 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd59c01fcd0 con 0x555b2250bcb0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.605+0000 7fd5ad272740 10 client.15645 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.605+0000 7fd5ad272740 1 -- 172.21.15.90:0/2820475740 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b2241e6f0 con 0x555b2250bcb0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.605+0000 7fd58d600640 1 -- 172.21.15.90:0/2820475740 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x555b22403a70 con 0x555b2250bcb0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.606+0000 7fd58d600640 1 -- 172.21.15.90:0/2820475740 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd59c00f040 con 0x555b2250bcb0 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.606+0000 7fd5ad272740 10 client.15645 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.606+0000 7fd5ad272740 10 client.15645 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:04:07.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.606+0000 7fd5ad272740 20 client.15645 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.606+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x555b2251c660 0x555b2251dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.606+0000 7fd5ad272740 4 client.15645 mds_command: new command op to 34607 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.606+0000 7fd5ad272740 1 -- 172.21.15.90:0/2820475740 --> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b22478ca0 con 0x555b2251c660 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.606+0000 7fd5a3400640 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x555b2251c660 0x555b2251dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.607+0000 7fd5a3400640 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x555b2251c660 0x555b2251dad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=52614259151213d3 server_cookie=c3aceb8e8c895be5 in_seq=0 out_seq=0 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.607+0000 7fd58d600640 10 client.15645 ms_handle_connect on v2:172.21.15.154:6832/2781539957 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd58d600640 1 -- 172.21.15.90:0/2820475740 <== mds.4 v2:172.21.15.154:6832/2781539957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b22478ca0 con 0x555b2251c660 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd58d600640 10 client.15645 handle_command_reply: tid=0 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 client.15645 shutdown 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 -- 172.21.15.90:0/2820475740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b2250bcb0 msgr2=0x555b22509700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b2250bcb0 0x555b22509700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009b00 tx=0x7fd59c0046c0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 -- 172.21.15.90:0/2820475740 shutdown_connections 2024-09-10T22:04:07.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x555b2251c660 0x555b2251dad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b2250bcb0 0x555b22509700 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 0x555b225091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2820475740 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ff400 0x555b22508c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd58d600640 0 client.15645 ms_handle_reset on v2:172.21.15.154:6832/2781539957 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 -- 172.21.15.90:0/2820475740 >> 172.21.15.90:0/2820475740 conn(0x555b224fd230 msgr2=0x555b22502dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 -- 172.21.15.90:0/2820475740 shutdown_connections 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 1 -- 172.21.15.90:0/2820475740 wait complete. 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.608+0000 7fd5ad272740 20 client.15645 trim_cache size 0 max 16384 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.627+0000 7fd5ad272740 1 Processor -- start 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.627+0000 7fd5ad272740 1 -- start start 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.627+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b2251cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.627+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ffd20 0x555b225035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.627+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 0x555b2251b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5ad272740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555b2251be10 con 0x555b224ff400 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5ad272740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555b2251c4e0 con 0x555b2250bcb0 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5ad272740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555b2251cbb0 con 0x555b224ffd20 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b2251cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b2251cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49062/0 (socket says 172.21.15.90:49062) 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a3400640 1 -- 172.21.15.90:0/2128672218 learned_addr learned my addr 172.21.15.90:0/2128672218 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/2128672218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 0x555b2251b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a2a00640 1 --2- 172.21.15.90:0/2128672218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ffd20 0x555b225035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a3400640 1 -- 172.21.15.90:0/2128672218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ffd20 msgr2=0x555b225035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a3400640 1 --2- 172.21.15.90:0/2128672218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ffd20 0x555b225035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a3400640 1 -- 172.21.15.90:0/2128672218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 msgr2=0x555b2251b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a3400640 1 --2- 172.21.15.90:0/2128672218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 0x555b2251b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.628+0000 7fd5a3400640 1 -- 172.21.15.90:0/2128672218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b224fd060 con 0x555b224ff400 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd5a3400640 1 --2- 172.21.15.90:0/2128672218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b2251cd30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd5940caa00 tx=0x7fd594112420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a904f342671f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd58fe00640 1 -- 172.21.15.90:0/2128672218 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594116070 con 0x555b224ff400 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd58fe00640 1 -- 172.21.15.90:0/2128672218 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd594036070 con 0x555b224ff400 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd58fe00640 1 -- 172.21.15.90:0/2128672218 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5940f0040 con 0x555b224ff400 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd5ad272740 1 -- 172.21.15.90:0/2128672218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 msgr2=0x555b2251cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2128672218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b2251cd30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd5940caa00 tx=0x7fd594112420 comp rx=0 tx=0).stop 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd5ad272740 1 -- 172.21.15.90:0/2128672218 shutdown_connections 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2128672218 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b2250bcb0 0x555b2251b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2128672218 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b224ffd20 0x555b225035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd5ad272740 1 --2- 172.21.15.90:0/2128672218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b2251cd30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.629+0000 7fd5ad272740 1 -- 172.21.15.90:0/2128672218 >> 172.21.15.90:0/2128672218 conn(0x555b224fd230 msgr2=0x555b22507500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.630+0000 7fd5ad272740 1 -- 172.21.15.90:0/2128672218 shutdown_connections 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.630+0000 7fd5ad272740 1 -- 172.21.15.90:0/2128672218 wait complete. 2024-09-10T22:04:07.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.630+0000 7fd5ad272740 1 -- start start 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.630+0000 7fd5ad272740 1 Processor -- start 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b22525f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 0x555b225264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5ad272740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b2250bcb0 0x555b22520ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5ad272740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555b2251d950 con 0x555b224ff400 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5ad272740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x555b2251d280 con 0x555b224ffd20 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5ad272740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x555b2251e020 con 0x555b2250bcb0 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5ad272740 10 client.0 _ll_register_callbacks cb 0x7fd59c009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b22525f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b22525f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49070/0 (socket says 172.21.15.90:49070) 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3400640 1 -- 172.21.15.90:0/3427113490 learned_addr learned my addr 172.21.15.90:0/3427113490 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b2250bcb0 0x555b22520ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a2a00640 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 0x555b225264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3400640 1 -- 172.21.15.90:0/3427113490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b2250bcb0 msgr2=0x555b22520ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3400640 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b2250bcb0 0x555b22520ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3400640 1 -- 172.21.15.90:0/3427113490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 msgr2=0x555b225264d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3400640 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 0x555b225264d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3400640 1 -- 172.21.15.90:0/3427113490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b224fdcb0 con 0x555b224ff400 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a3e00640 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b2250bcb0 0x555b22520ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.631+0000 7fd5a2a00640 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 0x555b225264d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd5a3400640 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b22525f30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd594004af0 tx=0x7fd59410dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab67773b7e264c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd58d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd58d600640 1 -- 172.21.15.90:0/3427113490 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594116070 con 0x555b224ff400 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd58d600640 1 -- 172.21.15.90:0/3427113490 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fd5940c9a00 con 0x555b224ff400 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd5ad272740 1 -- 172.21.15.90:0/3427113490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd594036070 con 0x555b224ff400 2024-09-10T22:04:07.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd58d600640 1 -- 172.21.15.90:0/3427113490 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd5940f0040 con 0x555b224ff400 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd5ad272740 10 client.15651 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd5ad272740 1 -- 172.21.15.90:0/3427113490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555b22589860 con 0x555b224ff400 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd58d600640 1 -- 172.21.15.90:0/3427113490 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd594036070 con 0x555b224ff400 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.632+0000 7fd58d600640 1 -- 172.21.15.90:0/3427113490 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd594107ce0 con 0x555b224ff400 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.633+0000 7fd5ad272740 10 client.15651 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.633+0000 7fd5ad272740 10 client.15651 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.633+0000 7fd5ad272740 20 client.15651 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.633+0000 7fd5ad272740 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x555b2258e200 0x555b2258cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.633+0000 7fd5ad272740 4 client.15651 mds_command: new command op to 34607 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.633+0000 7fd5ad272740 1 -- 172.21.15.90:0/3427113490 --> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] -- command(tid 0: {"prefix": "status"}) -- 0x555b22474750 con 0x555b2258e200 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.633+0000 7fd5a2a00640 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x555b2258e200 0x555b2258cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5a2a00640 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x555b2258e200 0x555b2258cf10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa71f9463784f4e9 server_cookie=9625c4273a4d4f7a in_seq=0 out_seq=0 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd58d600640 10 client.15651 ms_handle_connect on v2:172.21.15.154:6832/2781539957 2024-09-10T22:04:07.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd58d600640 1 -- 172.21.15.90:0/3427113490 <== mds.4 v2:172.21.15.154:6832/2781539957 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555b22474750 con 0x555b2258e200 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd58d600640 10 client.15651 handle_command_reply: tid=0 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5ad272740 1 client.15651 shutdown 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5ad272740 1 -- 172.21.15.90:0/3427113490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 msgr2=0x555b22525f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5ad272740 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b22525f30 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd594004af0 tx=0x7fd59410dc80 comp rx=0 tx=0).stop 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5ad272740 1 -- 172.21.15.90:0/3427113490 shutdown_connections 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5ad272740 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.154:6832/2781539957,v1:172.21.15.154:6833/2781539957] conn(0x555b2258e200 0x555b2258cf10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5ad272740 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x555b2250bcb0 0x555b22520ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5ad272740 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x555b224ffd20 0x555b225264d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5ad272740 1 --2- 172.21.15.90:0/3427113490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555b224ff400 0x555b22525f30 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd5ad272740 1 -- 172.21.15.90:0/3427113490 >> 172.21.15.90:0/3427113490 conn(0x555b224fd230 msgr2=0x555b2250a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:07.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.634+0000 7fd58d600640 0 client.15651 ms_handle_reset on v2:172.21.15.154:6832/2781539957 2024-09-10T22:04:07.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.635+0000 7fd5ad272740 1 -- 172.21.15.90:0/3427113490 shutdown_connections 2024-09-10T22:04:07.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.635+0000 7fd5ad272740 1 -- 172.21.15.90:0/3427113490 wait complete. 2024-09-10T22:04:07.656 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:04:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:04:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 4, 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34607, 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 155.90167459899999, 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 30, 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 264.13960082599999 2024-09-10T22:04:07.657 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:04:07.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.635+0000 7fd5ad272740 20 client.15651 trim_cache size 0 max 16384 2024-09-10T22:04:07.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.635+0000 7fd5a9200640 1 -- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd5880b5580 msgr2=0x7fd5880b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:07.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.635+0000 7fd5a9200640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd5880b5580 0x7fd5880b7a40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd59c005fd0 tx=0x7fd59c009420 comp rx=0 tx=0).stop 2024-09-10T22:04:07.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.635+0000 7fd5a9200640 1 -- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 msgr2=0x7fd5a419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:07.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.635+0000 7fd5a9200640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 0x7fd5a419b110 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd594031040 tx=0x7fd594032bc0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.636+0000 7fd5a9200640 1 -- 172.21.15.90:0/3739868609 shutdown_connections 2024-09-10T22:04:07.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.636+0000 7fd5a9200640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd5880b5580 0x7fd5880b7a40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.636+0000 7fd5a9200640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd5a410b180 0x7fd5a41a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.636+0000 7fd5a9200640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd5a410a800 0x7fd5a419b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.636+0000 7fd5a9200640 1 --2- 172.21.15.90:0/3739868609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5a4104df0 0x7fd5a419b110 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:07.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.636+0000 7fd5a9200640 1 -- 172.21.15.90:0/3739868609 >> 172.21.15.90:0/3739868609 conn(0x7fd5a4100840 msgr2=0x7fd5a41020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:07.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.636+0000 7fd5a9200640 1 -- 172.21.15.90:0/3739868609 shutdown_connections 2024-09-10T22:04:07.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:07.636+0000 7fd5a9200640 1 -- 172.21.15.90:0/3739868609 wait complete. 2024-09-10T22:04:08.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.e status 2024-09-10T22:04:08.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:08 smithi090 ceph-mon[30350]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-10T22:04:08.917 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:08.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:08 smithi171 ceph-mon[34138]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-10T22:04:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:08 smithi154 ceph-mon[34495]: pgmap v425: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-10T22:04:09.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.506+0000 7f03a3800640 1 -- 172.21.15.90:0/3183873738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c1045e0 msgr2=0x7f039c10cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:09.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.506+0000 7f03a3800640 1 --2- 172.21.15.90:0/3183873738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c1045e0 0x7f039c10cad0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0388009900 tx=0x7f038802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.506+0000 7f03a3800640 1 -- 172.21.15.90:0/3183873738 shutdown_connections 2024-09-10T22:04:09.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.506+0000 7f03a3800640 1 --2- 172.21.15.90:0/3183873738 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f039c10d120 0x7f039c10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.506+0000 7f03a3800640 1 --2- 172.21.15.90:0/3183873738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c1045e0 0x7f039c10cad0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.506+0000 7f03a3800640 1 --2- 172.21.15.90:0/3183873738 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f039c103cc0 0x7f039c1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.506+0000 7f03a3800640 1 -- 172.21.15.90:0/3183873738 >> 172.21.15.90:0/3183873738 conn(0x7f039c0fd700 msgr2=0x7f039c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:09.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.507+0000 7f03a3800640 1 -- 172.21.15.90:0/3183873738 shutdown_connections 2024-09-10T22:04:09.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.507+0000 7f03a3800640 1 -- 172.21.15.90:0/3183873738 wait complete. 2024-09-10T22:04:09.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.507+0000 7f03a3800640 1 Processor -- start 2024-09-10T22:04:09.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a3800640 1 -- start start 2024-09-10T22:04:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a3800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c103cc0 0x7f039c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a3800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f039c1045e0 0x7f039c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a3800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f039c10d120 0x7f039c19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a3800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f039c112c90 con 0x7f039c103cc0 2024-09-10T22:04:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a3800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f039c112b10 con 0x7f039c1045e0 2024-09-10T22:04:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a3800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f039c112e10 con 0x7f039c10d120 2024-09-10T22:04:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c103cc0 0x7f039c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c103cc0 0x7f039c199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43748/0 (socket says 172.21.15.90:43748) 2024-09-10T22:04:09.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a0e00640 1 -- 172.21.15.90:0/1159876650 learned_addr learned my addr 172.21.15.90:0/1159876650 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f03a1800640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f039c10d120 0x7f039c19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.508+0000 7f039be00640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f039c1045e0 0x7f039c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f03a0e00640 1 -- 172.21.15.90:0/1159876650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f039c10d120 msgr2=0x7f039c19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f03a0e00640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f039c10d120 0x7f039c19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f03a0e00640 1 -- 172.21.15.90:0/1159876650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f039c1045e0 msgr2=0x7f039c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f03a0e00640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f039c1045e0 0x7f039c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f03a0e00640 1 -- 172.21.15.90:0/1159876650 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c19f2d0 con 0x7f039c103cc0 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f039be00640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f039c1045e0 0x7f039c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f03a0e00640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c103cc0 0x7f039c199220 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f038c00d950 tx=0x7f038c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f0399600640 1 -- 172.21.15.90:0/1159876650 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c017030 con 0x7f039c103cc0 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f03a3800640 1 -- 172.21.15.90:0/1159876650 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c19f5c0 con 0x7f039c103cc0 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f0399600640 1 -- 172.21.15.90:0/1159876650 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f038c004020 con 0x7f039c103cc0 2024-09-10T22:04:09.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f0399600640 1 -- 172.21.15.90:0/1159876650 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c0055d0 con 0x7f039c103cc0 2024-09-10T22:04:09.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.509+0000 7f03a3800640 1 -- 172.21.15.90:0/1159876650 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c1a5c80 con 0x7f039c103cc0 2024-09-10T22:04:09.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.512+0000 7f03a63e6740 1 Processor -- start 2024-09-10T22:04:09.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f03a63e6740 1 -- start start 2024-09-10T22:04:09.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f0399600640 1 -- 172.21.15.90:0/1159876650 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f038c009a30 con 0x7f039c103cc0 2024-09-10T22:04:09.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73a650 0x5565af73aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73af70 0x5565af7469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af7492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f03a63e6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565af6add50 con 0x5565af746f00 2024-09-10T22:04:09.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f03a63e6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5565af69d180 con 0x5565af73af70 2024-09-10T22:04:09.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f03a63e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5565af6a0f90 con 0x5565af73a650 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f039be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af7492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f039be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af7492f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43764/0 (socket says 172.21.15.90:43764) 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f039be00640 1 -- 172.21.15.90:0/2141764067 learned_addr learned my addr 172.21.15.90:0/2141764067 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.513+0000 7f03a1800640 1 --2- 172.21.15.90:0/2141764067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73a650 0x5565af73aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.514+0000 7f039be00640 1 -- 172.21.15.90:0/2141764067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73a650 msgr2=0x5565af73aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.514+0000 7f039be00640 1 --2- 172.21.15.90:0/2141764067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73a650 0x5565af73aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.514+0000 7f039be00640 1 -- 172.21.15.90:0/2141764067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73af70 msgr2=0x5565af7469c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.514+0000 7f039be00640 1 --2- 172.21.15.90:0/2141764067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73af70 0x5565af7469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.514+0000 7f039be00640 1 -- 172.21.15.90:0/2141764067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565af667a70 con 0x5565af746f00 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.514+0000 7f039be00640 1 --2- 172.21.15.90:0/2141764067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af7492f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0388000c00 tx=0x7f03880042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2de69eec48668b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.515+0000 7f0392000640 1 -- 172.21.15.90:0/2141764067 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388035030 con 0x5565af746f00 2024-09-10T22:04:09.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.515+0000 7f0399600640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03800b5450 0x7f03800b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.515+0000 7f0392000640 1 -- 172.21.15.90:0/2141764067 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f03880044c0 con 0x5565af746f00 2024-09-10T22:04:09.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.515+0000 7f0392000640 1 -- 172.21.15.90:0/2141764067 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03880035f0 con 0x5565af746f00 2024-09-10T22:04:09.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.515+0000 7f0399600640 1 -- 172.21.15.90:0/1159876650 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f038c0e5e10 con 0x7f039c103cc0 2024-09-10T22:04:09.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.515+0000 7f03a1800640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03800b5450 0x7f03800b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.515+0000 7f03a63e6740 1 -- 172.21.15.90:0/2141764067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 msgr2=0x5565af7492f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:09.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.515+0000 7f03a63e6740 1 --2- 172.21.15.90:0/2141764067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af7492f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0388000c00 tx=0x7f03880042d0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a63e6740 1 -- 172.21.15.90:0/2141764067 shutdown_connections 2024-09-10T22:04:09.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a63e6740 1 --2- 172.21.15.90:0/2141764067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af7492f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a63e6740 1 --2- 172.21.15.90:0/2141764067 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73af70 0x5565af7469c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a63e6740 1 --2- 172.21.15.90:0/2141764067 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73a650 0x5565af73aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a63e6740 1 -- 172.21.15.90:0/2141764067 >> 172.21.15.90:0/2141764067 conn(0x5565af738430 msgr2=0x5565af739620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:09.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a1800640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03800b5450 0x7f03800b7910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0394005fd0 tx=0x7f0394009600 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:09.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a63e6740 1 -- 172.21.15.90:0/2141764067 shutdown_connections 2024-09-10T22:04:09.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a63e6740 1 -- 172.21.15.90:0/2141764067 wait complete. 2024-09-10T22:04:09.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a63e6740 1 -- start start 2024-09-10T22:04:09.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.516+0000 7f03a63e6740 1 Processor -- start 2024-09-10T22:04:09.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73a650 0x5565af743f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af744450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af744990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a63e6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565af4fe5f0 con 0x5565af746f00 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a63e6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5565af638460 con 0x5565af73a650 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a63e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5565af749bc0 con 0x5565af73af70 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a63e6740 10 client.0 _ll_register_callbacks cb 0x5565af648210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a1800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af744990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a1800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af744990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43770/0 (socket says 172.21.15.90:43770) 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f03a1800640 1 -- 172.21.15.90:0/1335826060 learned_addr learned my addr 172.21.15.90:0/1335826060 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.517+0000 7f039be00640 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af744450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f03a0e00640 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73a650 0x5565af743f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f03a1800640 1 -- 172.21.15.90:0/1335826060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 msgr2=0x5565af744450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f03a1800640 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af744450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f03a1800640 1 -- 172.21.15.90:0/1335826060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73a650 msgr2=0x5565af743f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:09.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f03a1800640 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73a650 0x5565af743f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f03a1800640 1 -- 172.21.15.90:0/1335826060 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565af3fb540 con 0x5565af746f00 2024-09-10T22:04:09.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f03a0e00640 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73a650 0x5565af743f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:09.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f03a1800640 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af744990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0394011d40 tx=0x7f0394004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b280dd938aa54b6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:09.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f0386a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:09.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f0386a00640 1 -- 172.21.15.90:0/1335826060 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394004d20 con 0x5565af746f00 2024-09-10T22:04:09.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f0386a00640 1 -- 172.21.15.90:0/1335826060 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f0394005020 con 0x5565af746f00 2024-09-10T22:04:09.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.518+0000 7f03a63e6740 1 -- 172.21.15.90:0/1335826060 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5565af63aa70 con 0x5565af746f00 2024-09-10T22:04:09.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f0386a00640 1 -- 172.21.15.90:0/1335826060 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0394005220 con 0x5565af746f00 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f03a63e6740 10 client.15663 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f03a63e6740 1 -- 172.21.15.90:0/1335826060 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565af6556f0 con 0x5565af746f00 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f0386a00640 1 -- 172.21.15.90:0/1335826060 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5565af63aa70 con 0x5565af746f00 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f0386a00640 1 -- 172.21.15.90:0/1335826060 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f039400f040 con 0x5565af746f00 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f03a63e6740 10 client.15663 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f03a63e6740 10 client.15663 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f03a63e6740 20 client.15663 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f03a63e6740 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x5565af757cd0 0x5565af759140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f03a63e6740 4 client.15663 mds_command: new command op to 34640 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.519+0000 7f03a63e6740 1 -- 172.21.15.90:0/1335826060 --> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565af6afca0 con 0x5565af757cd0 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.520+0000 7f03a0e00640 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x5565af757cd0 0x5565af759140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.520+0000 7f03a0e00640 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x5565af757cd0 0x5565af759140 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38e96a6fe4f4693 server_cookie=bf19c21f6bcb0cd1 in_seq=0 out_seq=0 2024-09-10T22:04:09.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.520+0000 7f0386a00640 10 client.15663 ms_handle_connect on v2:172.21.15.154:6834/622088003 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.521+0000 7f0386a00640 1 -- 172.21.15.90:0/1335826060 <== mds.3 v2:172.21.15.154:6834/622088003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565af6afca0 con 0x5565af757cd0 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.521+0000 7f0386a00640 10 client.15663 handle_command_reply: tid=0 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.521+0000 7f03a63e6740 1 client.15663 shutdown 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 -- 172.21.15.90:0/1335826060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 msgr2=0x5565af744990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af744990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0394011d40 tx=0x7f0394004a40 comp rx=0 tx=0).stop 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 -- 172.21.15.90:0/1335826060 shutdown_connections 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af746f00 0x5565af744990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x5565af757cd0 0x5565af759140 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af744450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 --2- 172.21.15.90:0/1335826060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af73a650 0x5565af743f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 -- 172.21.15.90:0/1335826060 >> 172.21.15.90:0/1335826060 conn(0x5565af738430 msgr2=0x5565af73e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:09.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f0386a00640 0 client.15663 ms_handle_reset on v2:172.21.15.154:6834/622088003 2024-09-10T22:04:09.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 -- 172.21.15.90:0/1335826060 shutdown_connections 2024-09-10T22:04:09.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 1 -- 172.21.15.90:0/1335826060 wait complete. 2024-09-10T22:04:09.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.522+0000 7f03a63e6740 20 client.15663 trim_cache size 0 max 16384 2024-09-10T22:04:09.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a63e6740 1 Processor -- start 2024-09-10T22:04:09.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a63e6740 1 -- start start 2024-09-10T22:04:09.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af72eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af745c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 0x5565af741120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a63e6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565af757480 con 0x5565af73a650 2024-09-10T22:04:09.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a63e6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5565af757b50 con 0x5565af746f00 2024-09-10T22:04:09.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a63e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5565af758220 con 0x5565af73af70 2024-09-10T22:04:09.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af72eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af72eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43780/0 (socket says 172.21.15.90:43780) 2024-09-10T22:04:09.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.541+0000 7f03a0e00640 1 -- 172.21.15.90:0/2556882685 learned_addr learned my addr 172.21.15.90:0/2556882685 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:09.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f03a1800640 1 --2- 172.21.15.90:0/2556882685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 0x5565af741120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f03a0e00640 1 -- 172.21.15.90:0/2556882685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 msgr2=0x5565af745c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:09.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f03a0e00640 1 --2- 172.21.15.90:0/2556882685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af745c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f03a0e00640 1 -- 172.21.15.90:0/2556882685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 msgr2=0x5565af741120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:09.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f03a0e00640 1 --2- 172.21.15.90:0/2556882685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 0x5565af741120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f03a0e00640 1 -- 172.21.15.90:0/2556882685 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565af73ada0 con 0x5565af73a650 2024-09-10T22:04:09.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f03a1800640 1 --2- 172.21.15.90:0/2556882685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 0x5565af741120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:09.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f03a0e00640 1 --2- 172.21.15.90:0/2556882685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af72eea0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f038c0a9130 tx=0x7f038c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e64b791b30ddb91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:09.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f0391600640 1 -- 172.21.15.90:0/2556882685 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c0b09a0 con 0x5565af73a650 2024-09-10T22:04:09.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f0391600640 1 -- 172.21.15.90:0/2556882685 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f038c0b5450 con 0x5565af73a650 2024-09-10T22:04:09.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.542+0000 7f0391600640 1 -- 172.21.15.90:0/2556882685 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c0930c0 con 0x5565af73a650 2024-09-10T22:04:09.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 -- 172.21.15.90:0/2556882685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 msgr2=0x5565af72eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:09.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 --2- 172.21.15.90:0/2556882685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af72eea0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f038c0a9130 tx=0x7f038c000bc0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 -- 172.21.15.90:0/2556882685 shutdown_connections 2024-09-10T22:04:09.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 --2- 172.21.15.90:0/2556882685 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 0x5565af741120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 --2- 172.21.15.90:0/2556882685 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af745c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 --2- 172.21.15.90:0/2556882685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af72eea0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 -- 172.21.15.90:0/2556882685 >> 172.21.15.90:0/2556882685 conn(0x5565af738430 msgr2=0x5565af745540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:09.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 -- 172.21.15.90:0/2556882685 shutdown_connections 2024-09-10T22:04:09.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 -- 172.21.15.90:0/2556882685 wait complete. 2024-09-10T22:04:09.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.543+0000 7f03a63e6740 1 -- start start 2024-09-10T22:04:09.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a63e6740 1 Processor -- start 2024-09-10T22:04:09.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af7619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af75bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a63e6740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 0x5565af75c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a63e6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5565af758fc0 con 0x5565af73a650 2024-09-10T22:04:09.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a63e6740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5565af7588f0 con 0x5565af746f00 2024-09-10T22:04:09.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a63e6740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5565af759690 con 0x5565af73af70 2024-09-10T22:04:09.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a63e6740 10 client.0 _ll_register_callbacks cb 0x7f03940119f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:09.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af7619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a0e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af7619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43792/0 (socket says 172.21.15.90:43792) 2024-09-10T22:04:09.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f03a0e00640 1 -- 172.21.15.90:0/878160215 learned_addr learned my addr 172.21.15.90:0/878160215 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:09.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.544+0000 7f039be00640 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af75bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a1800640 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 0x5565af75c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a0e00640 1 -- 172.21.15.90:0/878160215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 msgr2=0x5565af75bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:09.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a0e00640 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af75bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a0e00640 1 -- 172.21.15.90:0/878160215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 msgr2=0x5565af75c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:09.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a0e00640 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 0x5565af75c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a0e00640 1 -- 172.21.15.90:0/878160215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565af75cab0 con 0x5565af73a650 2024-09-10T22:04:09.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f039be00640 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af75bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:09.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a0e00640 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af7619f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f038c0abe50 tx=0x7f038c0a8620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2411a9437a68a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:09.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f0386a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:09.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f0386a00640 1 -- 172.21.15.90:0/878160215 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c0b09a0 con 0x5565af73a650 2024-09-10T22:04:09.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f0386a00640 1 -- 172.21.15.90:0/878160215 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f038c0041d0 con 0x5565af73a650 2024-09-10T22:04:09.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a63e6740 1 -- 172.21.15.90:0/878160215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f038c0b5450 con 0x5565af73a650 2024-09-10T22:04:09.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f0386a00640 1 -- 172.21.15.90:0/878160215 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f038c0e3410 con 0x5565af73a650 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a63e6740 10 client.15669 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.545+0000 7f03a63e6740 1 -- 172.21.15.90:0/878160215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565af7c5430 con 0x5565af73a650 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.546+0000 7f0386a00640 1 -- 172.21.15.90:0/878160215 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f038c0b5450 con 0x5565af73a650 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.546+0000 7f0386a00640 1 -- 172.21.15.90:0/878160215 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f038c0a7810 con 0x5565af73a650 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.546+0000 7f03a63e6740 10 client.15669 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.546+0000 7f03a63e6740 10 client.15669 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.546+0000 7f03a63e6740 20 client.15669 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.546+0000 7f03a63e6740 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x5565af7c99f0 0x5565af7c8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.546+0000 7f03a63e6740 4 client.15669 mds_command: new command op to 34640 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.546+0000 7f03a63e6740 1 -- 172.21.15.90:0/878160215 --> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] -- command(tid 0: {"prefix": "status"}) -- 0x5565af6ab750 con 0x5565af7c99f0 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.547+0000 7f039be00640 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x5565af7c99f0 0x5565af7c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:09.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.547+0000 7f039be00640 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x5565af7c99f0 0x5565af7c8700 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4622fb17ac9ae9d5 server_cookie=730b74921df82674 in_seq=0 out_seq=0 2024-09-10T22:04:09.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.547+0000 7f0386a00640 10 client.15669 ms_handle_connect on v2:172.21.15.154:6834/622088003 2024-09-10T22:04:09.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f0386a00640 1 -- 172.21.15.90:0/878160215 <== mds.3 v2:172.21.15.154:6834/622088003 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5565af6ab750 con 0x5565af7c99f0 2024-09-10T22:04:09.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f0386a00640 10 client.15669 handle_command_reply: tid=0 2024-09-10T22:04:09.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 client.15669 shutdown 2024-09-10T22:04:09.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 -- 172.21.15.90:0/878160215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 msgr2=0x5565af7619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:09.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af7619f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f038c0abe50 tx=0x7f038c0a8620 comp rx=0 tx=0).stop 2024-09-10T22:04:09.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 -- 172.21.15.90:0/878160215 shutdown_connections 2024-09-10T22:04:09.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5565af746f00 0x5565af75c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.154:6834/622088003,v1:172.21.15.154:6835/622088003] conn(0x5565af7c99f0 0x5565af7c8700 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5565af73af70 0x5565af75bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 --2- 172.21.15.90:0/878160215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5565af73a650 0x5565af7619f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f0386a00640 0 client.15669 ms_handle_reset on v2:172.21.15.154:6834/622088003 2024-09-10T22:04:09.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 -- 172.21.15.90:0/878160215 >> 172.21.15.90:0/878160215 conn(0x5565af738430 msgr2=0x5565af733140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:09.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 -- 172.21.15.90:0/878160215 shutdown_connections 2024-09-10T22:04:09.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 1 -- 172.21.15.90:0/878160215 wait complete. 2024-09-10T22:04:09.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.548+0000 7f03a63e6740 20 client.15669 trim_cache size 0 max 16384 2024-09-10T22:04:09.561 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 3, 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34640, 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 158.83035520600001, 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 30, 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 252.014269447 2024-09-10T22:04:09.562 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:04:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 -- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03800b5450 msgr2=0x7f03800b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03800b5450 0x7f03800b7910 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0394005fd0 tx=0x7f0394009600 comp rx=0 tx=0).stop 2024-09-10T22:04:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 -- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c103cc0 msgr2=0x7f039c199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c103cc0 0x7f039c199220 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f038c00d950 tx=0x7f038c00de10 comp rx=0 tx=0).stop 2024-09-10T22:04:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 -- 172.21.15.90:0/1159876650 shutdown_connections 2024-09-10T22:04:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f03800b5450 0x7f03800b7910 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f039c10d120 0x7f039c19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f039c1045e0 0x7f039c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 --2- 172.21.15.90:0/1159876650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f039c103cc0 0x7f039c199220 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:09.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 -- 172.21.15.90:0/1159876650 >> 172.21.15.90:0/1159876650 conn(0x7f039c0fd700 msgr2=0x7f039c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:09.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 -- 172.21.15.90:0/1159876650 shutdown_connections 2024-09-10T22:04:09.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:09.549+0000 7f03a3800640 1 -- 172.21.15.90:0/1159876650 wait complete. 2024-09-10T22:04:10.384 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T22:04:10.757 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:10.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:10 smithi171 ceph-mon[34138]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-10T22:04:11.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:10 smithi090 ceph-mon[30350]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-10T22:04:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:10 smithi154 ceph-mon[34495]: pgmap v426: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-10T22:04:11.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.289+0000 7f976ea00640 1 -- 172.21.15.90:0/183764641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 msgr2=0x7f97680758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:11.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.289+0000 7f976ea00640 1 --2- 172.21.15.90:0/183764641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 0x7f97680758a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f9758009900 tx=0x7f975802a410 comp rx=0 tx=0).stop 2024-09-10T22:04:11.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.289+0000 7f976ea00640 1 -- 172.21.15.90:0/183764641 shutdown_connections 2024-09-10T22:04:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.289+0000 7f976ea00640 1 --2- 172.21.15.90:0/183764641 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f976810f3f0 0x7f97681117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.289+0000 7f976ea00640 1 --2- 172.21.15.90:0/183764641 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9768075de0 0x7f976810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.289+0000 7f976ea00640 1 --2- 172.21.15.90:0/183764641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 0x7f97680758a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.289+0000 7f976ea00640 1 -- 172.21.15.90:0/183764641 >> 172.21.15.90:0/183764641 conn(0x7f97680fe830 msgr2=0x7f9768100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.289+0000 7f976ea00640 1 -- 172.21.15.90:0/183764641 shutdown_connections 2024-09-10T22:04:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.290+0000 7f976ea00640 1 -- 172.21.15.90:0/183764641 wait complete. 2024-09-10T22:04:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.291+0000 7f976ea00640 1 Processor -- start 2024-09-10T22:04:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.291+0000 7f976ea00640 1 -- start start 2024-09-10T22:04:11.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.291+0000 7f976ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 0x7f9768102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.291+0000 7f976ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9768075de0 0x7f97681030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.291+0000 7f976ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f976810f3f0 0x7f97681035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f976ea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9768114e80 con 0x7f97680754c0 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f976ea00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9768114d00 con 0x7f976810f3f0 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f976ea00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9768115000 con 0x7f9768075de0 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 0x7f9768102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 0x7f9768102b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43810/0 (socket says 172.21.15.90:43810) 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f9767e00640 1 -- 172.21.15.90:0/2705075568 learned_addr learned my addr 172.21.15.90:0/2705075568 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f976ca00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f976810f3f0 0x7f97681035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f9767400640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9768075de0 0x7f97681030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f9767e00640 1 -- 172.21.15.90:0/2705075568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9768075de0 msgr2=0x7f97681030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.292+0000 7f9767e00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9768075de0 0x7f97681030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.293+0000 7f9767e00640 1 -- 172.21.15.90:0/2705075568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f976810f3f0 msgr2=0x7f97681035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.293+0000 7f9767e00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f976810f3f0 0x7f97681035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.293+0000 7f9767e00640 1 -- 172.21.15.90:0/2705075568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97681aba60 con 0x7f97680754c0 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.293+0000 7f976ca00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f976810f3f0 0x7f97681035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.293+0000 7f9767e00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 0x7f9768102b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9758009230 tx=0x7f9758004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.293+0000 7f9764c00640 1 -- 172.21.15.90:0/2705075568 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975803f030 con 0x7f97680754c0 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.293+0000 7f976ea00640 1 -- 172.21.15.90:0/2705075568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97681abcf0 con 0x7f97680754c0 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.294+0000 7f9764c00640 1 -- 172.21.15.90:0/2705075568 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9758003a80 con 0x7f97680754c0 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.294+0000 7f9764c00640 1 -- 172.21.15.90:0/2705075568 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758035070 con 0x7f97680754c0 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.294+0000 7f976ea00640 1 -- 172.21.15.90:0/2705075568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97681ac1c0 con 0x7f97680754c0 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.298+0000 7f977147c740 1 Processor -- start 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.298+0000 7f977147c740 1 -- start start 2024-09-10T22:04:11.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.298+0000 7f977147c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431240 0x55a34f431620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.298+0000 7f977147c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.298+0000 7f977147c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f43daf0 0x55a34f43fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.298+0000 7f977147c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a34f3a8d50 con 0x55a34f43daf0 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.298+0000 7f977147c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a34f398180 con 0x55a34f431240 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.299+0000 7f977147c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a34f39bf90 con 0x55a34f431b60 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.299+0000 7f9764c00640 1 -- 172.21.15.90:0/2705075568 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9758003c30 con 0x7f97680754c0 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.299+0000 7f976ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431240 0x55a34f431620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.299+0000 7f9767400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f43daf0 0x55a34f43fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.299+0000 7f976ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431240 0x55a34f431620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:44232/0 (socket says 172.21.15.90:44232) 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.299+0000 7f976ca00640 1 -- 172.21.15.90:0/1667574290 learned_addr learned my addr 172.21.15.90:0/1667574290 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.300+0000 7f9767e00640 1 --2- 172.21.15.90:0/1667574290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.300+0000 7f9767400640 1 -- 172.21.15.90:0/1667574290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 msgr2=0x55a34f43d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:11.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.300+0000 7f9767400640 1 --2- 172.21.15.90:0/1667574290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.300+0000 7f9767400640 1 -- 172.21.15.90:0/1667574290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431240 msgr2=0x55a34f431620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.300+0000 7f9767400640 1 --2- 172.21.15.90:0/1667574290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431240 0x55a34f431620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.300+0000 7f9767400640 1 -- 172.21.15.90:0/1667574290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34f362a70 con 0x55a34f43daf0 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.301+0000 7f9767400640 1 --2- 172.21.15.90:0/1667574290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f43daf0 0x55a34f43fee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9754009b90 tx=0x7f9754010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50b457919d1c6df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.301+0000 7f975d200640 1 -- 172.21.15.90:0/1667574290 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754004020 con 0x55a34f43daf0 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.301+0000 7f975d200640 1 -- 172.21.15.90:0/1667574290 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97540041c0 con 0x55a34f43daf0 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.301+0000 7f975d200640 1 -- 172.21.15.90:0/1667574290 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97540124b0 con 0x55a34f43daf0 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.301+0000 7f9764c00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f974c0b54d0 0x7f974c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.301+0000 7f9764c00640 1 -- 172.21.15.90:0/2705075568 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9758106f80 con 0x7f97680754c0 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.302+0000 7f976ca00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f974c0b54d0 0x7f974c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.302+0000 7f977147c740 1 -- 172.21.15.90:0/1667574290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f43daf0 msgr2=0x55a34f43fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.302+0000 7f977147c740 1 --2- 172.21.15.90:0/1667574290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f43daf0 0x55a34f43fee0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9754009b90 tx=0x7f9754010740 comp rx=0 tx=0).stop 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.303+0000 7f977147c740 1 -- 172.21.15.90:0/1667574290 shutdown_connections 2024-09-10T22:04:11.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.303+0000 7f977147c740 1 --2- 172.21.15.90:0/1667574290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f43daf0 0x55a34f43fee0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.303+0000 7f977147c740 1 --2- 172.21.15.90:0/1667574290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.303+0000 7f977147c740 1 --2- 172.21.15.90:0/1667574290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431240 0x55a34f431620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.303+0000 7f977147c740 1 -- 172.21.15.90:0/1667574290 >> 172.21.15.90:0/1667574290 conn(0x55a34f42f020 msgr2=0x55a34f430210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.304+0000 7f977147c740 1 -- 172.21.15.90:0/1667574290 shutdown_connections 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.304+0000 7f977147c740 1 -- 172.21.15.90:0/1667574290 wait complete. 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.304+0000 7f976ca00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f974c0b54d0 0x7f974c0b7990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f97600118d0 tx=0x7f97600096c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.304+0000 7f977147c740 1 -- start start 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.305+0000 7f977147c740 1 Processor -- start 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.305+0000 7f977147c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f43aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.305+0000 7f977147c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.305+0000 7f977147c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 0x55a34f43b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.305+0000 7f977147c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a34f1f95f0 con 0x55a34f431240 2024-09-10T22:04:11.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.305+0000 7f977147c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a34f333460 con 0x55a34f43daf0 2024-09-10T22:04:11.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.305+0000 7f977147c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a34f440890 con 0x55a34f431b60 2024-09-10T22:04:11.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.305+0000 7f977147c740 10 client.0 _ll_register_callbacks cb 0x55a34f343210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:11.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.306+0000 7f9767400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.306+0000 7f9767400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:54864/0 (socket says 172.21.15.90:54864) 2024-09-10T22:04:11.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.306+0000 7f9767400640 1 -- 172.21.15.90:0/265322799 learned_addr learned my addr 172.21.15.90:0/265322799 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:11.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.306+0000 7f976ca00640 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 0x55a34f43b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.306+0000 7f9767e00640 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f43aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.307+0000 7f976ca00640 1 -- 172.21.15.90:0/265322799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 msgr2=0x55a34f43b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:11.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.307+0000 7f976ca00640 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.307+0000 7f976ca00640 1 -- 172.21.15.90:0/265322799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 msgr2=0x55a34f43aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.307+0000 7f976ca00640 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f43aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.307+0000 7f976ca00640 1 -- 172.21.15.90:0/265322799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a34f0f6540 con 0x55a34f43daf0 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.307+0000 7f9767400640 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.307+0000 7f976ca00640 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 0x55a34f43b560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9760010ac0 tx=0x7f9760004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fc1aa82a83e7039 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.307+0000 7f9751600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.308+0000 7f9751600640 1 -- 172.21.15.90:0/265322799 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976002c060 con 0x55a34f43daf0 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.308+0000 7f977147c740 1 -- 172.21.15.90:0/265322799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a34f335a70 con 0x55a34f43daf0 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.308+0000 7f9751600640 1 -- 172.21.15.90:0/265322799 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9760017030 con 0x55a34f43daf0 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.308+0000 7f9751600640 1 -- 172.21.15.90:0/265322799 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9760033050 con 0x55a34f43daf0 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.308+0000 7f977147c740 10 client.35375 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.308+0000 7f977147c740 1 -- 172.21.15.90:0/265322799 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a34f3506f0 con 0x55a34f43daf0 2024-09-10T22:04:11.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.309+0000 7f9751600640 1 -- 172.21.15.90:0/265322799 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a34f335a70 con 0x55a34f43daf0 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.309+0000 7f9751600640 1 -- 172.21.15.90:0/265322799 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9760027cc0 con 0x55a34f43daf0 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.309+0000 7f977147c740 10 client.35375 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.309+0000 7f977147c740 10 client.35375 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.310+0000 7f977147c740 20 client.35375 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.310+0000 7f977147c740 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55a34f44e980 0x55a34f44fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.310+0000 7f977147c740 4 client.35375 mds_command: new command op to 34646 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.310+0000 7f977147c740 1 -- 172.21.15.90:0/265322799 --> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a34f3aaca0 con 0x55a34f44e980 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.310+0000 7f9767e00640 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55a34f44e980 0x55a34f44fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.311+0000 7f9751600640 10 client.35375 ms_handle_connect on v2:172.21.15.171:6834/2969925217 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.311+0000 7f9767e00640 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55a34f44e980 0x55a34f44fdf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8be739e5c3b589a server_cookie=804ea1de5391b353 in_seq=0 out_seq=0 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f9751600640 1 -- 172.21.15.90:0/265322799 <== mds.2 v2:172.21.15.171:6834/2969925217 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a34f3aaca0 con 0x55a34f44e980 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f9751600640 10 client.35375 handle_command_reply: tid=0 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f977147c740 1 client.35375 shutdown 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f977147c740 1 -- 172.21.15.90:0/265322799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 msgr2=0x55a34f43b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:11.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f977147c740 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 0x55a34f43b560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9760010ac0 tx=0x7f9760004ab0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f977147c740 1 -- 172.21.15.90:0/265322799 shutdown_connections 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f977147c740 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55a34f44e980 0x55a34f44fdf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f977147c740 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 0x55a34f43b560 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f977147c740 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431b60 0x55a34f43b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f977147c740 1 --2- 172.21.15.90:0/265322799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f43aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f977147c740 1 -- 172.21.15.90:0/265322799 >> 172.21.15.90:0/265322799 conn(0x55a34f42f020 msgr2=0x55a34f434be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.312+0000 7f9751600640 0 client.35375 ms_handle_reset on v2:172.21.15.171:6834/2969925217 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.313+0000 7f977147c740 1 -- 172.21.15.90:0/265322799 shutdown_connections 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.313+0000 7f977147c740 1 -- 172.21.15.90:0/265322799 wait complete. 2024-09-10T22:04:11.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.313+0000 7f977147c740 20 client.35375 trim_cache size 0 max 16384 2024-09-10T22:04:11.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.333+0000 7f977147c740 1 Processor -- start 2024-09-10T22:04:11.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.333+0000 7f977147c740 1 -- start start 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f977147c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f435340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f977147c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431b60 0x55a34f430180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f977147c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f43daf0 0x55a34f4306c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f977147c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a34f44e130 con 0x55a34f431240 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f977147c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a34f44e800 con 0x55a34f431b60 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f977147c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a34f44eed0 con 0x55a34f43daf0 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f435340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f9767e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f435340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43850/0 (socket says 172.21.15.90:43850) 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f9767e00640 1 -- 172.21.15.90:0/3200719389 learned_addr learned my addr 172.21.15.90:0/3200719389 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.334+0000 7f9767400640 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431b60 0x55a34f430180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.335+0000 7f9767e00640 1 -- 172.21.15.90:0/3200719389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f43daf0 msgr2=0x55a34f4306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.335+0000 7f976ca00640 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f43daf0 0x55a34f4306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.335+0000 7f9767e00640 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f43daf0 0x55a34f4306c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.335+0000 7f9767e00640 1 -- 172.21.15.90:0/3200719389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431b60 msgr2=0x55a34f430180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:11.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.335+0000 7f9767e00640 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431b60 0x55a34f430180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.335+0000 7f9767e00640 1 -- 172.21.15.90:0/3200719389 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a34f44c3b0 con 0x55a34f431240 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.335+0000 7f9767e00640 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f435340 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9758005e70 tx=0x7f9758009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df92fb7bd7309baa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.335+0000 7f9767400640 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431b60 0x55a34f430180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.335+0000 7f9753e00640 1 -- 172.21.15.90:0/3200719389 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758049070 con 0x55a34f431240 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.336+0000 7f9753e00640 1 -- 172.21.15.90:0/3200719389 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97580ccba0 con 0x55a34f431240 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.336+0000 7f9753e00640 1 -- 172.21.15.90:0/3200719389 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580b2e60 con 0x55a34f431240 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.336+0000 7f977147c740 1 -- 172.21.15.90:0/3200719389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 msgr2=0x55a34f435340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.336+0000 7f977147c740 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f435340 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f9758005e70 tx=0x7f9758009260 comp rx=0 tx=0).stop 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.336+0000 7f977147c740 1 -- 172.21.15.90:0/3200719389 shutdown_connections 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.336+0000 7f977147c740 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f43daf0 0x55a34f4306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.336+0000 7f977147c740 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f431b60 0x55a34f430180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.336+0000 7f977147c740 1 --2- 172.21.15.90:0/3200719389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431240 0x55a34f435340 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.336+0000 7f977147c740 1 -- 172.21.15.90:0/3200719389 >> 172.21.15.90:0/3200719389 conn(0x55a34f42f020 msgr2=0x55a34f435d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.337+0000 7f977147c740 1 -- 172.21.15.90:0/3200719389 shutdown_connections 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.337+0000 7f977147c740 1 -- 172.21.15.90:0/3200719389 wait complete. 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.337+0000 7f977147c740 1 -- start start 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.337+0000 7f977147c740 1 Processor -- start 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f977147c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431240 0x55a34f4584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f977147c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431b60 0x55a34f458a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f977147c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 0x55a34f451b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f977147c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a34f44fc70 con 0x55a34f431b60 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f977147c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a34f44f5a0 con 0x55a34f43daf0 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f977147c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a34f450340 con 0x55a34f431240 2024-09-10T22:04:11.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f977147c740 10 client.0 _ll_register_callbacks cb 0x7f9760011eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:11.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f9767400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431b60 0x55a34f458a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f9767400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431b60 0x55a34f458a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43862/0 (socket says 172.21.15.90:43862) 2024-09-10T22:04:11.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f9767400640 1 -- 172.21.15.90:0/1867400988 learned_addr learned my addr 172.21.15.90:0/1867400988 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:11.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f9767e00640 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431240 0x55a34f4584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.338+0000 7f976ca00640 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 0x55a34f451b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f9767400640 1 -- 172.21.15.90:0/1867400988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431240 msgr2=0x55a34f4584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f9767400640 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431240 0x55a34f4584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f9767400640 1 -- 172.21.15.90:0/1867400988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 msgr2=0x55a34f451b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f9767400640 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 0x55a34f451b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f9767400640 1 -- 172.21.15.90:0/1867400988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a34f452350 con 0x55a34f431b60 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f9767400640 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431b60 0x55a34f458a40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9754003bd0 tx=0x7f9754005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83639a04f29d66c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f9751600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f9751600640 1 -- 172.21.15.90:0/1867400988 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754003cc0 con 0x55a34f431b60 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f9751600640 1 -- 172.21.15.90:0/1867400988 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9754005770 con 0x55a34f431b60 2024-09-10T22:04:11.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.339+0000 7f977147c740 1 -- 172.21.15.90:0/1867400988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97580ccba0 con 0x55a34f431b60 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f9751600640 1 -- 172.21.15.90:0/1867400988 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9754003e60 con 0x55a34f431b60 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f977147c740 10 client.15687 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f977147c740 1 -- 172.21.15.90:0/1867400988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a34f457e00 con 0x55a34f431b60 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f9751600640 1 -- 172.21.15.90:0/1867400988 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97580ccba0 con 0x55a34f431b60 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f9751600640 1 -- 172.21.15.90:0/1867400988 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f975400f040 con 0x55a34f431b60 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f977147c740 10 client.15687 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f977147c740 10 client.15687 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f977147c740 20 client.15687 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f977147c740 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55a34f4bf8f0 0x55a34f4be600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f977147c740 4 client.15687 mds_command: new command op to 34646 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.340+0000 7f977147c740 1 -- 172.21.15.90:0/1867400988 --> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] -- command(tid 0: {"prefix": "status"}) -- 0x55a34f3a6750 con 0x55a34f4bf8f0 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.341+0000 7f9767e00640 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55a34f4bf8f0 0x55a34f4be600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:11.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f9751600640 10 client.15687 ms_handle_connect on v2:172.21.15.171:6834/2969925217 2024-09-10T22:04:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f9767e00640 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55a34f4bf8f0 0x55a34f4be600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6d8e25eb68a8326 server_cookie=4090513b834a3686 in_seq=0 out_seq=0 2024-09-10T22:04:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f9751600640 1 -- 172.21.15.90:0/1867400988 <== mds.2 v2:172.21.15.171:6834/2969925217 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a34f3a6750 con 0x55a34f4bf8f0 2024-09-10T22:04:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f9751600640 10 client.15687 handle_command_reply: tid=0 2024-09-10T22:04:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f977147c740 1 client.15687 shutdown 2024-09-10T22:04:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f977147c740 1 -- 172.21.15.90:0/1867400988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431b60 msgr2=0x55a34f458a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:11.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f977147c740 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431b60 0x55a34f458a40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9754003bd0 tx=0x7f9754005270 comp rx=0 tx=0).stop 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f977147c740 1 -- 172.21.15.90:0/1867400988 shutdown_connections 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f977147c740 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.171:6834/2969925217,v1:172.21.15.171:6835/2969925217] conn(0x55a34f4bf8f0 0x55a34f4be600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f977147c740 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a34f43daf0 0x55a34f451b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f977147c740 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a34f431b60 0x55a34f458a40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f977147c740 1 --2- 172.21.15.90:0/1867400988 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a34f431240 0x55a34f4584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.342+0000 7f977147c740 1 -- 172.21.15.90:0/1867400988 >> 172.21.15.90:0/1867400988 conn(0x55a34f42f020 msgr2=0x55a34f430180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.343+0000 7f9751600640 0 client.15687 ms_handle_reset on v2:172.21.15.171:6834/2969925217 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.343+0000 7f977147c740 1 -- 172.21.15.90:0/1867400988 shutdown_connections 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.343+0000 7f977147c740 1 -- 172.21.15.90:0/1867400988 wait complete. 2024-09-10T22:04:11.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.343+0000 7f977147c740 20 client.15687 trim_cache size 0 max 16384 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34646, 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 161.62841764999999, 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 30, 2024-09-10T22:04:11.348 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:04:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-09-10T22:04:11.349 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 250.42784919799999 2024-09-10T22:04:11.349 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:04:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 -- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f974c0b54d0 msgr2=0x7f974c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f974c0b54d0 0x7f974c0b7990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f97600118d0 tx=0x7f97600096c0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 -- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 msgr2=0x7f9768102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:11.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 0x7f9768102b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9758009230 tx=0x7f9758004270 comp rx=0 tx=0).stop 2024-09-10T22:04:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 -- 172.21.15.90:0/2705075568 shutdown_connections 2024-09-10T22:04:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f974c0b54d0 0x7f974c0b7990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f976810f3f0 0x7f97681035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9768075de0 0x7f97681030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 --2- 172.21.15.90:0/2705075568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97680754c0 0x7f9768102b60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 -- 172.21.15.90:0/2705075568 >> 172.21.15.90:0/2705075568 conn(0x7f97680fe830 msgr2=0x7f97680ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 -- 172.21.15.90:0/2705075568 shutdown_connections 2024-09-10T22:04:11.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:11.344+0000 7f976ea00640 1 -- 172.21.15.90:0/2705075568 wait complete. 2024-09-10T22:04:12.246 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T22:04:12.644 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:12.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:12 smithi171 ceph-mon[34138]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-10T22:04:12.998 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:12 smithi090 ceph-mon[30350]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-10T22:04:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:12 smithi154 ceph-mon[34495]: pgmap v427: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 26 op/s 2024-09-10T22:04:13.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.222+0000 7ff52d600640 1 -- 172.21.15.90:0/3886836424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108620 msgr2=0x7ff528108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:13.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.222+0000 7ff52d600640 1 --2- 172.21.15.90:0/3886836424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108620 0x7ff528108a60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff514009900 tx=0x7ff51402a410 comp rx=0 tx=0).stop 2024-09-10T22:04:13.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.222+0000 7ff52d600640 1 -- 172.21.15.90:0/3886836424 shutdown_connections 2024-09-10T22:04:13.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.222+0000 7ff52d600640 1 --2- 172.21.15.90:0/3886836424 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff528108fa0 0x7ff52810f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.222+0000 7ff52d600640 1 --2- 172.21.15.90:0/3886836424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108620 0x7ff528108a60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.222+0000 7ff52d600640 1 --2- 172.21.15.90:0/3886836424 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528102c10 0x7ff528102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.222+0000 7ff52d600640 1 -- 172.21.15.90:0/3886836424 >> 172.21.15.90:0/3886836424 conn(0x7ff5280fe660 msgr2=0x7ff528100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:13.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.222+0000 7ff52d600640 1 -- 172.21.15.90:0/3886836424 shutdown_connections 2024-09-10T22:04:13.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.222+0000 7ff52d600640 1 -- 172.21.15.90:0/3886836424 wait complete. 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.223+0000 7ff52d600640 1 Processor -- start 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.223+0000 7ff52d600640 1 -- start start 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff52d600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528102c10 0x7ff52819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff52d600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff528108620 0x7ff52819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff52d600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108fa0 0x7ff5281a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff52d600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff528112b10 con 0x7ff528108fa0 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff52d600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff528112990 con 0x7ff528108620 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff52d600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff528112c90 con 0x7ff528102c10 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff52cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108fa0 0x7ff5281a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff52cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108fa0 0x7ff5281a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43878/0 (socket says 172.21.15.90:43878) 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff52cc00640 1 -- 172.21.15.90:0/1624369264 learned_addr learned my addr 172.21.15.90:0/1624369264 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:13.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.224+0000 7ff527e00640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528102c10 0x7ff52819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff52cc00640 1 -- 172.21.15.90:0/1624369264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528102c10 msgr2=0x7ff52819b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:13.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff52cc00640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528102c10 0x7ff52819b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff52cc00640 1 -- 172.21.15.90:0/1624369264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff528108620 msgr2=0x7ff52819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:13.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff527400640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff528108620 0x7ff52819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff52cc00640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff528108620 0x7ff52819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff52cc00640 1 -- 172.21.15.90:0/1624369264 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5281a1200 con 0x7ff528108fa0 2024-09-10T22:04:13.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff527400640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff528108620 0x7ff52819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:13.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff52cc00640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108fa0 0x7ff5281a0a30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff520009fd0 tx=0x7ff52000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:13.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff524c00640 1 -- 172.21.15.90:0/1624369264 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520017070 con 0x7ff528108fa0 2024-09-10T22:04:13.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff52d600640 1 -- 172.21.15.90:0/1624369264 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5281a14f0 con 0x7ff528108fa0 2024-09-10T22:04:13.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff524c00640 1 -- 172.21.15.90:0/1624369264 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff520003e00 con 0x7ff528108fa0 2024-09-10T22:04:13.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.225+0000 7ff524c00640 1 -- 172.21.15.90:0/1624369264 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520010460 con 0x7ff528108fa0 2024-09-10T22:04:13.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.226+0000 7ff52d600640 1 -- 172.21.15.90:0/1624369264 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5281a7bb0 con 0x7ff528108fa0 2024-09-10T22:04:13.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.229+0000 7ff5317e2740 1 Processor -- start 2024-09-10T22:04:13.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.230+0000 7ff524c00640 1 -- 172.21.15.90:0/1624369264 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff520003970 con 0x7ff528108fa0 2024-09-10T22:04:13.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.230+0000 7ff5317e2740 1 -- start start 2024-09-10T22:04:13.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 0x560c9c2337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c23f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 0x560c9c2420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff5317e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560c9c1aad50 con 0x560c9c233d20 2024-09-10T22:04:13.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff5317e2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560c9c19a180 con 0x560c9c233400 2024-09-10T22:04:13.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff5317e2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c9c19df90 con 0x560c9c23fcb0 2024-09-10T22:04:13.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff527e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c23f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff527e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c23f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43890/0 (socket says 172.21.15.90:43890) 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff527e00640 1 -- 172.21.15.90:0/1528840527 learned_addr learned my addr 172.21.15.90:0/1528840527 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff527400640 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 0x560c9c2420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff527400640 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 0x560c9c2337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.231+0000 7ff527e00640 1 -- 172.21.15.90:0/1528840527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 msgr2=0x560c9c2420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.232+0000 7ff527e00640 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 0x560c9c2420a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.232+0000 7ff527e00640 1 -- 172.21.15.90:0/1528840527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 msgr2=0x560c9c2337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.232+0000 7ff527e00640 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 0x560c9c2337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.232+0000 7ff527e00640 1 -- 172.21.15.90:0/1528840527 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c9c164a70 con 0x560c9c233d20 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.232+0000 7ff527400640 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 0x560c9c2337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:04:13.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.232+0000 7ff524c00640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b5500 0x7ff50c0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.232+0000 7ff524c00640 1 -- 172.21.15.90:0/1624369264 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff5200e6510 con 0x7ff528108fa0 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.232+0000 7ff527e00640 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c23f770 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff51800bda0 tx=0x7ff5180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980b884e5a17f9b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.232+0000 7ff527400640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b5500 0x7ff50c0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.233+0000 7ff51d200640 1 -- 172.21.15.90:0/1528840527 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518004230 con 0x560c9c233d20 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.233+0000 7ff51d200640 1 -- 172.21.15.90:0/1528840527 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5180043d0 con 0x560c9c233d20 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.233+0000 7ff51d200640 1 -- 172.21.15.90:0/1528840527 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518023680 con 0x560c9c233d20 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.233+0000 7ff527400640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b5500 0x7ff50c0b79c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x560c9c2333b0 tx=0x7ff514005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.233+0000 7ff5317e2740 1 -- 172.21.15.90:0/1528840527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 msgr2=0x560c9c23f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.234+0000 7ff5317e2740 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c23f770 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff51800bda0 tx=0x7ff5180107a0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.234+0000 7ff5317e2740 1 -- 172.21.15.90:0/1528840527 shutdown_connections 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.234+0000 7ff5317e2740 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 0x560c9c2420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.234+0000 7ff5317e2740 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c23f770 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.234+0000 7ff5317e2740 1 --2- 172.21.15.90:0/1528840527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 0x560c9c2337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.234+0000 7ff5317e2740 1 -- 172.21.15.90:0/1528840527 >> 172.21.15.90:0/1528840527 conn(0x560c9c231230 msgr2=0x560c9c2323d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:13.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.234+0000 7ff5317e2740 1 -- 172.21.15.90:0/1528840527 shutdown_connections 2024-09-10T22:04:13.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.234+0000 7ff5317e2740 1 -- 172.21.15.90:0/1528840527 wait complete. 2024-09-10T22:04:13.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.234+0000 7ff5317e2740 1 -- start start 2024-09-10T22:04:13.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.235+0000 7ff5317e2740 1 Processor -- start 2024-09-10T22:04:13.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.235+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c23ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.235+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233d20 0x560c9c23d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.235+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c23fcb0 0x560c9c23d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.235+0000 7ff5317e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560c9bffb5f0 con 0x560c9c23fcb0 2024-09-10T22:04:13.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.235+0000 7ff5317e2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560c9c135460 con 0x560c9c233d20 2024-09-10T22:04:13.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.235+0000 7ff5317e2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c9c242a00 con 0x560c9c233400 2024-09-10T22:04:13.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.235+0000 7ff5317e2740 10 client.0 _ll_register_callbacks cb 0x560c9c145210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233d20 0x560c9c23d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c23ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff52cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c23fcb0 0x560c9c23d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c23ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:54940/0 (socket says 172.21.15.90:54940) 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233d20 0x560c9c23d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:44300/0 (socket says 172.21.15.90:44300) 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527e00640 1 -- 172.21.15.90:0/977164045 learned_addr learned my addr 172.21.15.90:0/977164045 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527e00640 1 -- 172.21.15.90:0/977164045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233d20 msgr2=0x560c9c23d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527e00640 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233d20 0x560c9c23d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527e00640 1 -- 172.21.15.90:0/977164045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c23fcb0 msgr2=0x560c9c23d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527e00640 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c23fcb0 0x560c9c23d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527e00640 1 -- 172.21.15.90:0/977164045 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c9bef8540 con 0x560c9c233400 2024-09-10T22:04:13.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff527e00640 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c23ccb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff518003f10 tx=0x7ff51800ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a64efe487ef7523d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:13.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.236+0000 7ff511600640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:04:13.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.237+0000 7ff511600640 1 -- 172.21.15.90:0/977164045 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518031060 con 0x560c9c233400 2024-09-10T22:04:13.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.237+0000 7ff511600640 1 -- 172.21.15.90:0/977164045 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff518003fb0 con 0x560c9c233400 2024-09-10T22:04:13.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.237+0000 7ff5317e2740 1 -- 172.21.15.90:0/977164045 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560c9c137a70 con 0x560c9c233400 2024-09-10T22:04:13.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.237+0000 7ff511600640 1 -- 172.21.15.90:0/977164045 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff518038050 con 0x560c9c233400 2024-09-10T22:04:13.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.237+0000 7ff5317e2740 10 client.25363 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:13.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.237+0000 7ff5317e2740 1 -- 172.21.15.90:0/977164045 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c9c1526f0 con 0x560c9c233400 2024-09-10T22:04:13.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.238+0000 7ff511600640 1 -- 172.21.15.90:0/977164045 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560c9c137a70 con 0x560c9c233400 2024-09-10T22:04:13.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.238+0000 7ff511600640 1 -- 172.21.15.90:0/977164045 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff5180046c0 con 0x560c9c233400 2024-09-10T22:04:13.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.238+0000 7ff5317e2740 10 client.25363 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:13.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.238+0000 7ff5317e2740 10 client.25363 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:04:13.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.238+0000 7ff5317e2740 20 client.25363 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:13.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.238+0000 7ff5317e2740 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x560c9c2506b0 0x560c9c251b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.238+0000 7ff5317e2740 4 client.25363 mds_command: new command op to 34670 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:04:13.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.238+0000 7ff5317e2740 1 -- 172.21.15.90:0/977164045 --> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c9c1acca0 con 0x560c9c2506b0 2024-09-10T22:04:13.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.239+0000 7ff527400640 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x560c9c2506b0 0x560c9c251b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.239+0000 7ff527400640 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x560c9c2506b0 0x560c9c251b20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a96dbdb3a1ce6b5e server_cookie=e9192a60c1f962a1 in_seq=0 out_seq=0 2024-09-10T22:04:13.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.239+0000 7ff511600640 10 client.25363 ms_handle_connect on v2:172.21.15.154:6836/3907253955 2024-09-10T22:04:13.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff511600640 1 -- 172.21.15.90:0/977164045 <== mds.1 v2:172.21.15.154:6836/3907253955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560c9c1acca0 con 0x560c9c2506b0 2024-09-10T22:04:13.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff511600640 10 client.25363 handle_command_reply: tid=0 2024-09-10T22:04:13.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff5317e2740 1 client.25363 shutdown 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff5317e2740 1 -- 172.21.15.90:0/977164045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 msgr2=0x560c9c23ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff5317e2740 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c23ccb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff518003f10 tx=0x7ff51800ba50 comp rx=0 tx=0).stop 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff5317e2740 1 -- 172.21.15.90:0/977164045 shutdown_connections 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff5317e2740 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x560c9c2506b0 0x560c9c251b20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff5317e2740 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c23fcb0 0x560c9c23d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff5317e2740 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233d20 0x560c9c23d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff511600640 0 client.25363 ms_handle_reset on v2:172.21.15.154:6836/3907253955 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff5317e2740 1 --2- 172.21.15.90:0/977164045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c23ccb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.240+0000 7ff5317e2740 1 -- 172.21.15.90:0/977164045 >> 172.21.15.90:0/977164045 conn(0x560c9c231230 msgr2=0x560c9c236dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.241+0000 7ff5317e2740 1 -- 172.21.15.90:0/977164045 shutdown_connections 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.241+0000 7ff5317e2740 1 -- 172.21.15.90:0/977164045 wait complete. 2024-09-10T22:04:13.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.241+0000 7ff5317e2740 20 client.25363 trim_cache size 0 max 16384 2024-09-10T22:04:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.259+0000 7ff5317e2740 1 Processor -- start 2024-09-10T22:04:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.259+0000 7ff5317e2740 1 -- start start 2024-09-10T22:04:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 0x560c9c237470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c239ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 0x560c9c23a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff5317e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560c9c24fe60 con 0x560c9c233d20 2024-09-10T22:04:13.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff5317e2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560c9c250530 con 0x560c9c233400 2024-09-10T22:04:13.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff5317e2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c9c250c00 con 0x560c9c23fcb0 2024-09-10T22:04:13.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff527400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c239ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff527400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c239ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43902/0 (socket says 172.21.15.90:43902) 2024-09-10T22:04:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff527400640 1 -- 172.21.15.90:0/913707559 learned_addr learned my addr 172.21.15.90:0/913707559 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff527e00640 1 --2- 172.21.15.90:0/913707559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 0x560c9c237470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.260+0000 7ff52cc00640 1 --2- 172.21.15.90:0/913707559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 0x560c9c23a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.261+0000 7ff527400640 1 -- 172.21.15.90:0/913707559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 msgr2=0x560c9c23a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.261+0000 7ff527400640 1 --2- 172.21.15.90:0/913707559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 0x560c9c23a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.261+0000 7ff527400640 1 -- 172.21.15.90:0/913707559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 msgr2=0x560c9c237470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.261+0000 7ff527400640 1 --2- 172.21.15.90:0/913707559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 0x560c9c237470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.261+0000 7ff527400640 1 -- 172.21.15.90:0/913707559 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c9c233b50 con 0x560c9c233d20 2024-09-10T22:04:13.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.261+0000 7ff527400640 1 --2- 172.21.15.90:0/913707559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c239ec0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff5140057c0 tx=0x7ff514009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab08e9350017dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:13.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.262+0000 7ff513e00640 1 -- 172.21.15.90:0/913707559 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514035040 con 0x560c9c233d20 2024-09-10T22:04:13.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.262+0000 7ff513e00640 1 -- 172.21.15.90:0/913707559 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff514043040 con 0x560c9c233d20 2024-09-10T22:04:13.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.262+0000 7ff513e00640 1 -- 172.21.15.90:0/913707559 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5140046c0 con 0x560c9c233d20 2024-09-10T22:04:13.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.262+0000 7ff5317e2740 1 -- 172.21.15.90:0/913707559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 msgr2=0x560c9c239ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:13.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.262+0000 7ff5317e2740 1 --2- 172.21.15.90:0/913707559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c239ec0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff5140057c0 tx=0x7ff514009260 comp rx=0 tx=0).stop 2024-09-10T22:04:13.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.263+0000 7ff5317e2740 1 -- 172.21.15.90:0/913707559 shutdown_connections 2024-09-10T22:04:13.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.263+0000 7ff5317e2740 1 --2- 172.21.15.90:0/913707559 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c23fcb0 0x560c9c23a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.263+0000 7ff5317e2740 1 --2- 172.21.15.90:0/913707559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c239ec0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.263+0000 7ff5317e2740 1 --2- 172.21.15.90:0/913707559 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c233400 0x560c9c237470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.263+0000 7ff5317e2740 1 -- 172.21.15.90:0/913707559 >> 172.21.15.90:0/913707559 conn(0x560c9c231230 msgr2=0x560c9c237fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:13.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.263+0000 7ff5317e2740 1 -- 172.21.15.90:0/913707559 shutdown_connections 2024-09-10T22:04:13.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.263+0000 7ff5317e2740 1 -- 172.21.15.90:0/913707559 wait complete. 2024-09-10T22:04:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.263+0000 7ff5317e2740 1 -- start start 2024-09-10T22:04:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.264+0000 7ff5317e2740 1 Processor -- start 2024-09-10T22:04:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.264+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c254110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.264+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c254650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.264+0000 7ff5317e2740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c23fcb0 0x560c9c2598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.264+0000 7ff5317e2740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560c9c2519a0 con 0x560c9c233d20 2024-09-10T22:04:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.264+0000 7ff5317e2740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560c9c2512d0 con 0x560c9c23fcb0 2024-09-10T22:04:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.264+0000 7ff5317e2740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560c9c252070 con 0x560c9c233400 2024-09-10T22:04:13.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.264+0000 7ff5317e2740 10 client.0 _ll_register_callbacks cb 0x7ff51800bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.264+0000 7ff527400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c254650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff527400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c254650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43918/0 (socket says 172.21.15.90:43918) 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff527400640 1 -- 172.21.15.90:0/2049084045 learned_addr learned my addr 172.21.15.90:0/2049084045 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff527e00640 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c254110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff52cc00640 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c23fcb0 0x560c9c2598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff527400640 1 -- 172.21.15.90:0/2049084045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 msgr2=0x560c9c254110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff527400640 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c254110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff527400640 1 -- 172.21.15.90:0/2049084045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c23fcb0 msgr2=0x560c9c2598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff527400640 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c23fcb0 0x560c9c2598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff527400640 1 -- 172.21.15.90:0/2049084045 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c9c259de0 con 0x560c9c233d20 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff527400640 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c254650 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff514032fd0 tx=0x7ff5140315c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ca07e60e7455db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:13.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.265+0000 7ff511600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:13.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.266+0000 7ff511600640 1 -- 172.21.15.90:0/2049084045 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514035070 con 0x560c9c233d20 2024-09-10T22:04:13.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.266+0000 7ff511600640 1 -- 172.21.15.90:0/2049084045 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7ff514031a70 con 0x560c9c233d20 2024-09-10T22:04:13.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.266+0000 7ff5317e2740 1 -- 172.21.15.90:0/2049084045 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff514043040 con 0x560c9c233d20 2024-09-10T22:04:13.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff511600640 1 -- 172.21.15.90:0/2049084045 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff514031c70 con 0x560c9c233d20 2024-09-10T22:04:13.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff5317e2740 10 client.15705 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:13.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff5317e2740 1 -- 172.21.15.90:0/2049084045 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560c9c2bd730 con 0x560c9c233d20 2024-09-10T22:04:13.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff511600640 1 -- 172.21.15.90:0/2049084045 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff514043040 con 0x560c9c233d20 2024-09-10T22:04:13.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff511600640 1 -- 172.21.15.90:0/2049084045 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff514048070 con 0x560c9c233d20 2024-09-10T22:04:13.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff5317e2740 10 client.15705 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:13.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff5317e2740 10 client.15705 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T22:04:13.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff5317e2740 20 client.15705 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:13.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff5317e2740 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x560c9c2c1ea0 0x560c9c2c0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:13.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff5317e2740 4 client.15705 mds_command: new command op to 34670 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:04:13.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.267+0000 7ff5317e2740 1 -- 172.21.15.90:0/2049084045 --> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] -- command(tid 0: {"prefix": "status"}) -- 0x560c9c1a8750 con 0x560c9c2c1ea0 2024-09-10T22:04:13.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.268+0000 7ff527e00640 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x560c9c2c1ea0 0x560c9c2c0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:13.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff527e00640 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x560c9c2c1ea0 0x560c9c2c0bb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cdf323b9f393e582 server_cookie=7ddcd4ffe2d65b59 in_seq=0 out_seq=0 2024-09-10T22:04:13.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff511600640 10 client.15705 ms_handle_connect on v2:172.21.15.154:6836/3907253955 2024-09-10T22:04:13.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff511600640 1 -- 172.21.15.90:0/2049084045 <== mds.1 v2:172.21.15.154:6836/3907253955 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560c9c1a8750 con 0x560c9c2c1ea0 2024-09-10T22:04:13.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff511600640 10 client.15705 handle_command_reply: tid=0 2024-09-10T22:04:13.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 client.15705 shutdown 2024-09-10T22:04:13.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 -- 172.21.15.90:0/2049084045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 msgr2=0x560c9c254650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:13.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c254650 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff514032fd0 tx=0x7ff5140315c0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.272 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:04:13.272 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34670, 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 164.57060478899999, 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 30, 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 245.679352424 2024-09-10T22:04:13.273 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:04:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 -- 172.21.15.90:0/2049084045 shutdown_connections 2024-09-10T22:04:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.154:6836/3907253955,v1:172.21.15.154:6837/3907253955] conn(0x560c9c2c1ea0 0x560c9c2c0bb0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560c9c23fcb0 0x560c9c2598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560c9c233d20 0x560c9c254650 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 --2- 172.21.15.90:0/2049084045 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560c9c233400 0x560c9c254110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 -- 172.21.15.90:0/2049084045 >> 172.21.15.90:0/2049084045 conn(0x560c9c231230 msgr2=0x560c9c23e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:13.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff511600640 0 client.15705 ms_handle_reset on v2:172.21.15.154:6836/3907253955 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 -- 172.21.15.90:0/2049084045 shutdown_connections 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.269+0000 7ff5317e2740 1 -- 172.21.15.90:0/2049084045 wait complete. 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.270+0000 7ff5317e2740 20 client.15705 trim_cache size 0 max 16384 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.270+0000 7ff52d600640 1 -- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b5500 msgr2=0x7ff50c0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.270+0000 7ff52d600640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b5500 0x7ff50c0b79c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x560c9c2333b0 tx=0x7ff514005fd0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.270+0000 7ff52d600640 1 -- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108fa0 msgr2=0x7ff5281a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.270+0000 7ff52d600640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108fa0 0x7ff5281a0a30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff520009fd0 tx=0x7ff52000b260 comp rx=0 tx=0).stop 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.271+0000 7ff52d600640 1 -- 172.21.15.90:0/1624369264 shutdown_connections 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.271+0000 7ff52d600640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b5500 0x7ff50c0b79c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.271+0000 7ff52d600640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff528108fa0 0x7ff5281a0a30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.271+0000 7ff52d600640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff528108620 0x7ff52819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.271+0000 7ff52d600640 1 --2- 172.21.15.90:0/1624369264 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff528102c10 0x7ff52819b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:13.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.271+0000 7ff52d600640 1 -- 172.21.15.90:0/1624369264 >> 172.21.15.90:0/1624369264 conn(0x7ff5280fe660 msgr2=0x7ff528104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:13.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.271+0000 7ff52d600640 1 -- 172.21.15.90:0/1624369264 shutdown_connections 2024-09-10T22:04:13.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:13.271+0000 7ff52d600640 1 -- 172.21.15.90:0/1624369264 wait complete. 2024-09-10T22:04:14.059 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T22:04:14.457 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:14.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:14 smithi090 ceph-mon[30350]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-10T22:04:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.097+0000 7f5e88600640 1 -- 172.21.15.90:0/1996917060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104fc0 msgr2=0x7f5e801053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.097+0000 7f5e88600640 1 --2- 172.21.15.90:0/1996917060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104fc0 0x7f5e801053a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009900 tx=0x7f5e7402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.097+0000 7f5e88600640 1 -- 172.21.15.90:0/1996917060 shutdown_connections 2024-09-10T22:04:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.097+0000 7f5e88600640 1 --2- 172.21.15.90:0/1996917060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e8010b350 0x7f5e80111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.097+0000 7f5e88600640 1 --2- 172.21.15.90:0/1996917060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e8010a9d0 0x7f5e8010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.097+0000 7f5e88600640 1 --2- 172.21.15.90:0/1996917060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104fc0 0x7f5e801053a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.097+0000 7f5e88600640 1 -- 172.21.15.90:0/1996917060 >> 172.21.15.90:0/1996917060 conn(0x7f5e801009f0 msgr2=0x7f5e80102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:15.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.097+0000 7f5e88600640 1 -- 172.21.15.90:0/1996917060 shutdown_connections 2024-09-10T22:04:15.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.097+0000 7f5e88600640 1 -- 172.21.15.90:0/1996917060 wait complete. 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.098+0000 7f5e88600640 1 Processor -- start 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.098+0000 7f5e88600640 1 -- start start 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e88600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e80104fc0 0x7f5e8019b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e88600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e8010a9d0 0x7f5e8019b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e88600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e8010b350 0x7f5e801a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e88600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80114e30 con 0x7f5e8010a9d0 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e88600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80114cb0 con 0x7f5e8010b350 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e88600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80114fb0 con 0x7f5e80104fc0 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e85200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e8010a9d0 0x7f5e8019b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e85200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e8010a9d0 0x7f5e8019b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43930/0 (socket says 172.21.15.90:43930) 2024-09-10T22:04:15.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e85200640 1 -- 172.21.15.90:0/1716117012 learned_addr learned my addr 172.21.15.90:0/1716117012 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:15.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e85c00640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e80104fc0 0x7f5e8019b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e86600640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e8010b350 0x7f5e801a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.099+0000 7f5e85200640 1 -- 172.21.15.90:0/1716117012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e80104fc0 msgr2=0x7f5e8019b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e85200640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e80104fc0 0x7f5e8019b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e85200640 1 -- 172.21.15.90:0/1716117012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e8010b350 msgr2=0x7f5e801a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e85200640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e8010b350 0x7f5e801a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e85200640 1 -- 172.21.15.90:0/1716117012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e801a13b0 con 0x7f5e8010a9d0 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e85200640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e8010a9d0 0x7f5e8019b840 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00c920 tx=0x7f5e6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e7e000640 1 -- 172.21.15.90:0/1716117012 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c007c70 con 0x7f5e8010a9d0 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e7e000640 1 -- 172.21.15.90:0/1716117012 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e6c007e10 con 0x7f5e8010a9d0 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e7e000640 1 -- 172.21.15.90:0/1716117012 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c018710 con 0x7f5e8010a9d0 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e88600640 1 -- 172.21.15.90:0/1716117012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e801a16a0 con 0x7f5e8010a9d0 2024-09-10T22:04:15.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.100+0000 7f5e88600640 1 -- 172.21.15.90:0/1716117012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e801a7d60 con 0x7f5e8010a9d0 2024-09-10T22:04:15.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.104+0000 7f5e8b010740 1 Processor -- start 2024-09-10T22:04:15.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.104+0000 7f5e7e000640 1 -- 172.21.15.90:0/1716117012 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5e6c00f450 con 0x7f5e8010a9d0 2024-09-10T22:04:15.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.104+0000 7f5e8b010740 1 -- start start 2024-09-10T22:04:15.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.104+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657baa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.104+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 0x55a657bb65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.104+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657bb8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.104+0000 7f5e8b010740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a657b21d50 con 0x55a657baa240 2024-09-10T22:04:15.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.104+0000 7f5e8b010740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a657b11180 con 0x55a657baab60 2024-09-10T22:04:15.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.104+0000 7f5e8b010740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a657b14f90 con 0x55a657bb6af0 2024-09-10T22:04:15.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e86600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657baa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e86600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657baa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43936/0 (socket says 172.21.15.90:43936) 2024-09-10T22:04:15.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e86600640 1 -- 172.21.15.90:0/2585111177 learned_addr learned my addr 172.21.15.90:0/2585111177 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:15.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e85200640 1 --2- 172.21.15.90:0/2585111177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657bb8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e85c00640 1 --2- 172.21.15.90:0/2585111177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 0x55a657bb65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e86600640 1 -- 172.21.15.90:0/2585111177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 msgr2=0x55a657bb8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:15.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e86600640 1 --2- 172.21.15.90:0/2585111177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657bb8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e86600640 1 -- 172.21.15.90:0/2585111177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 msgr2=0x55a657bb65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:15.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e86600640 1 --2- 172.21.15.90:0/2585111177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 0x55a657bb65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.105+0000 7f5e86600640 1 -- 172.21.15.90:0/2585111177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a657adba70 con 0x55a657baa240 2024-09-10T22:04:15.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.106+0000 7f5e86600640 1 --2- 172.21.15.90:0/2585111177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657baa620 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009f10 tx=0x7f5e78010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c351313701b2ec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:15.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.106+0000 7f5e72a00640 1 -- 172.21.15.90:0/2585111177 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78016070 con 0x55a657baa240 2024-09-10T22:04:15.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.106+0000 7f5e72a00640 1 -- 172.21.15.90:0/2585111177 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e78004d20 con 0x55a657baa240 2024-09-10T22:04:15.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.106+0000 7f5e72a00640 1 -- 172.21.15.90:0/2585111177 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78012670 con 0x55a657baa240 2024-09-10T22:04:15.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.106+0000 7f5e7e000640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5e680b5450 0x7f5e680b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.106+0000 7f5e7e000640 1 -- 172.21.15.90:0/1716117012 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f5e6c0e65b0 con 0x7f5e8010a9d0 2024-09-10T22:04:15.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e85c00640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5e680b5450 0x7f5e680b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e8b010740 1 -- 172.21.15.90:0/2585111177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 msgr2=0x55a657baa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:15.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2585111177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657baa620 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009f10 tx=0x7f5e78010740 comp rx=0 tx=0).stop 2024-09-10T22:04:15.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e8b010740 1 -- 172.21.15.90:0/2585111177 shutdown_connections 2024-09-10T22:04:15.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2585111177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657bb8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2585111177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 0x55a657bb65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2585111177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657baa620 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e8b010740 1 -- 172.21.15.90:0/2585111177 >> 172.21.15.90:0/2585111177 conn(0x55a657ba8020 msgr2=0x55a657ba9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:15.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e8b010740 1 -- 172.21.15.90:0/2585111177 shutdown_connections 2024-09-10T22:04:15.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e85c00640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5e680b5450 0x7f5e680b7910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5e740057c0 tx=0x7f5e740058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:15.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.107+0000 7f5e8b010740 1 -- 172.21.15.90:0/2585111177 wait complete. 2024-09-10T22:04:15.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.108+0000 7f5e8b010740 1 -- start start 2024-09-10T22:04:15.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.108+0000 7f5e8b010740 1 Processor -- start 2024-09-10T22:04:15.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bb3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 0x55a657bb3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657bb44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e8b010740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a6579725f0 con 0x55a657baa240 2024-09-10T22:04:15.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e8b010740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a657aac460 con 0x55a657baab60 2024-09-10T22:04:15.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e8b010740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a657bb9840 con 0x55a657bb6af0 2024-09-10T22:04:15.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e8b010740 10 client.0 _ll_register_callbacks cb 0x55a657abc210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:15.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e85c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bb3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e85c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bb3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43950/0 (socket says 172.21.15.90:43950) 2024-09-10T22:04:15.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e85c00640 1 -- 172.21.15.90:0/1160179344 learned_addr learned my addr 172.21.15.90:0/1160179344 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:15.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e86600640 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657bb44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.109+0000 7f5e85200640 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 0x55a657bb3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e85c00640 1 -- 172.21.15.90:0/1160179344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 msgr2=0x55a657bb44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:15.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e85c00640 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657bb44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e85c00640 1 -- 172.21.15.90:0/1160179344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 msgr2=0x55a657bb3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:15.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e85c00640 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 0x55a657bb3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e85c00640 1 -- 172.21.15.90:0/1160179344 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a65786f540 con 0x55a657baa240 2024-09-10T22:04:15.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e85c00640 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bb3a50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5e74033000 tx=0x7f5e74009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5933cb57ca083189 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:15.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e67400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:15.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e67400640 1 -- 172.21.15.90:0/1160179344 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74035070 con 0x55a657baa240 2024-09-10T22:04:15.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e67400640 1 -- 172.21.15.90:0/1160179344 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f5e740037f0 con 0x55a657baa240 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.110+0000 7f5e8b010740 1 -- 172.21.15.90:0/1160179344 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a657aaea70 con 0x55a657baa240 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.111+0000 7f5e67400640 1 -- 172.21.15.90:0/1160179344 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e740039f0 con 0x55a657baa240 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.111+0000 7f5e8b010740 10 client.15717 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.111+0000 7f5e8b010740 1 -- 172.21.15.90:0/1160179344 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a657ac96f0 con 0x55a657baa240 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.111+0000 7f5e67400640 1 -- 172.21.15.90:0/1160179344 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55a657aaea70 con 0x55a657baa240 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.111+0000 7f5e67400640 1 -- 172.21.15.90:0/1160179344 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5e74043070 con 0x55a657baa240 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.111+0000 7f5e8b010740 10 client.15717 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.111+0000 7f5e8b010740 10 client.15717 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.111+0000 7f5e8b010740 20 client.15717 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.111+0000 7f5e8b010740 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55a657bc7880 0x55a657bc8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.112+0000 7f5e8b010740 4 client.15717 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.112+0000 7f5e8b010740 1 -- 172.21.15.90:0/1160179344 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a657b23ca0 con 0x55a657bc7880 2024-09-10T22:04:15.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.112+0000 7f5e85200640 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55a657bc7880 0x55a657bc8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.112+0000 7f5e85200640 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55a657bc7880 0x55a657bc8cf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdb95c9db1fd7611 server_cookie=cccbc950faf006f9 in_seq=0 out_seq=0 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.112+0000 7f5e67400640 10 client.15717 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.113+0000 7f5e67400640 1 -- 172.21.15.90:0/1160179344 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a657b23ca0 con 0x55a657bc7880 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.113+0000 7f5e67400640 10 client.15717 handle_command_reply: tid=0 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.113+0000 7f5e8b010740 1 client.15717 shutdown 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 -- 172.21.15.90:0/1160179344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 msgr2=0x55a657bb3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bb3a50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5e74033000 tx=0x7f5e74009260 comp rx=0 tx=0).stop 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 -- 172.21.15.90:0/1160179344 shutdown_connections 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55a657bc7880 0x55a657bc8cf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657bb44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baab60 0x55a657bb3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 --2- 172.21.15.90:0/1160179344 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bb3a50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 -- 172.21.15.90:0/1160179344 >> 172.21.15.90:0/1160179344 conn(0x55a657ba8020 msgr2=0x55a657badbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:15.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e67400640 0 client.15717 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:04:15.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 -- 172.21.15.90:0/1160179344 shutdown_connections 2024-09-10T22:04:15.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 1 -- 172.21.15.90:0/1160179344 wait complete. 2024-09-10T22:04:15.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.114+0000 7f5e8b010740 20 client.15717 trim_cache size 0 max 16384 2024-09-10T22:04:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e8b010740 1 Processor -- start 2024-09-10T22:04:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e8b010740 1 -- start start 2024-09-10T22:04:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baa240 0x55a657bc8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baab60 0x55a657bae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657baece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e8b010740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a657bc7030 con 0x55a657baab60 2024-09-10T22:04:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e8b010740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a657bc7700 con 0x55a657baa240 2024-09-10T22:04:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e8b010740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a657bc7dd0 con 0x55a657bb6af0 2024-09-10T22:04:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e85200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baab60 0x55a657bae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e85200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baab60 0x55a657bae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43954/0 (socket says 172.21.15.90:43954) 2024-09-10T22:04:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e85200640 1 -- 172.21.15.90:0/2634306153 learned_addr learned my addr 172.21.15.90:0/2634306153 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.136+0000 7f5e85c00640 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baa240 0x55a657bc8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.137+0000 7f5e86600640 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657baece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.137+0000 7f5e85200640 1 -- 172.21.15.90:0/2634306153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 msgr2=0x55a657baece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.137+0000 7f5e85200640 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657baece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.137+0000 7f5e85200640 1 -- 172.21.15.90:0/2634306153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baa240 msgr2=0x55a657bc8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.137+0000 7f5e85200640 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baa240 0x55a657bc8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.137+0000 7f5e85200640 1 -- 172.21.15.90:0/2634306153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a657ba8aa0 con 0x55a657baab60 2024-09-10T22:04:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.137+0000 7f5e85c00640 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baa240 0x55a657bc8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.137+0000 7f5e85200640 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baab60 0x55a657bae2b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0e3990 tx=0x7f5e6c0cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1716bd5bcf7929ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.137+0000 7f5e72000640 1 -- 172.21.15.90:0/2634306153 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0ac170 con 0x55a657baab60 2024-09-10T22:04:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.138+0000 7f5e72000640 1 -- 172.21.15.90:0/2634306153 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e6c0acd60 con 0x55a657baab60 2024-09-10T22:04:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.138+0000 7f5e72000640 1 -- 172.21.15.90:0/2634306153 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0b5bf0 con 0x55a657baab60 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.138+0000 7f5e8b010740 1 -- 172.21.15.90:0/2634306153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baab60 msgr2=0x55a657bae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.138+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baab60 0x55a657bae2b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0e3990 tx=0x7f5e6c0cf040 comp rx=0 tx=0).stop 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.138+0000 7f5e8b010740 1 -- 172.21.15.90:0/2634306153 shutdown_connections 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.138+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657bb6af0 0x55a657baece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.138+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baab60 0x55a657bae2b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.138+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2634306153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657baa240 0x55a657bc8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.138+0000 7f5e8b010740 1 -- 172.21.15.90:0/2634306153 >> 172.21.15.90:0/2634306153 conn(0x55a657ba8020 msgr2=0x55a657ba2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.139+0000 7f5e8b010740 1 -- 172.21.15.90:0/2634306153 shutdown_connections 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.139+0000 7f5e8b010740 1 -- 172.21.15.90:0/2634306153 wait complete. 2024-09-10T22:04:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.139+0000 7f5e8b010740 1 -- start start 2024-09-10T22:04:15.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.139+0000 7f5e8b010740 1 Processor -- start 2024-09-10T22:04:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bcafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657baab60 0x55a657bcb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e8b010740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657bb6af0 0x55a657bcba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e8b010740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a657bc8b70 con 0x55a657baa240 2024-09-10T22:04:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e8b010740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55a657bc84a0 con 0x55a657bb6af0 2024-09-10T22:04:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e8b010740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55a657bc9240 con 0x55a657baab60 2024-09-10T22:04:15.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e8b010740 10 client.0 _ll_register_callbacks cb 0x7f5e74031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e85c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bcafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e85c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bcafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43960/0 (socket says 172.21.15.90:43960) 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e85c00640 1 -- 172.21.15.90:0/2547484596 learned_addr learned my addr 172.21.15.90:0/2547484596 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.140+0000 7f5e85200640 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657baab60 0x55a657bcb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e86600640 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657bb6af0 0x55a657bcba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e85c00640 1 -- 172.21.15.90:0/2547484596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657baab60 msgr2=0x55a657bcb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e85c00640 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657baab60 0x55a657bcb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e85c00640 1 -- 172.21.15.90:0/2547484596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657bb6af0 msgr2=0x55a657bcba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e85c00640 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657bb6af0 0x55a657bcba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e85c00640 1 -- 172.21.15.90:0/2547484596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a657bd0890 con 0x55a657baa240 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e85c00640 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bcafd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009230 tx=0x7f5e74004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6fc08491d2c9d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e67400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:15.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e67400640 1 -- 172.21.15.90:0/2547484596 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74003d90 con 0x55a657baa240 2024-09-10T22:04:15.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.141+0000 7f5e67400640 1 -- 172.21.15.90:0/2547484596 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f5e7403c440 con 0x55a657baa240 2024-09-10T22:04:15.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e8b010740 1 -- 172.21.15.90:0/2547484596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e6c0acd60 con 0x55a657baa240 2024-09-10T22:04:15.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e67400640 1 -- 172.21.15.90:0/2547484596 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5e74003800 con 0x55a657baa240 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e8b010740 10 client.15723 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e8b010740 1 -- 172.21.15.90:0/2547484596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a657c346b0 con 0x55a657baa240 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e67400640 1 -- 172.21.15.90:0/2547484596 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e6c0acd60 con 0x55a657baa240 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e67400640 1 -- 172.21.15.90:0/2547484596 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5e74043040 con 0x55a657baa240 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e8b010740 10 client.15723 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e8b010740 10 client.15723 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e8b010740 20 client.15723 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55a657c392f0 0x55a657c38000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e8b010740 4 client.15723 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.142+0000 7f5e8b010740 1 -- 172.21.15.90:0/2547484596 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "status"}) -- 0x55a657b1f750 con 0x55a657c392f0 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.143+0000 7f5e85200640 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55a657c392f0 0x55a657c38000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e85200640 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55a657c392f0 0x55a657c38000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe3a5e82db66a91 server_cookie=b3c28ce084582b9 in_seq=0 out_seq=0 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e67400640 10 client.15723 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e67400640 1 -- 172.21.15.90:0/2547484596 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a657b1f750 con 0x55a657c392f0 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e67400640 10 client.15723 handle_command_reply: tid=0 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 client.15723 shutdown 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 -- 172.21.15.90:0/2547484596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 msgr2=0x55a657bcafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bcafd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009230 tx=0x7f5e74004550 comp rx=0 tx=0).stop 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 -- 172.21.15.90:0/2547484596 shutdown_connections 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55a657c392f0 0x55a657c38000 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55a657bb6af0 0x55a657bcba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55a657baab60 0x55a657bcb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 --2- 172.21.15.90:0/2547484596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a657baa240 0x55a657bcafd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 -- 172.21.15.90:0/2547484596 >> 172.21.15.90:0/2547484596 conn(0x55a657ba8020 msgr2=0x55a657baece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e67400640 0 client.15723 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:04:15.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 -- 172.21.15.90:0/2547484596 shutdown_connections 2024-09-10T22:04:15.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.144+0000 7f5e8b010740 1 -- 172.21.15.90:0/2547484596 wait complete. 2024-09-10T22:04:15.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e8b010740 20 client.15723 trim_cache size 0 max 16384 2024-09-10T22:04:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:04:15.149 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "815bde3e-6fbe-11ef-bcea-c7b262605968", 2024-09-10T22:04:15.149 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-09-10T22:04:15.149 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 34703, 2024-09-10T22:04:15.149 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-09-10T22:04:15.149 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-09-10T22:04:15.150 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-09-10T22:04:15.150 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 218.65008885200001, 2024-09-10T22:04:15.150 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 30, 2024-09-10T22:04:15.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 90, 2024-09-10T22:04:15.150 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-09-10T22:04:15.150 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 231.66192395799999 2024-09-10T22:04:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:04:15.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e88600640 1 -- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5e680b5450 msgr2=0x7f5e680b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e88600640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5e680b5450 0x7f5e680b7910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5e740057c0 tx=0x7f5e740058d0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e88600640 1 -- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e8010a9d0 msgr2=0x7f5e8019b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e88600640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e8010a9d0 0x7f5e8019b840 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00c920 tx=0x7f5e6c00cde0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e88600640 1 -- 172.21.15.90:0/1716117012 shutdown_connections 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e88600640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5e680b5450 0x7f5e680b7910 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e88600640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5e8010b350 0x7f5e801a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e88600640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e8010a9d0 0x7f5e8019b840 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.145+0000 7f5e88600640 1 --2- 172.21.15.90:0/1716117012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5e80104fc0 0x7f5e8019b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.146+0000 7f5e88600640 1 -- 172.21.15.90:0/1716117012 >> 172.21.15.90:0/1716117012 conn(0x7f5e801009f0 msgr2=0x7f5e80101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.146+0000 7f5e88600640 1 -- 172.21.15.90:0/1716117012 shutdown_connections 2024-09-10T22:04:15.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:15.146+0000 7f5e88600640 1 -- 172.21.15.90:0/1716117012 wait complete. 2024-09-10T22:04:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:14 smithi154 ceph-mon[34495]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-10T22:04:15.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:14 smithi171 ceph-mon[34138]: pgmap v428: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-09-10T22:04:15.997 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-10T22:04:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:16 smithi090 ceph-mon[30350]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-10T22:04:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:16 smithi154 ceph-mon[34495]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-10T22:04:17.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:16 smithi171 ceph-mon[34138]: pgmap v429: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-10T22:04:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:18 smithi090 ceph-mon[30350]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T22:04:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:18 smithi154 ceph-mon[34495]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T22:04:19.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:18 smithi171 ceph-mon[34138]: pgmap v430: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 853 B/s wr, 8 op/s 2024-09-10T22:04:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:20 smithi090 ceph-mon[30350]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-10T22:04:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:20 smithi154 ceph-mon[34495]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-10T22:04:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:20 smithi171 ceph-mon[34138]: pgmap v431: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-10T22:04:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:22 smithi090 ceph-mon[30350]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T22:04:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:22 smithi154 ceph-mon[34495]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T22:04:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:22 smithi171 ceph-mon[34138]: pgmap v432: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T22:04:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:24 smithi090 ceph-mon[30350]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T22:04:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:24 smithi154 ceph-mon[34495]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T22:04:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:24 smithi171 ceph-mon[34138]: pgmap v433: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-10T22:04:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:26 smithi090 ceph-mon[30350]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T22:04:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:26 smithi154 ceph-mon[34495]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T22:04:27.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:26 smithi171 ceph-mon[34138]: pgmap v434: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T22:04:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:28 smithi090 ceph-mon[30350]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:04:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:28 smithi154 ceph-mon[34495]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:04:29.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:28 smithi171 ceph-mon[34138]: pgmap v435: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:04:30.999 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-09-10T22:04:31.000 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:04:31.060 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:30 smithi090 ceph-mon[30350]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:04:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:30 smithi154 ceph-mon[34495]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:04:31.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:30 smithi171 ceph-mon[34138]: pgmap v436: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:04:31.578 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:32.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.354+0000 7f9f16600640 1 -- 172.21.15.90:0/1945210677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f1010f200 msgr2=0x7f9f101115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.354+0000 7f9f16600640 1 --2- 172.21.15.90:0/1945210677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f1010f200 0x7f9f101115f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009ca0 tx=0x7f9f0802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.354+0000 7f9f16600640 1 -- 172.21.15.90:0/1945210677 shutdown_connections 2024-09-10T22:04:32.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.355+0000 7f9f16600640 1 --2- 172.21.15.90:0/1945210677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f1010f200 0x7f9f101115f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.355+0000 7f9f16600640 1 --2- 172.21.15.90:0/1945210677 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10075e00 0x7f9f1010ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.355+0000 7f9f16600640 1 --2- 172.21.15.90:0/1945210677 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f100754e0 0x7f9f100758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.355+0000 7f9f16600640 1 -- 172.21.15.90:0/1945210677 >> 172.21.15.90:0/1945210677 conn(0x7f9f100fe660 msgr2=0x7f9f10100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:32.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.355+0000 7f9f16600640 1 -- 172.21.15.90:0/1945210677 shutdown_connections 2024-09-10T22:04:32.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.355+0000 7f9f16600640 1 -- 172.21.15.90:0/1945210677 wait complete. 2024-09-10T22:04:32.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.356+0000 7f9f16600640 1 Processor -- start 2024-09-10T22:04:32.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.356+0000 7f9f16600640 1 -- start start 2024-09-10T22:04:32.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.357+0000 7f9f16600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f100754e0 0x7f9f10102970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:32.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.357+0000 7f9f16600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10075e00 0x7f9f10102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:32.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.357+0000 7f9f16600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f1010f200 0x7f9f101033f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:32.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.357+0000 7f9f16600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10114ce0 con 0x7f9f1010f200 2024-09-10T22:04:32.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.357+0000 7f9f16600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10114b60 con 0x7f9f100754e0 2024-09-10T22:04:32.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.357+0000 7f9f16600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10114e60 con 0x7f9f10075e00 2024-09-10T22:04:32.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.357+0000 7f9f0fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10075e00 0x7f9f10102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:32.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.357+0000 7f9f0fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10075e00 0x7f9f10102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:52002/0 (socket says 172.21.15.90:52002) 2024-09-10T22:04:32.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.357+0000 7f9f0fe00640 1 -- 172.21.15.90:0/2190745076 learned_addr learned my addr 172.21.15.90:0/2190745076 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:32.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.358+0000 7f9f15c00640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f1010f200 0x7f9f101033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:32.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.358+0000 7f9f15200640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f100754e0 0x7f9f10102970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:32.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.358+0000 7f9f0fe00640 1 -- 172.21.15.90:0/2190745076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f100754e0 msgr2=0x7f9f10102970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.358+0000 7f9f0fe00640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f100754e0 0x7f9f10102970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.358+0000 7f9f0fe00640 1 -- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f1010f200 msgr2=0x7f9f101033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.358+0000 7f9f0fe00640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f1010f200 0x7f9f101033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.358+0000 7f9f0fe00640 1 -- 172.21.15.90:0/2190745076 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f101ab680 con 0x7f9f10075e00 2024-09-10T22:04:32.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.359+0000 7f9f15200640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f100754e0 0x7f9f10102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:04:32.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.359+0000 7f9f0fe00640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10075e00 0x7f9f10102eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00e9f0 tx=0x7f9efc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:32.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.359+0000 7f9f0d600640 1 -- 172.21.15.90:0/2190745076 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc00cd80 con 0x7f9f10075e00 2024-09-10T22:04:32.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.359+0000 7f9f0d600640 1 -- 172.21.15.90:0/2190745076 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9efc0108c0 con 0x7f9f10075e00 2024-09-10T22:04:32.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.359+0000 7f9f0d600640 1 -- 172.21.15.90:0/2190745076 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0187c0 con 0x7f9f10075e00 2024-09-10T22:04:32.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.359+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f101ab970 con 0x7f9f10075e00 2024-09-10T22:04:32.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.359+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f101abf00 con 0x7f9f10075e00 2024-09-10T22:04:32.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.361+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f100758c0 con 0x7f9f10075e00 2024-09-10T22:04:32.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.365+0000 7f9f0d600640 1 -- 172.21.15.90:0/2190745076 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9efc010430 con 0x7f9f10075e00 2024-09-10T22:04:32.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.368+0000 7f9f0d600640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ef80b5580 0x7f9ef80b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:32.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.368+0000 7f9f0d600640 1 -- 172.21.15.90:0/2190745076 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f9efc0e6c20 con 0x7f9f10075e00 2024-09-10T22:04:32.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.368+0000 7f9f15200640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ef80b5580 0x7f9ef80b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:32.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.369+0000 7f9f0d600640 1 -- 172.21.15.90:0/2190745076 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9efc1172c0 con 0x7f9f10075e00 2024-09-10T22:04:32.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.369+0000 7f9f15200640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ef80b5580 0x7f9ef80b7a40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400baa0 tx=0x7f9f04005d90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:32.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.826+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f10103c70 con 0x7f9f10075e00 2024-09-10T22:04:32.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.827+0000 7f9f0d600640 1 -- 172.21.15.90:0/2190745076 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11986 (secure 0 0 0) 0x7f9efc0ad3d0 con 0x7f9f10075e00 2024-09-10T22:04:32.829 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:04:32.829 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"btime":"2024-09-10T22:03:47:815790+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:03:47.057424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:04:32.829 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 30 2024-09-10T22:04:32.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.830+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ef80b5580 msgr2=0x7f9ef80b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.830+0000 7f9f16600640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ef80b5580 0x7f9ef80b7a40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400baa0 tx=0x7f9f04005d90 comp rx=0 tx=0).stop 2024-09-10T22:04:32.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.830+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10075e00 msgr2=0x7f9f10102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:32.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.830+0000 7f9f16600640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10075e00 0x7f9f10102eb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9efc00e9f0 tx=0x7f9efc00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.830+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 shutdown_connections 2024-09-10T22:04:32.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.830+0000 7f9f16600640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ef80b5580 0x7f9ef80b7a40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.830+0000 7f9f16600640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f1010f200 0x7f9f101033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.830+0000 7f9f16600640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9f10075e00 0x7f9f10102eb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.830+0000 7f9f16600640 1 --2- 172.21.15.90:0/2190745076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9f100754e0 0x7f9f10102970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:32.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.831+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 >> 172.21.15.90:0/2190745076 conn(0x7f9f100fe660 msgr2=0x7f9f10106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:32.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.831+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 shutdown_connections 2024-09-10T22:04:32.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:32.831+0000 7f9f16600640 1 -- 172.21.15.90:0/2190745076 wait complete. 2024-09-10T22:04:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:32 smithi090 ceph-mon[30350]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:04:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:32 smithi154 ceph-mon[34495]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:04:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:32 smithi171 ceph-mon[34138]: pgmap v437: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:04:33.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:33 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2190745076' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:33.867 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-09-10T22:04:33.867 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-09-10T22:04:34.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:33 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2190745076' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:34.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:33 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2190745076' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:34.304 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:34.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:34 smithi090 ceph-mon[30350]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:04:35.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.988+0000 7f8fdee00640 1 -- 172.21.15.90:0/3980365871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fd810b1a0 msgr2=0x7f8fd81119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:35.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.988+0000 7f8fdee00640 1 --2- 172.21.15.90:0/3980365871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fd810b1a0 0x7f8fd81119a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009ca0 tx=0x7f8fd002a8b0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.989+0000 7f8fdee00640 1 -- 172.21.15.90:0/3980365871 shutdown_connections 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.989+0000 7f8fdee00640 1 --2- 172.21.15.90:0/3980365871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fd810b1a0 0x7f8fd81119a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.989+0000 7f8fdee00640 1 --2- 172.21.15.90:0/3980365871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd810a820 0x7f8fd810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.989+0000 7f8fdee00640 1 --2- 172.21.15.90:0/3980365871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd8104e10 0x7f8fd81051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.989+0000 7f8fdee00640 1 -- 172.21.15.90:0/3980365871 >> 172.21.15.90:0/3980365871 conn(0x7f8fd8100840 msgr2=0x7f8fd8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.990+0000 7f8fdee00640 1 -- 172.21.15.90:0/3980365871 shutdown_connections 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.990+0000 7f8fdee00640 1 -- 172.21.15.90:0/3980365871 wait complete. 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.991+0000 7f8fdee00640 1 Processor -- start 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.991+0000 7f8fdee00640 1 -- start start 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdee00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8104e10 0x7f8fd819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdee00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd810a820 0x7f8fd819b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fd810b1a0 0x7f8fd81a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8114d60 con 0x7f8fd810b1a0 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdee00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8114be0 con 0x7f8fd8104e10 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdee00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8114ee0 con 0x7f8fd810a820 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdd000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd810a820 0x7f8fd819b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdd000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd810a820 0x7f8fd819b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:52020/0 (socket says 172.21.15.90:52020) 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdd000640 1 -- 172.21.15.90:0/690166697 learned_addr learned my addr 172.21.15.90:0/690166697 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fde400640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fd810b1a0 0x7f8fd81a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.992+0000 7f8fdda00640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8104e10 0x7f8fd819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.993+0000 7f8fde400640 1 -- 172.21.15.90:0/690166697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd810a820 msgr2=0x7f8fd819b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.993+0000 7f8fde400640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd810a820 0x7f8fd819b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.993+0000 7f8fde400640 1 -- 172.21.15.90:0/690166697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8104e10 msgr2=0x7f8fd819b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.993+0000 7f8fde400640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8104e10 0x7f8fd819b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.993+0000 7f8fde400640 1 -- 172.21.15.90:0/690166697 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd81a1140 con 0x7f8fd810b1a0 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.993+0000 7f8fde400640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fd810b1a0 0x7f8fd81a0970 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8fd002add0 tx=0x7f8fd0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.994+0000 7f8fd6000640 1 -- 172.21.15.90:0/690166697 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0004240 con 0x7f8fd810b1a0 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.994+0000 7f8fd6000640 1 -- 172.21.15.90:0/690166697 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fd0040070 con 0x7f8fd810b1a0 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.994+0000 7f8fd6000640 1 -- 172.21.15.90:0/690166697 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0045a00 con 0x7f8fd810b1a0 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.994+0000 7f8fdee00640 1 -- 172.21.15.90:0/690166697 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd81a13d0 con 0x7f8fd810b1a0 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.994+0000 7f8fdee00640 1 -- 172.21.15.90:0/690166697 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd81a7af0 con 0x7f8fd810b1a0 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.997+0000 7f8fe2fed740 1 Processor -- start 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:34.999+0000 7f8fd6000640 1 -- 172.21.15.90:0/690166697 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8fd0045ba0 con 0x7f8fd810b1a0 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.000+0000 7f8fe2fed740 1 -- start start 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.000+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.000+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b32660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.000+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45b34f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.000+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556b45a9bca0 con 0x556b45b26c10 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.000+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556b45a8b0d0 con 0x556b45b262f0 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.000+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556b45a8eee0 con 0x556b45b32ba0 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdd000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdd000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b266d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:37096/0 (socket says 172.21.15.90:37096) 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdd000640 1 -- 172.21.15.90:0/3777046723 learned_addr learned my addr 172.21.15.90:0/3777046723 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3777046723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45b34f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdda00640 1 --2- 172.21.15.90:0/3777046723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b32660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdd000640 1 -- 172.21.15.90:0/3777046723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 msgr2=0x556b45b34f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3777046723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45b34f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdd000640 1 -- 172.21.15.90:0/3777046723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 msgr2=0x556b45b32660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3777046723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b32660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.001+0000 7f8fdd000640 1 -- 172.21.15.90:0/3777046723 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b45a3d400 con 0x556b45b262f0 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.002+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3777046723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b266d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4012830 tx=0x7f8fc400b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7801e90baad109d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.002+0000 7f8fd6000640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8fc00b5580 0x7f8fc00b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.002+0000 7f8fd6000640 1 -- 172.21.15.90:0/690166697 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f8fd01075e0 con 0x7f8fd810b1a0 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.002+0000 7f8fcaa00640 1 -- 172.21.15.90:0/3777046723 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4012db0 con 0x556b45b262f0 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.002+0000 7f8fdda00640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8fc00b5580 0x7f8fc00b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.002+0000 7f8fcaa00640 1 -- 172.21.15.90:0/3777046723 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fc4010c00 con 0x556b45b262f0 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.002+0000 7f8fcaa00640 1 -- 172.21.15.90:0/3777046723 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc401e9e0 con 0x556b45b262f0 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.003+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3777046723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 msgr2=0x556b45b266d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.003+0000 7f8fdda00640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8fc00b5580 0x7f8fc00b7a40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x556b45b25000 tx=0x7f8fcc00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.003+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3777046723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b266d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4012830 tx=0x7f8fc400b830 comp rx=0 tx=0).stop 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.004+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3777046723 shutdown_connections 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.004+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3777046723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45b34f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.004+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3777046723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b32660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.004+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3777046723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b266d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.004+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3777046723 >> 172.21.15.90:0/3777046723 conn(0x556b45b24120 msgr2=0x556b45b252c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.004+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3777046723 shutdown_connections 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.004+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3777046723 wait complete. 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.004+0000 7f8fe2fed740 1 -- start start 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.004+0000 7f8fe2fed740 1 Processor -- start 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b2fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b30110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45b30650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556b458ec7a0 con 0x556b45b26c10 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556b45a30a10 con 0x556b45b262f0 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556b45b3de50 con 0x556b45b32ba0 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fe2fed740 10 client.0 _ll_register_callbacks cb 0x556b45a40be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fdd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b30110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fdd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b30110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42900/0 (socket says 172.21.15.90:42900) 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.005+0000 7f8fdd000640 1 -- 172.21.15.90:0/3841573719 learned_addr learned my addr 172.21.15.90:0/3841573719 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.006+0000 7f8fdda00640 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b2fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.006+0000 7f8fdd000640 1 -- 172.21.15.90:0/3841573719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 msgr2=0x556b45b30650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.006+0000 7f8fde400640 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45b30650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.006+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45b30650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.006+0000 7f8fdd000640 1 -- 172.21.15.90:0/3841573719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 msgr2=0x556b45b2fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.006+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b2fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.006+0000 7f8fdd000640 1 -- 172.21.15.90:0/3841573719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b457f8b90 con 0x556b45b26c10 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.007+0000 7f8fdda00640 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b2fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.007+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b30110 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400bfd0 tx=0x7f8fc401e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d57986aae90767 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.007+0000 7f8fbf400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:35.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.007+0000 7f8fbf400640 1 -- 172.21.15.90:0/3841573719 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4004030 con 0x556b45b26c10 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.007+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3841573719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc4010c00 con 0x556b45b26c10 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.007+0000 7f8fbf400640 1 -- 172.21.15.90:0/3841573719 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f8fc40041d0 con 0x556b45b26c10 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fbf400640 1 -- 172.21.15.90:0/3841573719 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8fc40043d0 con 0x556b45b26c10 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fe2fed740 10 client.15741 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3841573719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b45b49a40 con 0x556b45b26c10 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fbf400640 1 -- 172.21.15.90:0/3841573719 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fc4010c00 con 0x556b45b26c10 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fbf400640 1 -- 172.21.15.90:0/3841573719 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8fc40036e0 con 0x556b45b26c10 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fe2fed740 10 client.15741 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fe2fed740 10 client.15741 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fe2fed740 20 client.15741 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556b45b4ad90 0x556b45b4d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fe2fed740 4 client.15741 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:04:35.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.008+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3841573719 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b45a9dbf0 con 0x556b45b4ad90 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.009+0000 7f8fdda00640 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556b45b4ad90 0x556b45b4d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.010+0000 7f8fbf400640 10 client.15741 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.010+0000 7f8fdda00640 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556b45b4ad90 0x556b45b4d180 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959cacac6ba8945f server_cookie=653b292a892c55de in_seq=0 out_seq=0 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.011+0000 7f8fbf400640 1 -- 172.21.15.90:0/3841573719 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b45a9dbf0 con 0x556b45b4ad90 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.011+0000 7f8fbf400640 10 client.15741 handle_command_reply: tid=0 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.011+0000 7f8fe2fed740 1 client.15741 shutdown 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.011+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3841573719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 msgr2=0x556b45b30110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b30110 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400bfd0 tx=0x7f8fc401e6f0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3841573719 shutdown_connections 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556b45b4ad90 0x556b45b4d180 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45b30650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b30110 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3841573719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b2fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3841573719 >> 172.21.15.90:0/3841573719 conn(0x556b45b24120 msgr2=0x556b45b29cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:35.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fbf400640 0 client.15741 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:04:35.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3841573719 shutdown_connections 2024-09-10T22:04:35.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3841573719 wait complete. 2024-09-10T22:04:35.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.012+0000 7f8fe2fed740 20 client.15741 trim_cache size 0 max 16384 2024-09-10T22:04:35.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.046+0000 7f8fe2fed740 1 Processor -- start 2024-09-10T22:04:35.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.046+0000 7f8fe2fed740 1 -- start start 2024-09-10T22:04:35.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.047+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b262f0 0x556b45b31bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.047+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b26c10 0x556b45b2f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.047+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b32ba0 0x556b45b2cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.047+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81ad7e0 con 0x556b45b32ba0 2024-09-10T22:04:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.047+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81af320 con 0x556b45b26c10 2024-09-10T22:04:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.047+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81adeb0 con 0x556b45b262f0 2024-09-10T22:04:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.047+0000 7f8fdda00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b262f0 0x556b45b31bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.047+0000 7f8fdda00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b262f0 0x556b45b31bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:52032/0 (socket says 172.21.15.90:52032) 2024-09-10T22:04:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.047+0000 7f8fdda00640 1 -- 172.21.15.90:0/1414216242 learned_addr learned my addr 172.21.15.90:0/1414216242 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.048+0000 7f8fdd000640 1 --2- 172.21.15.90:0/1414216242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b26c10 0x556b45b2f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.048+0000 7f8fdda00640 1 -- 172.21.15.90:0/1414216242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b26c10 msgr2=0x556b45b2f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.048+0000 7f8fdda00640 1 --2- 172.21.15.90:0/1414216242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b26c10 0x556b45b2f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.048+0000 7f8fdda00640 1 -- 172.21.15.90:0/1414216242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b32ba0 msgr2=0x556b45b2cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.048+0000 7f8fdda00640 1 --2- 172.21.15.90:0/1414216242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b32ba0 0x556b45b2cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.048+0000 7f8fdda00640 1 -- 172.21.15.90:0/1414216242 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b45b1fa70 con 0x556b45b262f0 2024-09-10T22:04:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.048+0000 7f8fdda00640 1 --2- 172.21.15.90:0/1414216242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b262f0 0x556b45b31bc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc013fd0 tx=0x7f8fcc015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a571253eb8e0e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.049+0000 7f8fca000640 1 -- 172.21.15.90:0/1414216242 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc010da0 con 0x556b45b262f0 2024-09-10T22:04:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.049+0000 7f8fca000640 1 -- 172.21.15.90:0/1414216242 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fcc0054b0 con 0x556b45b262f0 2024-09-10T22:04:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.049+0000 7f8fca000640 1 -- 172.21.15.90:0/1414216242 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc030480 con 0x556b45b262f0 2024-09-10T22:04:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.050+0000 7f8fe2fed740 1 -- 172.21.15.90:0/1414216242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b262f0 msgr2=0x556b45b31bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:35.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.050+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/1414216242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b262f0 0x556b45b31bc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc013fd0 tx=0x7f8fcc015000 comp rx=0 tx=0).stop 2024-09-10T22:04:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.050+0000 7f8fe2fed740 1 -- 172.21.15.90:0/1414216242 shutdown_connections 2024-09-10T22:04:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.050+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/1414216242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b32ba0 0x556b45b2cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.050+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/1414216242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b26c10 0x556b45b2f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.050+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/1414216242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b262f0 0x556b45b31bc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.050+0000 7f8fe2fed740 1 -- 172.21.15.90:0/1414216242 >> 172.21.15.90:0/1414216242 conn(0x556b45b24120 msgr2=0x556b45b303c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.050+0000 7f8fe2fed740 1 -- 172.21.15.90:0/1414216242 shutdown_connections 2024-09-10T22:04:35.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.050+0000 7f8fe2fed740 1 -- 172.21.15.90:0/1414216242 wait complete. 2024-09-10T22:04:35.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.051+0000 7f8fe2fed740 1 -- start start 2024-09-10T22:04:35.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.051+0000 7f8fe2fed740 1 Processor -- start 2024-09-10T22:04:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b3fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b40050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fe2fed740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45bb0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81aec50 con 0x556b45b26c10 2024-09-10T22:04:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81ae580 con 0x556b45b262f0 2024-09-10T22:04:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fe2fed740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81ab820 con 0x556b45b32ba0 2024-09-10T22:04:35.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fe2fed740 10 client.0 _ll_register_callbacks cb 0x7f8fc400bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fdd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b40050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fdd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b40050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42918/0 (socket says 172.21.15.90:42918) 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fdd000640 1 -- 172.21.15.90:0/3985255592 learned_addr learned my addr 172.21.15.90:0/3985255592 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fdda00640 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b3fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.052+0000 7f8fde400640 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45bb0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.053+0000 7f8fdd000640 1 -- 172.21.15.90:0/3985255592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 msgr2=0x556b45bb0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.053+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45bb0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.053+0000 7f8fdd000640 1 -- 172.21.15.90:0/3985255592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 msgr2=0x556b45b3fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.053+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b3fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.053+0000 7f8fdd000640 1 -- 172.21.15.90:0/3985255592 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b45bb1190 con 0x556b45b26c10 2024-09-10T22:04:35.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.053+0000 7f8fdda00640 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b3fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:35.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.053+0000 7f8fdd000640 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b40050 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400bfd0 tx=0x7f8fc4010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ec3d48afa6cbb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:35.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.053+0000 7f8fbf400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:04:35.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.054+0000 7f8fbf400640 1 -- 172.21.15.90:0/3985255592 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4039070 con 0x556b45b26c10 2024-09-10T22:04:35.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.054+0000 7f8fbf400640 1 -- 172.21.15.90:0/3985255592 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f8fc4003cf0 con 0x556b45b26c10 2024-09-10T22:04:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.054+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3985255592 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fcc0054b0 con 0x556b45b26c10 2024-09-10T22:04:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.055+0000 7f8fbf400640 1 -- 172.21.15.90:0/3985255592 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8fc400bb40 con 0x556b45b26c10 2024-09-10T22:04:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.055+0000 7f8fe2fed740 10 client.15744 fetch_fsmap learned FSMap version 30 2024-09-10T22:04:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.055+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3985255592 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556b45bb8540 con 0x556b45b26c10 2024-09-10T22:04:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.055+0000 7f8fbf400640 1 -- 172.21.15.90:0/3985255592 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8fcc0054b0 con 0x556b45b26c10 2024-09-10T22:04:35.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.055+0000 7f8fbf400640 1 -- 172.21.15.90:0/3985255592 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8fc403c020 con 0x556b45b26c10 2024-09-10T22:04:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.055+0000 7f8fe2fed740 10 client.15744 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:04:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.055+0000 7f8fe2fed740 10 client.15744 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:04:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.055+0000 7f8fe2fed740 20 client.15744 populate_metadata read hostname 'smithi090' 2024-09-10T22:04:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.055+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556b45bb8d30 0x556b45bbb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.056+0000 7f8fe2fed740 4 client.15744 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-09-10T22:04:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.056+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3985255592 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x556b45a996a0 con 0x556b45bb8d30 2024-09-10T22:04:35.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.056+0000 7f8fdda00640 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556b45bb8d30 0x556b45bbb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:35.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.057+0000 7f8fdda00640 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556b45bb8d30 0x556b45bbb150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87495c1fa585a79 server_cookie=be05d37be8ea65f3 in_seq=0 out_seq=0 2024-09-10T22:04:35.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.057+0000 7f8fbf400640 10 client.15744 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:04:35.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.067+0000 7f8fbf400640 1 -- 172.21.15.90:0/3985255592 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556b45a996a0 con 0x556b45bb8d30 2024-09-10T22:04:35.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.067+0000 7f8fbf400640 10 client.15744 handle_command_reply: tid=0 2024-09-10T22:04:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fe2fed740 1 client.15744 shutdown 2024-09-10T22:04:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3985255592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 msgr2=0x556b45b40050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b40050 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400bfd0 tx=0x7f8fc4010bb0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3985255592 shutdown_connections 2024-09-10T22:04:35.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556b45bb8d30 0x556b45bbb150 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556b45b32ba0 0x556b45bb0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556b45b26c10 0x556b45b40050 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fe2fed740 1 --2- 172.21.15.90:0/3985255592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556b45b262f0 0x556b45b3fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3985255592 >> 172.21.15.90:0/3985255592 conn(0x556b45b24120 msgr2=0x556b45b2fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:35.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.068+0000 7f8fbf400640 0 client.15744 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:04:35.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.069+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3985255592 shutdown_connections 2024-09-10T22:04:35.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.069+0000 7f8fe2fed740 1 -- 172.21.15.90:0/3985255592 wait complete. 2024-09-10T22:04:35.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.069+0000 7f8fe2fed740 20 client.15744 trim_cache size 0 max 16384 2024-09-10T22:04:35.071 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 30, 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 2, 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "51ab65e2": { 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 2, 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:04:35.072 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 145.2, 2024-09-10T22:04:35.073 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:04:35.073 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:04:35.073 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:04:35.073 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:04:35.073 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:04:35.073 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:04:35.073 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:04:35.073 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:04:35.073 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:04:35.074 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:04:35.074 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:04:35.074 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 -- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8fc00b5580 msgr2=0x7f8fc00b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8fc00b5580 0x7f8fc00b7a40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x556b45b25000 tx=0x7f8fcc00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 -- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fd810b1a0 msgr2=0x7f8fd81a0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fd810b1a0 0x7f8fd81a0970 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8fd002add0 tx=0x7f8fd0003ba0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 -- 172.21.15.90:0/690166697 shutdown_connections 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8fc00b5580 0x7f8fc00b7a40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fd810b1a0 0x7f8fd81a0970 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8fd810a820 0x7f8fd819b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 --2- 172.21.15.90:0/690166697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8fd8104e10 0x7f8fd819b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.070+0000 7f8fdee00640 1 -- 172.21.15.90:0/690166697 >> 172.21.15.90:0/690166697 conn(0x7f8fd8100840 msgr2=0x7f8fd8103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:35.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.071+0000 7f8fdee00640 1 -- 172.21.15.90:0/690166697 shutdown_connections 2024-09-10T22:04:35.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:35.071+0000 7f8fdee00640 1 -- 172.21.15.90:0/690166697 wait complete. 2024-09-10T22:04:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:34 smithi154 ceph-mon[34495]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:04:35.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:34 smithi171 ceph-mon[34138]: pgmap v438: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:04:36.115 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '51ab65e2', quiesce took 5.1 seconds. Will release after: 20.1 2024-09-10T22:04:37.050 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:36 smithi171 ceph-mon[34138]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:04:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:36 smithi090 ceph-mon[30350]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:04:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:36 smithi154 ceph-mon[34495]: pgmap v439: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:04:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:38 smithi090 ceph-mon[30350]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:04:39.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:38 smithi154 ceph-mon[34495]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:04:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:38 smithi171 ceph-mon[34138]: pgmap v440: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:04:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:40 smithi090 ceph-mon[30350]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:04:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:40 smithi154 ceph-mon[34495]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:04:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:40 smithi171 ceph-mon[34138]: pgmap v441: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:04:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:42 smithi090 ceph-mon[30350]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:04:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:42 smithi154 ceph-mon[34495]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:04:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:42 smithi171 ceph-mon[34138]: pgmap v442: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:04:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:44 smithi090 ceph-mon[30350]: pgmap v443: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T22:04:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:44 smithi154 ceph-mon[34495]: pgmap v443: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T22:04:45.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:44 smithi171 ceph-mon[34138]: pgmap v443: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T22:04:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:46 smithi090 ceph-mon[30350]: pgmap v444: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T22:04:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:46 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:04:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:46 smithi154 ceph-mon[34495]: pgmap v444: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T22:04:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:46 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:04:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:46 smithi171 ceph-mon[34138]: pgmap v444: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T22:04:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:46 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:04:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:04:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:04:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:04:48.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:04:48.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:04:48.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:04:48.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:04:48.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:04:48.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:04:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:48 smithi090 ceph-mon[30350]: pgmap v445: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T22:04:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:48 smithi154 ceph-mon[34495]: pgmap v445: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T22:04:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:48 smithi171 ceph-mon[34138]: pgmap v445: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T22:04:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:50 smithi090 ceph-mon[30350]: pgmap v446: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:04:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:50 smithi154 ceph-mon[34495]: pgmap v446: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:04:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:50 smithi171 ceph-mon[34138]: pgmap v446: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T22:04:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:51 smithi154 ceph-mon[34495]: pgmap v447: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:04:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:51 smithi171 ceph-mon[34138]: pgmap v447: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:04:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:51 smithi090 ceph-mon[30350]: pgmap v447: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:04:54.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:54 smithi171 ceph-mon[34138]: pgmap v448: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:04:55.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:54 smithi090 ceph-mon[30350]: pgmap v448: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:04:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:54 smithi154 ceph-mon[34495]: pgmap v448: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T22:04:56.217 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:04:56.662 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:04:56.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:56 smithi090 ceph-mon[30350]: pgmap v449: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:04:56.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:56 smithi171 ceph-mon[34138]: pgmap v449: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:04:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:56 smithi154 ceph-mon[34495]: pgmap v449: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T22:04:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.323+0000 7f7b7b400640 1 -- 172.21.15.90:0/1386089491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b7410b350 msgr2=0x7f7b74111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.323+0000 7f7b7b400640 1 --2- 172.21.15.90:0/1386089491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b7410b350 0x7f7b74111b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009ca0 tx=0x7f7b6c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.323+0000 7f7b7b400640 1 -- 172.21.15.90:0/1386089491 shutdown_connections 2024-09-10T22:04:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.323+0000 7f7b7b400640 1 --2- 172.21.15.90:0/1386089491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b7410b350 0x7f7b74111b50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.323+0000 7f7b7b400640 1 --2- 172.21.15.90:0/1386089491 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b7410a9d0 0x7f7b7410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.323+0000 7f7b7b400640 1 --2- 172.21.15.90:0/1386089491 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b74104fc0 0x7f7b741053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.323+0000 7f7b7b400640 1 -- 172.21.15.90:0/1386089491 >> 172.21.15.90:0/1386089491 conn(0x7f7b741009f0 msgr2=0x7f7b74102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.323+0000 7f7b7b400640 1 -- 172.21.15.90:0/1386089491 shutdown_connections 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.323+0000 7f7b7b400640 1 -- 172.21.15.90:0/1386089491 wait complete. 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.325+0000 7f7b7b400640 1 Processor -- start 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.325+0000 7f7b7b400640 1 -- start start 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.325+0000 7f7b7b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b74104fc0 0x7f7b7419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.325+0000 7f7b7b400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b7410a9d0 0x7f7b7419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.325+0000 7f7b7b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b7410b350 0x7f7b741a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.325+0000 7f7b7b400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74114ee0 con 0x7f7b7410b350 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.325+0000 7f7b7b400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74114d60 con 0x7f7b74104fc0 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.325+0000 7f7b7b400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74115060 con 0x7f7b7410a9d0 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b74104fc0 0x7f7b7419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b79400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b7410b350 0x7f7b741a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b74104fc0 0x7f7b7419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:51698/0 (socket says 172.21.15.90:51698) 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b78a00640 1 -- 172.21.15.90:0/703962506 learned_addr learned my addr 172.21.15.90:0/703962506 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b73e00640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b7410a9d0 0x7f7b7419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b79400640 1 -- 172.21.15.90:0/703962506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b7410a9d0 msgr2=0x7f7b7419b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b79400640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b7410a9d0 0x7f7b7419b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b79400640 1 -- 172.21.15.90:0/703962506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b74104fc0 msgr2=0x7f7b7419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b79400640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b74104fc0 0x7f7b7419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b79400640 1 -- 172.21.15.90:0/703962506 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b741a1480 con 0x7f7b7410b350 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.326+0000 7f7b78a00640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b74104fc0 0x7f7b7419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.327+0000 7f7b79400640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b7410b350 0x7f7b741a0cb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c031040 tx=0x7f7b6c032990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.327+0000 7f7b71600640 1 -- 172.21.15.90:0/703962506 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c032d80 con 0x7f7b7410b350 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.327+0000 7f7b71600640 1 -- 172.21.15.90:0/703962506 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b6c0044d0 con 0x7f7b7410b350 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.327+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b741a1710 con 0x7f7b7410b350 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.327+0000 7f7b71600640 1 -- 172.21.15.90:0/703962506 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c030b90 con 0x7f7b7410b350 2024-09-10T22:04:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.328+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a7e30 con 0x7f7b7410b350 2024-09-10T22:04:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.329+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b741053a0 con 0x7f7b7410b350 2024-09-10T22:04:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.331+0000 7f7b71600640 1 -- 172.21.15.90:0/703962506 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7b6c004040 con 0x7f7b7410b350 2024-09-10T22:04:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.332+0000 7f7b71600640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7b5c0b54d0 0x7f7b5c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:04:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.333+0000 7f7b71600640 1 -- 172.21.15.90:0/703962506 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f7b6c107440 con 0x7f7b7410b350 2024-09-10T22:04:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.333+0000 7f7b78a00640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7b5c0b54d0 0x7f7b5c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:04:57.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.334+0000 7f7b78a00640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7b5c0b54d0 0x7f7b5c0b7990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009e10 tx=0x7f7b680091c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:04:57.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.337+0000 7f7b71600640 1 -- 172.21.15.90:0/703962506 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7b6c0cdbf0 con 0x7f7b7410b350 2024-09-10T22:04:57.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.569+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b7402e5c0 con 0x7f7b7410b350 2024-09-10T22:04:57.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.570+0000 7f7b71600640 1 -- 172.21.15.90:0/703962506 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11986 (secure 0 0 0) 0x7f7b6c0d63a0 con 0x7f7b7410b350 2024-09-10T22:04:57.571 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:04:57.571 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"btime":"2024-09-10T22:03:47:815790+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:03:47.057424+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:04:57.571 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 30 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7b5c0b54d0 msgr2=0x7f7b5c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7b5c0b54d0 0x7f7b5c0b7990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009e10 tx=0x7f7b680091c0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b7410b350 msgr2=0x7f7b741a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b7410b350 0x7f7b741a0cb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c031040 tx=0x7f7b6c032990 comp rx=0 tx=0).stop 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 shutdown_connections 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7b5c0b54d0 0x7f7b5c0b7990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b7410b350 0x7f7b741a0cb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7b7410a9d0 0x7f7b7419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 --2- 172.21.15.90:0/703962506 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7b74104fc0 0x7f7b7419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 >> 172.21.15.90:0/703962506 conn(0x7f7b741009f0 msgr2=0x7f7b74102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 shutdown_connections 2024-09-10T22:04:57.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:04:57.573+0000 7f7b7b400640 1 -- 172.21.15.90:0/703962506 wait complete. 2024-09-10T22:04:57.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:57 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/703962506' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:57.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:57 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/703962506' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:57 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/703962506' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:04:58.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:04:58 smithi171 ceph-mon[34138]: pgmap v450: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:04:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:04:58 smithi090 ceph-mon[30350]: pgmap v450: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:04:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:04:58 smithi154 ceph-mon[34495]: pgmap v450: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:05:00.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:00 smithi171 ceph-mon[34138]: pgmap v451: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:05:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:00 smithi090 ceph-mon[30350]: pgmap v451: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:05:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:00 smithi154 ceph-mon[34495]: pgmap v451: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-10T22:05:02.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:02 smithi171 ceph-mon[34138]: pgmap v452: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:05:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:02 smithi090 ceph-mon[30350]: pgmap v452: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:05:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:02 smithi154 ceph-mon[34495]: pgmap v452: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T22:05:03.479 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 51ab65e2 --release --await' 2024-09-10T22:05:03.479 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 51ab65e2 --release --await 2024-09-10T22:05:03.867 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:05:04.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.572+0000 7f7e7b200640 1 -- 172.21.15.90:0/34860793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 msgr2=0x7f7e7c111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.572+0000 7f7e7b200640 1 --2- 172.21.15.90:0/34860793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 0x7f7e7c111610 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7e74009ca0 tx=0x7f7e7402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.572+0000 7f7e7b200640 1 -- 172.21.15.90:0/34860793 shutdown_connections 2024-09-10T22:05:04.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.572+0000 7f7e7b200640 1 --2- 172.21.15.90:0/34860793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 0x7f7e7c111610 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.572+0000 7f7e7b200640 1 --2- 172.21.15.90:0/34860793 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e7c075de0 0x7f7e7c10ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.572+0000 7f7e7b200640 1 --2- 172.21.15.90:0/34860793 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e7c0754c0 0x7f7e7c0758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.572+0000 7f7e7b200640 1 -- 172.21.15.90:0/34860793 >> 172.21.15.90:0/34860793 conn(0x7f7e7c0fe660 msgr2=0x7f7e7c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.572+0000 7f7e7b200640 1 -- 172.21.15.90:0/34860793 shutdown_connections 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.572+0000 7f7e7b200640 1 -- 172.21.15.90:0/34860793 wait complete. 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.573+0000 7f7e7b200640 1 Processor -- start 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.573+0000 7f7e7b200640 1 -- start start 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.573+0000 7f7e7b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e7c0754c0 0x7f7e7c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e7c075de0 0x7f7e7c19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 0x7f7e7c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c114d40 con 0x7f7e7c10f220 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c114bc0 con 0x7f7e7c075de0 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c114ec0 con 0x7f7e7c0754c0 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 0x7f7e7c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e79e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e7c0754c0 0x7f7e7c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 0x7f7e7c1a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39870/0 (socket says 172.21.15.90:39870) 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7a800640 1 -- 172.21.15.90:0/4184828763 learned_addr learned my addr 172.21.15.90:0/4184828763 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e79400640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e7c075de0 0x7f7e7c19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7a800640 1 -- 172.21.15.90:0/4184828763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e7c0754c0 msgr2=0x7f7e7c19b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7a800640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e7c0754c0 0x7f7e7c19b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7a800640 1 -- 172.21.15.90:0/4184828763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e7c075de0 msgr2=0x7f7e7c19b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7a800640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e7c075de0 0x7f7e7c19b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.574+0000 7f7e7a800640 1 -- 172.21.15.90:0/4184828763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c1a11e0 con 0x7f7e7c10f220 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.575+0000 7f7e7a800640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 0x7f7e7c1a0a10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7e74031040 tx=0x7f7e74032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.575+0000 7f7e72a00640 1 -- 172.21.15.90:0/4184828763 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74032eb0 con 0x7f7e7c10f220 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.575+0000 7f7e72a00640 1 -- 172.21.15.90:0/4184828763 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e740044d0 con 0x7f7e7c10f220 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.575+0000 7f7e7b200640 1 -- 172.21.15.90:0/4184828763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7c1a1470 con 0x7f7e7c10f220 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.575+0000 7f7e72a00640 1 -- 172.21.15.90:0/4184828763 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74030bf0 con 0x7f7e7c10f220 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.575+0000 7f7e7b200640 1 -- 172.21.15.90:0/4184828763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e7c1a7b90 con 0x7f7e7c10f220 2024-09-10T22:05:04.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.579+0000 7f7e83a77740 1 Processor -- start 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.579+0000 7f7e72a00640 1 -- 172.21.15.90:0/4184828763 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7e74004680 con 0x7f7e7c10f220 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e83a77740 1 -- start start 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b41d0 0x55c9144b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b4af0 0x55c9144c0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144c0a80 0x55c9144c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e83a77740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c91442b290 con 0x55c9144c0a80 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e83a77740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c91441a6c0 con 0x55c9144b41d0 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e83a77740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c91441e4d0 con 0x55c9144b4af0 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e79400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144c0a80 0x55c9144c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e79400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144c0a80 0x55c9144c2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39878/0 (socket says 172.21.15.90:39878) 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.580+0000 7f7e79400640 1 -- 172.21.15.90:0/249592927 learned_addr learned my addr 172.21.15.90:0/249592927 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.581+0000 7f7e79e00640 1 --2- 172.21.15.90:0/249592927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b4af0 0x55c9144c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.581+0000 7f7e79400640 1 --2- 172.21.15.90:0/249592927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b41d0 0x55c9144b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.581+0000 7f7e79e00640 1 -- 172.21.15.90:0/249592927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b41d0 msgr2=0x55c9144b45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.581+0000 7f7e79e00640 1 --2- 172.21.15.90:0/249592927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b41d0 0x55c9144b45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.581+0000 7f7e79e00640 1 -- 172.21.15.90:0/249592927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144c0a80 msgr2=0x55c9144c2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.581+0000 7f7e79e00640 1 --2- 172.21.15.90:0/249592927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144c0a80 0x55c9144c2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.581+0000 7f7e79e00640 1 -- 172.21.15.90:0/249592927 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9143a7460 con 0x55c9144b4af0 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.581+0000 7f7e72a00640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7e600b5500 0x7f7e600b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.581+0000 7f7e79e00640 1 --2- 172.21.15.90:0/249592927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b4af0 0x55c9144c0540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0099d0 tx=0x7f7e6c009e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33011de7d0a114ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.582+0000 7f7e72a00640 1 -- 172.21.15.90:0/4184828763 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f7e741076c0 con 0x7f7e7c10f220 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.582+0000 7f7e79400640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7e600b5500 0x7f7e600b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.582+0000 7f7e67400640 1 -- 172.21.15.90:0/249592927 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c017070 con 0x55c9144b4af0 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.582+0000 7f7e67400640 1 -- 172.21.15.90:0/249592927 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e6c013470 con 0x55c9144b4af0 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.582+0000 7f7e67400640 1 -- 172.21.15.90:0/249592927 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c012630 con 0x55c9144b4af0 2024-09-10T22:05:04.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 -- 172.21.15.90:0/249592927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b4af0 msgr2=0x55c9144c0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 --2- 172.21.15.90:0/249592927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b4af0 0x55c9144c0540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0099d0 tx=0x7f7e6c009e90 comp rx=0 tx=0).stop 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e79400640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7e600b5500 0x7f7e600b79c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55c9144b4180 tx=0x7f7e6800a760 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 -- 172.21.15.90:0/249592927 shutdown_connections 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 --2- 172.21.15.90:0/249592927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144c0a80 0x55c9144c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 --2- 172.21.15.90:0/249592927 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b4af0 0x55c9144c0540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 --2- 172.21.15.90:0/249592927 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b41d0 0x55c9144b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 -- 172.21.15.90:0/249592927 >> 172.21.15.90:0/249592927 conn(0x55c9144b2000 msgr2=0x55c9144b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 -- 172.21.15.90:0/249592927 shutdown_connections 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 -- 172.21.15.90:0/249592927 wait complete. 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.583+0000 7f7e83a77740 1 -- start start 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e83a77740 1 Processor -- start 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b41d0 0x55c9144c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b4af0 0x55c9144bd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144c0a80 0x55c9144be0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e83a77740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c91427bbb0 con 0x55c9144b4af0 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e83a77740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c9143b56c0 con 0x55c9144c0a80 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e83a77740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c9144c3730 con 0x55c9144b41d0 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e83a77740 10 client.0 _ll_register_callbacks cb 0x55c9143d0490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e79400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b4af0 0x55c9144bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e79400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b4af0 0x55c9144bd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39892/0 (socket says 172.21.15.90:39892) 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.584+0000 7f7e79400640 1 -- 172.21.15.90:0/3627865631 learned_addr learned my addr 172.21.15.90:0/3627865631 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e79e00640 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b41d0 0x55c9144c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e7a800640 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144c0a80 0x55c9144be0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e79400640 1 -- 172.21.15.90:0/3627865631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b41d0 msgr2=0x55c9144c04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e79400640 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b41d0 0x55c9144c04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e79400640 1 -- 172.21.15.90:0/3627865631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144c0a80 msgr2=0x55c9144be0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e79400640 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144c0a80 0x55c9144be0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e79400640 1 -- 172.21.15.90:0/3627865631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c914178b00 con 0x55c9144b4af0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e7a800640 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144c0a80 0x55c9144be0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e79400640 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b4af0 0x55c9144bd920 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800fe40 tx=0x7f7e68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2955480e9686ed63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.585+0000 7f7e5be00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.586+0000 7f7e5be00640 1 -- 172.21.15.90:0/3627865631 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6802f060 con 0x55c9144b4af0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.586+0000 7f7e5be00640 1 -- 172.21.15.90:0/3627865631 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f7e6801a030 con 0x55c9144b4af0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.586+0000 7f7e83a77740 1 -- 172.21.15.90:0/3627865631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e6c013470 con 0x55c9144b4af0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.586+0000 7f7e5be00640 1 -- 172.21.15.90:0/3627865631 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e68036050 con 0x55c9144b4af0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.586+0000 7f7e83a77740 10 client.15762 fetch_fsmap learned FSMap version 30 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.586+0000 7f7e83a77740 1 -- 172.21.15.90:0/3627865631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c9141ef960 con 0x55c9144b4af0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.586+0000 7f7e5be00640 1 -- 172.21.15.90:0/3627865631 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e6c013470 con 0x55c9144b4af0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.586+0000 7f7e5be00640 1 -- 172.21.15.90:0/3627865631 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7e6802a410 con 0x55c9144b4af0 2024-09-10T22:05:04.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.587+0000 7f7e83a77740 10 client.15762 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.587+0000 7f7e83a77740 10 client.15762 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.587+0000 7f7e83a77740 20 client.15762 populate_metadata read hostname 'smithi090' 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.587+0000 7f7e83a77740 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c9144d0590 0x55c9144d2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.587+0000 7f7e83a77740 4 client.15762 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.587+0000 7f7e83a77740 1 -- 172.21.15.90:0/3627865631 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c91442d1e0 con 0x55c9144d0590 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.587+0000 7f7e79e00640 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c9144d0590 0x55c9144d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.588+0000 7f7e79e00640 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c9144d0590 0x55c9144d2980 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f8c1c7cf1476c0 server_cookie=bd1f5309b28eee13 in_seq=0 out_seq=0 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.588+0000 7f7e5be00640 10 client.15762 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e5be00640 1 -- 172.21.15.90:0/3627865631 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c91442d1e0 con 0x55c9144d0590 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e5be00640 10 client.15762 handle_command_reply: tid=0 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 client.15762 shutdown 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 -- 172.21.15.90:0/3627865631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b4af0 msgr2=0x55c9144bd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b4af0 0x55c9144bd920 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800fe40 tx=0x7f7e68004730 comp rx=0 tx=0).stop 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 -- 172.21.15.90:0/3627865631 shutdown_connections 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c9144d0590 0x55c9144d2980 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144c0a80 0x55c9144be0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b4af0 0x55c9144bd920 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 --2- 172.21.15.90:0/3627865631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144b41d0 0x55c9144c04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 -- 172.21.15.90:0/3627865631 >> 172.21.15.90:0/3627865631 conn(0x55c9144b2000 msgr2=0x55c9144b7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e5be00640 0 client.15762 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 -- 172.21.15.90:0/3627865631 shutdown_connections 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.589+0000 7f7e83a77740 1 -- 172.21.15.90:0/3627865631 wait complete. 2024-09-10T22:05:04.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.590+0000 7f7e83a77740 20 client.15762 trim_cache size 0 max 16384 2024-09-10T22:05:04.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.607+0000 7f7e83a77740 1 Processor -- start 2024-09-10T22:05:04.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.607+0000 7f7e83a77740 1 -- start start 2024-09-10T22:05:04.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 0x55c9144b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 0x55c9144bad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e83a77740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c1b1c20 con 0x55c9144b41d0 2024-09-10T22:05:04.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e83a77740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c1b3760 con 0x55c9144b4af0 2024-09-10T22:05:04.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e83a77740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c1b22f0 con 0x55c9144c0a80 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e79e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e79e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39898/0 (socket says 172.21.15.90:39898) 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e79e00640 1 -- 172.21.15.90:0/4133430702 learned_addr learned my addr 172.21.15.90:0/4133430702 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e79400640 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 0x55c9144b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e79e00640 1 -- 172.21.15.90:0/4133430702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 msgr2=0x55c9144bad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.608+0000 7f7e7a800640 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 0x55c9144bad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.609+0000 7f7e79e00640 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 0x55c9144bad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.609+0000 7f7e79e00640 1 -- 172.21.15.90:0/4133430702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 msgr2=0x55c9144b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.609+0000 7f7e79e00640 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 0x55c9144b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.609+0000 7f7e79e00640 1 -- 172.21.15.90:0/4133430702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9144b2a80 con 0x55c9144b41d0 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.609+0000 7f7e7a800640 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 0x55c9144bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.609+0000 7f7e79e00640 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144b3110 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009690 tx=0x7f7e6c009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbb9ee2068d0535 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.609+0000 7f7e66a00640 1 -- 172.21.15.90:0/4133430702 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c017040 con 0x55c9144b41d0 2024-09-10T22:05:04.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.609+0000 7f7e66a00640 1 -- 172.21.15.90:0/4133430702 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e6c00ee40 con 0x55c9144b41d0 2024-09-10T22:05:04.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.609+0000 7f7e66a00640 1 -- 172.21.15.90:0/4133430702 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c01c950 con 0x55c9144b41d0 2024-09-10T22:05:04.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.610+0000 7f7e83a77740 1 -- 172.21.15.90:0/4133430702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 msgr2=0x55c9144b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.610+0000 7f7e83a77740 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144b3110 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009690 tx=0x7f7e6c009110 comp rx=0 tx=0).stop 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.610+0000 7f7e83a77740 1 -- 172.21.15.90:0/4133430702 shutdown_connections 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.610+0000 7f7e83a77740 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 0x55c9144bad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.610+0000 7f7e83a77740 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 0x55c9144b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.610+0000 7f7e83a77740 1 --2- 172.21.15.90:0/4133430702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144b3110 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.610+0000 7f7e83a77740 1 -- 172.21.15.90:0/4133430702 >> 172.21.15.90:0/4133430702 conn(0x55c9144b2000 msgr2=0x55c9144bc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.611+0000 7f7e83a77740 1 -- 172.21.15.90:0/4133430702 shutdown_connections 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.611+0000 7f7e83a77740 1 -- 172.21.15.90:0/4133430702 wait complete. 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.611+0000 7f7e83a77740 1 -- start start 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.611+0000 7f7e83a77740 1 Processor -- start 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 0x55c9144c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e83a77740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 0x55c9144c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e83a77740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c1b3090 con 0x55c9144b41d0 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e83a77740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c1b29c0 con 0x55c9144b4af0 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e83a77740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7c1afc00 con 0x55c9144c0a80 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e83a77740 10 client.0 _ll_register_callbacks cb 0x7f7e68011d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e79e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e79e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144c8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39902/0 (socket says 172.21.15.90:39902) 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e7a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 0x55c9144c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e79e00640 1 -- 172.21.15.90:0/94261290 learned_addr learned my addr 172.21.15.90:0/94261290 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.612+0000 7f7e79400640 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 0x55c9144c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.613+0000 7f7e79e00640 1 -- 172.21.15.90:0/94261290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 msgr2=0x55c9144c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.613+0000 7f7e79e00640 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 0x55c9144c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.613+0000 7f7e79e00640 1 -- 172.21.15.90:0/94261290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 msgr2=0x55c9144c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:04.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.613+0000 7f7e79e00640 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 0x55c9144c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.613+0000 7f7e79e00640 1 -- 172.21.15.90:0/94261290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9144c8530 con 0x55c9144b41d0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.613+0000 7f7e79e00640 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144c8f10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c000c00 tx=0x7f7e6c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308c05c6af6ef3f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.613+0000 7f7e5be00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.613+0000 7f7e5be00640 1 -- 172.21.15.90:0/94261290 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c034070 con 0x55c9144b41d0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e5be00640 1 -- 172.21.15.90:0/94261290 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f7e6c005dd0 con 0x55c9144b41d0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e83a77740 1 -- 172.21.15.90:0/94261290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c914428c90 con 0x55c9144b41d0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e5be00640 1 -- 172.21.15.90:0/94261290 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7e6c040050 con 0x55c9144b41d0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e83a77740 10 client.15768 fetch_fsmap learned FSMap version 30 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e83a77740 1 -- 172.21.15.90:0/94261290 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c91453e1f0 con 0x55c9144b41d0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e5be00640 1 -- 172.21.15.90:0/94261290 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c914428c90 con 0x55c9144b41d0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e5be00640 1 -- 172.21.15.90:0/94261290 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7e6c02c020 con 0x55c9144b41d0 2024-09-10T22:05:04.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e83a77740 10 client.15768 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T22:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e83a77740 10 client.15768 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.614+0000 7f7e83a77740 20 client.15768 populate_metadata read hostname 'smithi090' 2024-09-10T22:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.615+0000 7f7e83a77740 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c91453e600 0x55c914540a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.615+0000 7f7e83a77740 4 client.15768 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "51ab65e2", "release": true, "await": true}] 2024-09-10T22:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.615+0000 7f7e83a77740 1 -- 172.21.15.90:0/94261290 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "51ab65e2", "release": true, "await": true}) -- 0x55c9144260c0 con 0x55c91453e600 2024-09-10T22:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.615+0000 7f7e79400640 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c91453e600 0x55c914540a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.616+0000 7f7e79400640 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c91453e600 0x55c914540a20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d72a49a8d9343f server_cookie=943af54d0493714 in_seq=0 out_seq=0 2024-09-10T22:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.616+0000 7f7e5be00640 10 client.15768 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:05:04.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e5be00640 1 -- 172.21.15.90:0/94261290 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55c9144260c0 con 0x55c91453e600 2024-09-10T22:05:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e5be00640 10 client.15768 handle_command_reply: tid=0 2024-09-10T22:05:04.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e83a77740 1 client.15768 shutdown 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e83a77740 1 -- 172.21.15.90:0/94261290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 msgr2=0x55c9144c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e83a77740 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144c8f10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c000c00 tx=0x7f7e6c01c450 comp rx=0 tx=0).stop 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e83a77740 1 -- 172.21.15.90:0/94261290 shutdown_connections 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e83a77740 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c91453e600 0x55c914540a20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e83a77740 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c9144c0a80 0x55c9144c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e83a77740 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c9144b4af0 0x55c9144c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.618+0000 7f7e83a77740 1 --2- 172.21.15.90:0/94261290 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c9144b41d0 0x55c9144c8f10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.619+0000 7f7e83a77740 1 -- 172.21.15.90:0/94261290 >> 172.21.15.90:0/94261290 conn(0x55c9144b2000 msgr2=0x55c9144bc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.619+0000 7f7e5be00640 0 client.15768 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.619+0000 7f7e83a77740 1 -- 172.21.15.90:0/94261290 shutdown_connections 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.619+0000 7f7e83a77740 1 -- 172.21.15.90:0/94261290 wait complete. 2024-09-10T22:05:04.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.619+0000 7f7e83a77740 20 client.15768 trim_cache size 0 max 16384 2024-09-10T22:05:04.622 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:05:04.622 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 30, 2024-09-10T22:05:04.622 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:05:04.622 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 4, 2024-09-10T22:05:04.622 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:05:04.622 INFO:teuthology.orchestra.run.smithi090.stdout: "51ab65e2": { 2024-09-10T22:05:04.622 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 4, 2024-09-10T22:05:04.622 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:05:04.622 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 145.2, 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:04.623 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:04.624 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:04.624 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:04.624 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:05:04.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 -- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7e600b5500 msgr2=0x7f7e600b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7e600b5500 0x7f7e600b79c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55c9144b4180 tx=0x7f7e6800a760 comp rx=0 tx=0).stop 2024-09-10T22:05:04.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 -- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 msgr2=0x7f7e7c1a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 0x7f7e7c1a0a10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7e74031040 tx=0x7f7e74032bc0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 -- 172.21.15.90:0/4184828763 shutdown_connections 2024-09-10T22:05:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7e600b5500 0x7f7e600b79c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e7c10f220 0x7f7e7c1a0a10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7e7c075de0 0x7f7e7c19b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 --2- 172.21.15.90:0/4184828763 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7e7c0754c0 0x7f7e7c19b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 -- 172.21.15.90:0/4184828763 >> 172.21.15.90:0/4184828763 conn(0x7f7e7c0fe660 msgr2=0x7f7e7c106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 -- 172.21.15.90:0/4184828763 shutdown_connections 2024-09-10T22:05:04.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:04.620+0000 7f7e7b200640 1 -- 172.21.15.90:0/4184828763 wait complete. 2024-09-10T22:05:04.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:04 smithi090 ceph-mon[30350]: pgmap v453: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-10T22:05:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:04 smithi171 ceph-mon[34138]: pgmap v453: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-10T22:05:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:04 smithi154 ceph-mon[34495]: pgmap v453: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 24 op/s 2024-09-10T22:05:05.594 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '51ab65e2', total seconds elapsed: 34.6 2024-09-10T22:05:05.641 INFO:tasks.workunit.client.0.smithi090.stderr:Updating files: 62% (7522/12088) Updating files: 63% (7616/12088) Updating files: 64% (7737/12088) Updating files: 65% (7858/12088) Updating files: 66% (7979/12088) Updating files: 67% (8099/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-09-10T22:05:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:06 smithi090 ceph-mon[30350]: pgmap v454: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr:Note: switching to '8f219c476cb4f26fc5719c661c0323719a45cb30'. 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr: git switch -c 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-09-10T22:05:06.918 INFO:tasks.workunit.client.0.smithi090.stderr:Or undo this operation with: 2024-09-10T22:05:06.919 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-09-10T22:05:06.919 INFO:tasks.workunit.client.0.smithi090.stderr: git switch - 2024-09-10T22:05:06.919 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-09-10T22:05:06.919 INFO:tasks.workunit.client.0.smithi090.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T22:05:06.919 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-09-10T22:05:06.919 INFO:tasks.workunit.client.0.smithi090.stderr:HEAD is now at 8f219c476cb Merge branch 'mgr-vol-mod-disable' of https://github.com/rishabh-d-dave/ceph into wip-hemanth-testing-2024-09-10-0723 2024-09-10T22:05:06.930 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T22:05:06.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:06 smithi171 ceph-mon[34138]: pgmap v454: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-10T22:05:07.020 INFO:tasks.workunit.client.0.smithi090.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T22:05:07.024 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T22:05:07.025 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T22:05:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:06 smithi154 ceph-mon[34495]: pgmap v454: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-09-10T22:05:07.932 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T22:05:08.048 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T22:05:08.114 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T22:05:08.117 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T22:05:08.117 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T22:05:08.182 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T22:05:08.189 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T22:05:08.189 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T22:05:08.225 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-10T22:05:08.227 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-10T22:05:08.227 DEBUG:teuthology.orchestra.run.smithi090:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f219c476cb4f26fc5719c661c0323719a45cb30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-10T22:05:08.328 INFO:tasks.workunit.client.0.smithi090.stderr:+ dbench 1 2024-09-10T22:05:08.341 INFO:tasks.workunit.client.0.smithi090.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T22:05:08.341 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-09-10T22:05:08.341 INFO:tasks.workunit.client.0.smithi090.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T22:05:08.345 INFO:tasks.workunit.client.0.smithi090.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-10T22:05:08.445 INFO:tasks.workunit.client.0.smithi090.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-10T22:05:08.446 INFO:tasks.workunit.client.0.smithi090.stdout:releasing clients 2024-09-10T22:05:08.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:08 smithi171 ceph-mon[34138]: pgmap v455: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-10T22:05:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:08 smithi090 ceph-mon[30350]: pgmap v455: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-10T22:05:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:08 smithi154 ceph-mon[34495]: pgmap v455: 129 pgs: 129 active+clean; 1.4 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-10T22:05:09.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 682 21.18 MB/sec warmup 1 sec latency 19.329 ms 2024-09-10T22:05:10.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 1114 11.48 MB/sec warmup 2 sec latency 22.013 ms 2024-09-10T22:05:10.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:10 smithi171 ceph-mon[34138]: pgmap v456: 129 pgs: 129 active+clean; 6.7 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 472 KiB/s wr, 26 op/s 2024-09-10T22:05:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:10 smithi090 ceph-mon[30350]: pgmap v456: 129 pgs: 129 active+clean; 6.7 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 472 KiB/s wr, 26 op/s 2024-09-10T22:05:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:10 smithi154 ceph-mon[34495]: pgmap v456: 129 pgs: 129 active+clean; 6.7 MiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 472 KiB/s wr, 26 op/s 2024-09-10T22:05:11.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 1589 8.72 MB/sec warmup 3 sec latency 20.945 ms 2024-09-10T22:05:12.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 2060 6.66 MB/sec warmup 4 sec latency 21.515 ms 2024-09-10T22:05:12.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:12 smithi171 ceph-mon[34138]: pgmap v457: 129 pgs: 129 active+clean; 14 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 34 op/s 2024-09-10T22:05:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:12 smithi090 ceph-mon[30350]: pgmap v457: 129 pgs: 129 active+clean; 14 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 34 op/s 2024-09-10T22:05:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:12 smithi154 ceph-mon[34495]: pgmap v457: 129 pgs: 129 active+clean; 14 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 34 op/s 2024-09-10T22:05:13.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 2470 5.72 MB/sec warmup 5 sec latency 22.546 ms 2024-09-10T22:05:14.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 3170 5.69 MB/sec warmup 6 sec latency 20.073 ms 2024-09-10T22:05:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:14 smithi171 ceph-mon[34138]: pgmap v458: 129 pgs: 129 active+clean; 20 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 42 op/s 2024-09-10T22:05:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:14 smithi090 ceph-mon[30350]: pgmap v458: 129 pgs: 129 active+clean; 20 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 42 op/s 2024-09-10T22:05:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:14 smithi154 ceph-mon[34495]: pgmap v458: 129 pgs: 129 active+clean; 20 MiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 42 op/s 2024-09-10T22:05:15.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 3395 5.07 MB/sec warmup 7 sec latency 19.030 ms 2024-09-10T22:05:16.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 3870 4.97 MB/sec warmup 8 sec latency 17.740 ms 2024-09-10T22:05:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:16 smithi090 ceph-mon[30350]: pgmap v459: 129 pgs: 129 active+clean; 24 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.0 MiB/s wr, 44 op/s 2024-09-10T22:05:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:16 smithi154 ceph-mon[34495]: pgmap v459: 129 pgs: 129 active+clean; 24 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.0 MiB/s wr, 44 op/s 2024-09-10T22:05:17.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:16 smithi171 ceph-mon[34138]: pgmap v459: 129 pgs: 129 active+clean; 24 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 2.0 MiB/s wr, 44 op/s 2024-09-10T22:05:17.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 4155 4.45 MB/sec warmup 9 sec latency 17.578 ms 2024-09-10T22:05:18.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 4563 4.10 MB/sec warmup 10 sec latency 19.916 ms 2024-09-10T22:05:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:18 smithi090 ceph-mon[30350]: pgmap v460: 129 pgs: 129 active+clean; 24 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.0 MiB/s wr, 41 op/s 2024-09-10T22:05:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:18 smithi154 ceph-mon[34495]: pgmap v460: 129 pgs: 129 active+clean; 24 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.0 MiB/s wr, 41 op/s 2024-09-10T22:05:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:18 smithi171 ceph-mon[34138]: pgmap v460: 129 pgs: 129 active+clean; 24 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.0 MiB/s wr, 41 op/s 2024-09-10T22:05:19.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 5032 4.10 MB/sec warmup 11 sec latency 22.710 ms 2024-09-10T22:05:20.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 5463 3.79 MB/sec warmup 12 sec latency 18.891 ms 2024-09-10T22:05:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:20 smithi090 ceph-mon[30350]: pgmap v461: 129 pgs: 129 active+clean; 25 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-09-10T22:05:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:20 smithi154 ceph-mon[34495]: pgmap v461: 129 pgs: 129 active+clean; 25 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-09-10T22:05:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:20 smithi171 ceph-mon[34138]: pgmap v461: 129 pgs: 129 active+clean; 25 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 2.4 MiB/s wr, 50 op/s 2024-09-10T22:05:21.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 5924 3.62 MB/sec warmup 13 sec latency 21.200 ms 2024-09-10T22:05:22.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 6483 3.76 MB/sec warmup 14 sec latency 16.095 ms 2024-09-10T22:05:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:22 smithi090 ceph-mon[30350]: pgmap v462: 129 pgs: 129 active+clean; 26 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-09-10T22:05:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:22 smithi154 ceph-mon[34495]: pgmap v462: 129 pgs: 129 active+clean; 26 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-09-10T22:05:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:22 smithi171 ceph-mon[34138]: pgmap v462: 129 pgs: 129 active+clean; 26 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 2.1 MiB/s wr, 68 op/s 2024-09-10T22:05:23.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 6881 3.63 MB/sec warmup 15 sec latency 20.093 ms 2024-09-10T22:05:24.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 7328 3.68 MB/sec warmup 16 sec latency 16.650 ms 2024-09-10T22:05:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:24 smithi090 ceph-mon[30350]: pgmap v463: 129 pgs: 129 active+clean; 26 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.7 MiB/s wr, 66 op/s 2024-09-10T22:05:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:24 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:05:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:24 smithi154 ceph-mon[34495]: pgmap v463: 129 pgs: 129 active+clean; 26 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.7 MiB/s wr, 66 op/s 2024-09-10T22:05:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:24 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:05:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:24 smithi171 ceph-mon[34138]: pgmap v463: 129 pgs: 129 active+clean; 26 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.7 MiB/s wr, 66 op/s 2024-09-10T22:05:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:24 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:05:25.446 INFO:tasks.workunit.client.0.smithi090.stdout: 1 7541 3.48 MB/sec warmup 17 sec latency 21.091 ms 2024-09-10T22:05:26.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 7823 3.30 MB/sec warmup 18 sec latency 20.455 ms 2024-09-10T22:05:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:26 smithi090 ceph-mon[30350]: pgmap v464: 129 pgs: 129 active+clean; 27 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-09-10T22:05:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:26 smithi154 ceph-mon[34495]: pgmap v464: 129 pgs: 129 active+clean; 27 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-09-10T22:05:27.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:26 smithi171 ceph-mon[34138]: pgmap v464: 129 pgs: 129 active+clean; 27 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-09-10T22:05:27.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 8186 3.22 MB/sec warmup 19 sec latency 23.476 ms 2024-09-10T22:05:28.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 8618 3.21 MB/sec warmup 20 sec latency 18.852 ms 2024-09-10T22:05:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:28 smithi090 ceph-mon[30350]: pgmap v465: 129 pgs: 129 active+clean; 27 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.0 MiB/s wr, 62 op/s 2024-09-10T22:05:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:28 smithi154 ceph-mon[34495]: pgmap v465: 129 pgs: 129 active+clean; 27 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.0 MiB/s wr, 62 op/s 2024-09-10T22:05:29.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:28 smithi171 ceph-mon[34138]: pgmap v465: 129 pgs: 129 active+clean; 27 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.0 MiB/s wr, 62 op/s 2024-09-10T22:05:29.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 9018 3.08 MB/sec warmup 21 sec latency 21.209 ms 2024-09-10T22:05:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:29 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:05:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:29 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:05:30.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:29 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:05:30.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 9463 3.01 MB/sec warmup 22 sec latency 21.148 ms 2024-09-10T22:05:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:30 smithi090 ceph-mon[30350]: pgmap v466: 129 pgs: 129 active+clean; 27 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-09-10T22:05:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:30 smithi154 ceph-mon[34495]: pgmap v466: 129 pgs: 129 active+clean; 27 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-09-10T22:05:31.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:30 smithi171 ceph-mon[34138]: pgmap v466: 129 pgs: 129 active+clean; 27 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.3 MiB/s wr, 78 op/s 2024-09-10T22:05:31.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 9821 3.05 MB/sec warmup 23 sec latency 20.445 ms 2024-09-10T22:05:32.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 10379 3.07 MB/sec warmup 24 sec latency 18.285 ms 2024-09-10T22:05:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:32 smithi090 ceph-mon[30350]: pgmap v467: 129 pgs: 129 active+clean; 27 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 984 KiB/s wr, 88 op/s 2024-09-10T22:05:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:32 smithi154 ceph-mon[34495]: pgmap v467: 129 pgs: 129 active+clean; 27 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 984 KiB/s wr, 88 op/s 2024-09-10T22:05:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:32 smithi171 ceph-mon[34138]: pgmap v467: 129 pgs: 129 active+clean; 27 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 984 KiB/s wr, 88 op/s 2024-09-10T22:05:33.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 10630 3.06 MB/sec warmup 25 sec latency 20.517 ms 2024-09-10T22:05:34.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 10922 3.00 MB/sec warmup 26 sec latency 24.480 ms 2024-09-10T22:05:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:34 smithi090 ceph-mon[30350]: pgmap v468: 129 pgs: 129 active+clean; 29 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-10T22:05:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:34 smithi154 ceph-mon[34495]: pgmap v468: 129 pgs: 129 active+clean; 29 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-10T22:05:35.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:34 smithi171 ceph-mon[34138]: pgmap v468: 129 pgs: 129 active+clean; 29 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-09-10T22:05:35.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 11098 2.90 MB/sec warmup 27 sec latency 17.449 ms 2024-09-10T22:05:36.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 11322 2.80 MB/sec warmup 28 sec latency 18.596 ms 2024-09-10T22:05:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:36 smithi090 ceph-mon[30350]: pgmap v469: 129 pgs: 129 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-10T22:05:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:36 smithi154 ceph-mon[34495]: pgmap v469: 129 pgs: 129 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-10T22:05:37.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:36 smithi171 ceph-mon[34138]: pgmap v469: 129 pgs: 129 active+clean; 29 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-10T22:05:37.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 11704 2.77 MB/sec warmup 29 sec latency 21.028 ms 2024-09-10T22:05:38.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 12085 2.78 MB/sec warmup 30 sec latency 19.702 ms 2024-09-10T22:05:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:38 smithi090 ceph-mon[30350]: pgmap v470: 129 pgs: 129 active+clean; 29 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 788 KiB/s wr, 73 op/s 2024-09-10T22:05:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:38 smithi154 ceph-mon[34495]: pgmap v470: 129 pgs: 129 active+clean; 29 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 788 KiB/s wr, 73 op/s 2024-09-10T22:05:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:38 smithi171 ceph-mon[34138]: pgmap v470: 129 pgs: 129 active+clean; 29 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 788 KiB/s wr, 73 op/s 2024-09-10T22:05:39.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 12494 2.70 MB/sec warmup 31 sec latency 26.082 ms 2024-09-10T22:05:40.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 12949 2.66 MB/sec warmup 32 sec latency 20.332 ms 2024-09-10T22:05:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:40 smithi090 ceph-mon[30350]: pgmap v471: 129 pgs: 129 active+clean; 30 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-09-10T22:05:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:40 smithi154 ceph-mon[34495]: pgmap v471: 129 pgs: 129 active+clean; 30 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-09-10T22:05:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:40 smithi171 ceph-mon[34138]: pgmap v471: 129 pgs: 129 active+clean; 30 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 115 op/s 2024-09-10T22:05:41.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 13124 2.61 MB/sec warmup 33 sec latency 328.828 ms 2024-09-10T22:05:42.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 13623 2.67 MB/sec warmup 34 sec latency 20.847 ms 2024-09-10T22:05:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:42 smithi090 ceph-mon[30350]: pgmap v472: 129 pgs: 129 active+clean; 30 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 882 KiB/s wr, 108 op/s 2024-09-10T22:05:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:42 smithi154 ceph-mon[34495]: pgmap v472: 129 pgs: 129 active+clean; 30 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 882 KiB/s wr, 108 op/s 2024-09-10T22:05:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:42 smithi171 ceph-mon[34138]: pgmap v472: 129 pgs: 129 active+clean; 30 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 882 KiB/s wr, 108 op/s 2024-09-10T22:05:43.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 13974 2.65 MB/sec warmup 35 sec latency 19.393 ms 2024-09-10T22:05:44.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 14335 2.67 MB/sec warmup 36 sec latency 19.555 ms 2024-09-10T22:05:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:44 smithi090 ceph-mon[30350]: pgmap v473: 129 pgs: 129 active+clean; 30 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-09-10T22:05:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:44 smithi154 ceph-mon[34495]: pgmap v473: 129 pgs: 129 active+clean; 30 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-09-10T22:05:45.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:44 smithi171 ceph-mon[34138]: pgmap v473: 129 pgs: 129 active+clean; 30 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-09-10T22:05:45.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 14558 2.63 MB/sec warmup 37 sec latency 20.466 ms 2024-09-10T22:05:46.447 INFO:tasks.workunit.client.0.smithi090.stdout: 1 14739 2.56 MB/sec warmup 38 sec latency 20.204 ms 2024-09-10T22:05:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:46 smithi090 ceph-mon[30350]: pgmap v474: 129 pgs: 129 active+clean; 31 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 953 KiB/s wr, 108 op/s 2024-09-10T22:05:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:46 smithi154 ceph-mon[34495]: pgmap v474: 129 pgs: 129 active+clean; 31 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 953 KiB/s wr, 108 op/s 2024-09-10T22:05:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:46 smithi171 ceph-mon[34138]: pgmap v474: 129 pgs: 129 active+clean; 31 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 953 KiB/s wr, 108 op/s 2024-09-10T22:05:47.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 15059 2.51 MB/sec warmup 39 sec latency 17.343 ms 2024-09-10T22:05:48.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:47 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:05:48.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:47 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:05:48.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:47 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:05:48.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 15377 2.49 MB/sec warmup 40 sec latency 20.041 ms 2024-09-10T22:05:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:48 smithi090 ceph-mon[30350]: pgmap v475: 129 pgs: 129 active+clean; 31 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 781 KiB/s wr, 94 op/s 2024-09-10T22:05:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:05:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:05:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:48 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:05:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:48 smithi154 ceph-mon[34495]: pgmap v475: 129 pgs: 129 active+clean; 31 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 781 KiB/s wr, 94 op/s 2024-09-10T22:05:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:05:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:05:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:48 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:05:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:48 smithi171 ceph-mon[34138]: pgmap v475: 129 pgs: 129 active+clean; 31 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 781 KiB/s wr, 94 op/s 2024-09-10T22:05:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:05:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:05:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:48 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:05:49.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 15789 2.50 MB/sec warmup 41 sec latency 20.407 ms 2024-09-10T22:05:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:49 smithi154 ceph-mon[34495]: pgmap v476: 129 pgs: 129 active+clean; 32 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-10T22:05:50.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:49 smithi171 ceph-mon[34138]: pgmap v476: 129 pgs: 129 active+clean; 32 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-10T22:05:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:49 smithi090 ceph-mon[30350]: pgmap v476: 129 pgs: 129 active+clean; 32 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-10T22:05:50.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 16230 2.45 MB/sec warmup 42 sec latency 22.499 ms 2024-09-10T22:05:51.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 16594 2.43 MB/sec warmup 43 sec latency 19.295 ms 2024-09-10T22:05:52.395 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T22:05:52.395 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.5 2024-09-10T22:05:52.395 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:05:52.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 17064 2.47 MB/sec warmup 44 sec latency 22.433 ms 2024-09-10T22:05:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:05:52.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:52 smithi171 ceph-mon[34138]: pgmap v477: 129 pgs: 129 active+clean; 32 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 781 KiB/s wr, 100 op/s 2024-09-10T22:05:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:52 smithi090 ceph-mon[30350]: pgmap v477: 129 pgs: 129 active+clean; 32 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 781 KiB/s wr, 100 op/s 2024-09-10T22:05:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:52 smithi154 ceph-mon[34495]: pgmap v477: 129 pgs: 129 active+clean; 32 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 781 KiB/s wr, 100 op/s 2024-09-10T22:05:53.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 17498 2.49 MB/sec warmup 45 sec latency 20.445 ms 2024-09-10T22:05:53.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.594+0000 7effa6200640 1 -- 172.21.15.90:0/49507406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa0108620 msgr2=0x7effa0108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.594+0000 7effa6200640 1 --2- 172.21.15.90:0/49507406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa0108620 0x7effa0108a60 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009900 tx=0x7eff8c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.595+0000 7effa6200640 1 -- 172.21.15.90:0/49507406 shutdown_connections 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.595+0000 7effa6200640 1 --2- 172.21.15.90:0/49507406 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effa0108fa0 0x7effa010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.595+0000 7effa6200640 1 --2- 172.21.15.90:0/49507406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa0108620 0x7effa0108a60 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.595+0000 7effa6200640 1 --2- 172.21.15.90:0/49507406 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effa0102c10 0x7effa0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.595+0000 7effa6200640 1 -- 172.21.15.90:0/49507406 >> 172.21.15.90:0/49507406 conn(0x7effa00fe660 msgr2=0x7effa0100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.595+0000 7effa6200640 1 -- 172.21.15.90:0/49507406 shutdown_connections 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.595+0000 7effa6200640 1 -- 172.21.15.90:0/49507406 wait complete. 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.596+0000 7effa6200640 1 Processor -- start 2024-09-10T22:05:53.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.596+0000 7effa6200640 1 -- start start 2024-09-10T22:05:53.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.596+0000 7effa6200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effa0102c10 0x7effa019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:53.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.596+0000 7effa6200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effa0108620 0x7effa019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:53.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.596+0000 7effa6200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa0108fa0 0x7effa01a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:53.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.596+0000 7effa6200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7effa0112b20 con 0x7effa0108fa0 2024-09-10T22:05:53.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.596+0000 7effa6200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7effa01129a0 con 0x7effa0102c10 2024-09-10T22:05:53.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.596+0000 7effa6200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7effa0112ca0 con 0x7effa0108620 2024-09-10T22:05:53.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa5800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa0108fa0 0x7effa01a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:53.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effa0102c10 0x7effa019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:53.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effa0102c10 0x7effa019b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:55628/0 (socket says 172.21.15.90:55628) 2024-09-10T22:05:53.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa4e00640 1 -- 172.21.15.90:0/550658018 learned_addr learned my addr 172.21.15.90:0/550658018 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:53.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7eff9fe00640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effa0108620 0x7effa019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:53.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa5800640 1 -- 172.21.15.90:0/550658018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effa0108620 msgr2=0x7effa019b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:53.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa5800640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effa0108620 0x7effa019b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa5800640 1 -- 172.21.15.90:0/550658018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effa0102c10 msgr2=0x7effa019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:53.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa5800640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effa0102c10 0x7effa019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa5800640 1 -- 172.21.15.90:0/550658018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa01a1230 con 0x7effa0108fa0 2024-09-10T22:05:53.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.597+0000 7effa4e00640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effa0102c10 0x7effa019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:05:53.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.598+0000 7effa5800640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa0108fa0 0x7effa01a0a60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7eff9800e940 tx=0x7eff9800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:53.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.598+0000 7eff9d600640 1 -- 172.21.15.90:0/550658018 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98004d20 con 0x7effa0108fa0 2024-09-10T22:05:53.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.598+0000 7eff9d600640 1 -- 172.21.15.90:0/550658018 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7eff98005020 con 0x7effa0108fa0 2024-09-10T22:05:53.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.598+0000 7eff9d600640 1 -- 172.21.15.90:0/550658018 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff980116d0 con 0x7effa0108fa0 2024-09-10T22:05:53.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.598+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa01a1520 con 0x7effa0108fa0 2024-09-10T22:05:53.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.598+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa01a7be0 con 0x7effa0108fa0 2024-09-10T22:05:53.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.599+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa0102ff0 con 0x7effa0108fa0 2024-09-10T22:05:53.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.602+0000 7eff9d600640 1 -- 172.21.15.90:0/550658018 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7eff9800f040 con 0x7effa0108fa0 2024-09-10T22:05:53.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.603+0000 7eff9d600640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7eff880b5580 0x7eff880b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:53.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.603+0000 7eff9d600640 1 -- 172.21.15.90:0/550658018 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7eff980e6420 con 0x7effa0108fa0 2024-09-10T22:05:53.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.603+0000 7effa4e00640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7eff880b5580 0x7eff880b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:53.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.604+0000 7effa4e00640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7eff880b5580 0x7eff880b7a40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7eff940095f0 tx=0x7eff94005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:53.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.606+0000 7eff9d600640 1 -- 172.21.15.90:0/550658018 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7eff98016030 con 0x7effa0108fa0 2024-09-10T22:05:53.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.869+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effa002e5c0 con 0x7effa0108fa0 2024-09-10T22:05:53.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.870+0000 7eff9d600640 1 -- 172.21.15.90:0/550658018 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11993 (secure 0 0 0) 0x7eff980ac600 con 0x7effa0108fa0 2024-09-10T22:05:53.873 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:05:53.873 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 32 2024-09-10T22:05:53.874 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":32,"btime":"2024-09-10T22:05:28:775491+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:05:28.494576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:05:53.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7eff880b5580 msgr2=0x7eff880b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:53.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7eff880b5580 0x7eff880b7a40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7eff940095f0 tx=0x7eff94005f90 comp rx=0 tx=0).stop 2024-09-10T22:05:53.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa0108fa0 msgr2=0x7effa01a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:53.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa0108fa0 0x7effa01a0a60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7eff9800e940 tx=0x7eff9800ee00 comp rx=0 tx=0).stop 2024-09-10T22:05:53.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 shutdown_connections 2024-09-10T22:05:53.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7eff880b5580 0x7eff880b7a40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7effa0108fa0 0x7effa01a0a60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7effa0108620 0x7effa019b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 --2- 172.21.15.90:0/550658018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7effa0102c10 0x7effa019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:53.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 >> 172.21.15.90:0/550658018 conn(0x7effa00fe660 msgr2=0x7effa0105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:53.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 shutdown_connections 2024-09-10T22:05:53.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:53.873+0000 7effa6200640 1 -- 172.21.15.90:0/550658018 wait complete. 2024-09-10T22:05:54.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 17813 2.50 MB/sec warmup 46 sec latency 18.464 ms 2024-09-10T22:05:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:54 smithi171 ceph-mon[34138]: pgmap v478: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 983 KiB/s wr, 122 op/s 2024-09-10T22:05:54.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:54 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/550658018' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:05:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:54 smithi090 ceph-mon[30350]: pgmap v478: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 983 KiB/s wr, 122 op/s 2024-09-10T22:05:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:54 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/550658018' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:05:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:54 smithi154 ceph-mon[34495]: pgmap v478: 129 pgs: 129 active+clean; 31 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 983 KiB/s wr, 122 op/s 2024-09-10T22:05:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:54 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/550658018' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:05:55.260 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0' 2024-09-10T22:05:55.260 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0 2024-09-10T22:05:55.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18045 2.47 MB/sec warmup 47 sec latency 18.135 ms 2024-09-10T22:05:55.691 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:05:56.449 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18218 2.43 MB/sec warmup 48 sec latency 16.496 ms 2024-09-10T22:05:56.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:56 smithi171 ceph-mon[34138]: pgmap v479: 129 pgs: 129 active+clean; 32 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 916 KiB/s wr, 103 op/s 2024-09-10T22:05:57.047 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:56 smithi090 ceph-mon[30350]: pgmap v479: 129 pgs: 129 active+clean; 32 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 916 KiB/s wr, 103 op/s 2024-09-10T22:05:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:56 smithi154 ceph-mon[34495]: pgmap v479: 129 pgs: 129 active+clean; 32 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 916 KiB/s wr, 103 op/s 2024-09-10T22:05:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.329+0000 7fe052e00640 1 -- 172.21.15.90:0/3837919392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe04c10a800 msgr2=0x7fe04c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.329+0000 7fe052e00640 1 --2- 172.21.15.90:0/3837919392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe04c10a800 0x7fe04c10ac40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe038009900 tx=0x7fe03802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.330+0000 7fe052e00640 1 -- 172.21.15.90:0/3837919392 shutdown_connections 2024-09-10T22:05:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.330+0000 7fe052e00640 1 --2- 172.21.15.90:0/3837919392 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe04c10b180 0x7fe04c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.330+0000 7fe052e00640 1 --2- 172.21.15.90:0/3837919392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe04c10a800 0x7fe04c10ac40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.330+0000 7fe052e00640 1 --2- 172.21.15.90:0/3837919392 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe04c104df0 0x7fe04c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.330+0000 7fe052e00640 1 -- 172.21.15.90:0/3837919392 >> 172.21.15.90:0/3837919392 conn(0x7fe04c100840 msgr2=0x7fe04c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.330+0000 7fe052e00640 1 -- 172.21.15.90:0/3837919392 shutdown_connections 2024-09-10T22:05:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.330+0000 7fe052e00640 1 -- 172.21.15.90:0/3837919392 wait complete. 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.331+0000 7fe052e00640 1 Processor -- start 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.331+0000 7fe052e00640 1 -- start start 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.331+0000 7fe052e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe04c104df0 0x7fe04c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe04c10a800 0x7fe04c19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe04c10b180 0x7fe04c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c114cd0 con 0x7fe04c10a800 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c114b50 con 0x7fe04c10b180 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c114e50 con 0x7fe04c104df0 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe051000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe04c10a800 0x7fe04c19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe04c10b180 0x7fe04c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe051000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe04c10a800 0x7fe04c19b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60720/0 (socket says 172.21.15.90:60720) 2024-09-10T22:05:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe051000640 1 -- 172.21.15.90:0/2766239062 learned_addr learned my addr 172.21.15.90:0/2766239062 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe051a00640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe04c104df0 0x7fe04c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052400640 1 -- 172.21.15.90:0/2766239062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe04c104df0 msgr2=0x7fe04c19b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052400640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe04c104df0 0x7fe04c19b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052400640 1 -- 172.21.15.90:0/2766239062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe04c10a800 msgr2=0x7fe04c19b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052400640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe04c10a800 0x7fe04c19b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.332+0000 7fe052400640 1 -- 172.21.15.90:0/2766239062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04c1a1170 con 0x7fe04c10b180 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.333+0000 7fe052400640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe04c10b180 0x7fe04c1a09a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe044009fd0 tx=0x7fe04400ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.333+0000 7fe04a000640 1 -- 172.21.15.90:0/2766239062 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04400cd60 con 0x7fe04c10b180 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.333+0000 7fe04a000640 1 -- 172.21.15.90:0/2766239062 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe044005020 con 0x7fe04c10b180 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.333+0000 7fe052e00640 1 -- 172.21.15.90:0/2766239062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04c1a1460 con 0x7fe04c10b180 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.333+0000 7fe04a000640 1 -- 172.21.15.90:0/2766239062 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe044010690 con 0x7fe04c10b180 2024-09-10T22:05:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.333+0000 7fe052e00640 1 -- 172.21.15.90:0/2766239062 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe04c1a7b20 con 0x7fe04c10b180 2024-09-10T22:05:57.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.337+0000 7fe057006740 1 Processor -- start 2024-09-10T22:05:57.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.337+0000 7fe04a000640 1 -- 172.21.15.90:0/2766239062 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe044004a40 con 0x7fe04c10b180 2024-09-10T22:05:57.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.337+0000 7fe057006740 1 -- start start 2024-09-10T22:05:57.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe057006740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 0x55b8061896d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe057006740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b806195660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe057006740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806195ba0 0x55b806197f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe057006740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b8060feca0 con 0x55b806195ba0 2024-09-10T22:05:57.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe057006740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b8060ee0d0 con 0x55b806189c10 2024-09-10T22:05:57.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe057006740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b8060f1ee0 con 0x55b8061892f0 2024-09-10T22:05:57.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806195ba0 0x55b806197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b806195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b806195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:55666/0 (socket says 172.21.15.90:55666) 2024-09-10T22:05:57.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051a00640 1 -- 172.21.15.90:0/1435796686 learned_addr learned my addr 172.21.15.90:0/1435796686 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051000640 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806195ba0 0x55b806197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60722/0 (socket says 172.21.15.90:60722) 2024-09-10T22:05:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051000640 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 0x55b8061896d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051a00640 1 -- 172.21.15.90:0/1435796686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 msgr2=0x55b8061896d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051a00640 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 0x55b8061896d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051a00640 1 -- 172.21.15.90:0/1435796686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806195ba0 msgr2=0x55b806197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051a00640 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806195ba0 0x55b806197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.338+0000 7fe051a00640 1 -- 172.21.15.90:0/1435796686 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8060a0400 con 0x55b806189c10 2024-09-10T22:05:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.339+0000 7fe051000640 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806195ba0 0x55b806197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:05:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.339+0000 7fe051a00640 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b806195660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe04000ad00 tx=0x7fe04000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4900383cb6d88877 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:57.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.339+0000 7fe03ea00640 1 -- 172.21.15.90:0/1435796686 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04001d070 con 0x55b806189c10 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.339+0000 7fe03ea00640 1 -- 172.21.15.90:0/1435796686 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe040018ad0 con 0x55b806189c10 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.339+0000 7fe03ea00640 1 -- 172.21.15.90:0/1435796686 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040010ad0 con 0x55b806189c10 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.339+0000 7fe04a000640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0340b5500 0x7fe0340b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.339+0000 7fe04a000640 1 -- 172.21.15.90:0/2766239062 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe0440e69a0 con 0x7fe04c10b180 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe051000640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0340b5500 0x7fe0340b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe057006740 1 -- 172.21.15.90:0/1435796686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 msgr2=0x55b806195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe057006740 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b806195660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe04000ad00 tx=0x7fe04000f790 comp rx=0 tx=0).stop 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe051000640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0340b5500 0x7fe0340b79c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe04c102ed0 tx=0x7fe038009560 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe057006740 1 -- 172.21.15.90:0/1435796686 shutdown_connections 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe057006740 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806195ba0 0x55b806197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe057006740 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b806195660 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe057006740 1 --2- 172.21.15.90:0/1435796686 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 0x55b8061896d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe057006740 1 -- 172.21.15.90:0/1435796686 >> 172.21.15.90:0/1435796686 conn(0x55b806187120 msgr2=0x55b8061882c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.340+0000 7fe057006740 1 -- 172.21.15.90:0/1435796686 shutdown_connections 2024-09-10T22:05:57.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.341+0000 7fe057006740 1 -- 172.21.15.90:0/1435796686 wait complete. 2024-09-10T22:05:57.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.341+0000 7fe057006740 1 -- start start 2024-09-10T22:05:57.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.341+0000 7fe057006740 1 Processor -- start 2024-09-10T22:05:57.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe057006740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe057006740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806189c10 0x55b806192840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe057006740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806195ba0 0x55b806192da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe057006740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b805f4f7a0 con 0x55b8061892f0 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe057006740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55b806093a10 con 0x55b806195ba0 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe057006740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55b806198820 con 0x55b806189c10 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe057006740 10 client.0 _ll_register_callbacks cb 0x55b8060a3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe051a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe052400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806195ba0 0x55b806192da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe051a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60728/0 (socket says 172.21.15.90:60728) 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.342+0000 7fe051a00640 1 -- 172.21.15.90:0/1521968403 learned_addr learned my addr 172.21.15.90:0/1521968403 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe051000640 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806189c10 0x55b806192840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe051a00640 1 -- 172.21.15.90:0/1521968403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806189c10 msgr2=0x55b806192840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe051a00640 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806189c10 0x55b806192840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe051a00640 1 -- 172.21.15.90:0/1521968403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806195ba0 msgr2=0x55b806192da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe051a00640 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806195ba0 0x55b806192da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe051a00640 1 -- 172.21.15.90:0/1521968403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b805e5bb90 con 0x55b8061892f0 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe052400640 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806195ba0 0x55b806192da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe051a00640 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061953f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe040010610 tx=0x7fe040018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a94663aff3f879 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe033400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe033400640 1 -- 172.21.15.90:0/1521968403 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04001d070 con 0x55b8061892f0 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe057006740 1 -- 172.21.15.90:0/1521968403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe040018ad0 con 0x55b8061892f0 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.343+0000 7fe033400640 1 -- 172.21.15.90:0/1521968403 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe040005240 con 0x55b8061892f0 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe033400640 1 -- 172.21.15.90:0/1521968403 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe040030050 con 0x55b8061892f0 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe057006740 10 client.15786 fetch_fsmap learned FSMap version 32 2024-09-10T22:05:57.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe057006740 1 -- 172.21.15.90:0/1521968403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b8061a4110 con 0x55b8061892f0 2024-09-10T22:05:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe033400640 1 -- 172.21.15.90:0/1521968403 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe040018ad0 con 0x55b8061892f0 2024-09-10T22:05:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe033400640 1 -- 172.21.15.90:0/1521968403 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe0400046c0 con 0x55b8061892f0 2024-09-10T22:05:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe057006740 10 client.15786 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:05:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe057006740 10 client.15786 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:05:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe057006740 20 client.15786 populate_metadata read hostname 'smithi090' 2024-09-10T22:05:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe057006740 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55b8061a5460 0x55b8061a7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe057006740 4 client.15786 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:05:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.344+0000 7fe057006740 1 -- 172.21.15.90:0/1521968403 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b806100bf0 con 0x55b8061a5460 2024-09-10T22:05:57.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.345+0000 7fe051000640 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55b8061a5460 0x55b8061a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.345+0000 7fe051000640 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55b8061a5460 0x55b8061a7850 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b110836a2207fd74 server_cookie=d0efc9cd9997e4b5 in_seq=0 out_seq=0 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.345+0000 7fe033400640 10 client.15786 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.346+0000 7fe033400640 1 -- 172.21.15.90:0/1521968403 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b806100bf0 con 0x55b8061a5460 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.346+0000 7fe033400640 10 client.15786 handle_command_reply: tid=0 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.346+0000 7fe057006740 1 client.15786 shutdown 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 -- 172.21.15.90:0/1521968403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 msgr2=0x55b8061953f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061953f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe040010610 tx=0x7fe040018670 comp rx=0 tx=0).stop 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 -- 172.21.15.90:0/1521968403 shutdown_connections 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55b8061a5460 0x55b8061a7850 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806195ba0 0x55b806192da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806189c10 0x55b806192840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 --2- 172.21.15.90:0/1521968403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061953f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 -- 172.21.15.90:0/1521968403 >> 172.21.15.90:0/1521968403 conn(0x55b806187120 msgr2=0x55b80618cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe033400640 0 client.15786 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:05:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 -- 172.21.15.90:0/1521968403 shutdown_connections 2024-09-10T22:05:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 1 -- 172.21.15.90:0/1521968403 wait complete. 2024-09-10T22:05:57.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.347+0000 7fe057006740 20 client.15786 trim_cache size 0 max 16384 2024-09-10T22:05:57.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.369+0000 7fe057006740 1 Processor -- start 2024-09-10T22:05:57.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.369+0000 7fe057006740 1 -- start start 2024-09-10T22:05:57.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.369+0000 7fe057006740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806187120 0x55b806186af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.369+0000 7fe057006740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061a7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.369+0000 7fe057006740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b8061910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.369+0000 7fe057006740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c1b1bb0 con 0x55b8061892f0 2024-09-10T22:05:57.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.369+0000 7fe057006740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c1b37b0 con 0x55b806189c10 2024-09-10T22:05:57.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.369+0000 7fe057006740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c1b2280 con 0x55b806187120 2024-09-10T22:05:57.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806187120 0x55b806186af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806187120 0x55b806186af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:50068/0 (socket says 172.21.15.90:50068) 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60744/0 (socket says 172.21.15.90:60744) 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051a00640 1 -- 172.21.15.90:0/1925475357 learned_addr learned my addr 172.21.15.90:0/1925475357 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe052400640 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b8061910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051a00640 1 -- 172.21.15.90:0/1925475357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 msgr2=0x55b8061910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051a00640 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b8061910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051a00640 1 -- 172.21.15.90:0/1925475357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 msgr2=0x55b8061a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051a00640 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051a00640 1 -- 172.21.15.90:0/1925475357 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b806187ba0 con 0x55b806187120 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe051000640 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.370+0000 7fe052400640 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b8061910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:05:57.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.371+0000 7fe051a00640 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806187120 0x55b806186af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe04000fc90 tx=0x7fe04000fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d75cae202de8644b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:57.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.371+0000 7fe03e000640 1 -- 172.21.15.90:0/1925475357 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040038020 con 0x55b806187120 2024-09-10T22:05:57.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.371+0000 7fe03e000640 1 -- 172.21.15.90:0/1925475357 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe040027070 con 0x55b806187120 2024-09-10T22:05:57.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.371+0000 7fe03e000640 1 -- 172.21.15.90:0/1925475357 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04001d070 con 0x55b806187120 2024-09-10T22:05:57.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 -- 172.21.15.90:0/1925475357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806187120 msgr2=0x55b806186af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:57.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806187120 0x55b806186af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe04000fc90 tx=0x7fe04000fdc0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 -- 172.21.15.90:0/1925475357 shutdown_connections 2024-09-10T22:05:57.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806189c10 0x55b8061910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8061892f0 0x55b8061a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 --2- 172.21.15.90:0/1925475357 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b806187120 0x55b806186af0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 -- 172.21.15.90:0/1925475357 >> 172.21.15.90:0/1925475357 conn(0x55b806192410 msgr2=0x55b806187e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:57.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 -- 172.21.15.90:0/1925475357 shutdown_connections 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 -- 172.21.15.90:0/1925475357 wait complete. 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 -- start start 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.372+0000 7fe057006740 1 Processor -- start 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe057006740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806187120 0x55b806199b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe057006740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 0x55b80619a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe057006740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806189c10 0x55b80619a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe057006740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c1b3020 con 0x55b806189c10 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe057006740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c1b2950 con 0x55b806187120 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe057006740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c1afb90 con 0x55b8061892f0 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe057006740 10 client.0 _ll_register_callbacks cb 0x7fe04001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe051a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806187120 0x55b806199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe051a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806187120 0x55b806199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:55702/0 (socket says 172.21.15.90:55702) 2024-09-10T22:05:57.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe051000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 0x55b80619a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe051a00640 1 -- 172.21.15.90:0/2036949245 learned_addr learned my addr 172.21.15.90:0/2036949245 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:05:57.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.373+0000 7fe052400640 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806189c10 0x55b80619a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe052400640 1 -- 172.21.15.90:0/2036949245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 msgr2=0x55b80619a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe052400640 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 0x55b80619a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe052400640 1 -- 172.21.15.90:0/2036949245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806187120 msgr2=0x55b806199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:05:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe052400640 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806187120 0x55b806199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe052400640 1 -- 172.21.15.90:0/2036949245 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b806211180 con 0x55b806189c10 2024-09-10T22:05:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe052400640 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806189c10 0x55b80619a580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe0440e3d20 tx=0x7fe0440ac8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd2868dadb75a68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:05:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe033400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:05:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe033400640 1 -- 172.21.15.90:0/2036949245 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0440b1530 con 0x55b806189c10 2024-09-10T22:05:57.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe033400640 1 -- 172.21.15.90:0/2036949245 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fe0440b5b50 con 0x55b806189c10 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.374+0000 7fe057006740 1 -- 172.21.15.90:0/2036949245 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe040027070 con 0x55b806189c10 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe033400640 1 -- 172.21.15.90:0/2036949245 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0440b5fe0 con 0x55b806189c10 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe057006740 10 client.15792 fetch_fsmap learned FSMap version 32 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe057006740 1 -- 172.21.15.90:0/2036949245 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b8062124c0 con 0x55b806189c10 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe033400640 1 -- 172.21.15.90:0/2036949245 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe040027070 con 0x55b806189c10 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe033400640 1 -- 172.21.15.90:0/2036949245 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe0440ab720 con 0x55b806189c10 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe057006740 10 client.15792 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe057006740 10 client.15792 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe057006740 20 client.15792 populate_metadata read hostname 'smithi090' 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe057006740 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55b806195ba0 0x55b806214e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe057006740 4 client.15792 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}] 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.375+0000 7fe057006740 1 -- 172.21.15.90:0/2036949245 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}) -- 0x55b8060fc6a0 con 0x55b806195ba0 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.376+0000 7fe051a00640 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55b806195ba0 0x55b806214e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:05:57.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.376+0000 7fe051a00640 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55b806195ba0 0x55b806214e00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8abccad2b65eb233 server_cookie=3da500f2412f2021 in_seq=0 out_seq=0 2024-09-10T22:05:57.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.376+0000 7fe033400640 10 client.15792 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe033400640 1 -- 172.21.15.90:0/2036949245 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b8060fc6a0 con 0x55b806195ba0 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe033400640 10 client.15792 handle_command_reply: tid=0 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe057006740 1 client.15792 shutdown 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe057006740 1 -- 172.21.15.90:0/2036949245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806189c10 msgr2=0x55b80619a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe057006740 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806189c10 0x55b80619a580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe0440e3d20 tx=0x7fe0440ac8e0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe057006740 1 -- 172.21.15.90:0/2036949245 shutdown_connections 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe057006740 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55b806195ba0 0x55b806214e00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe057006740 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b806189c10 0x55b80619a580 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe057006740 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55b8061892f0 0x55b80619a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe057006740 1 --2- 172.21.15.90:0/2036949245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55b806187120 0x55b806199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe057006740 1 -- 172.21.15.90:0/2036949245 >> 172.21.15.90:0/2036949245 conn(0x55b806192410 msgr2=0x55b806191040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.430+0000 7fe033400640 0 client.15792 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.431+0000 7fe057006740 1 -- 172.21.15.90:0/2036949245 shutdown_connections 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.431+0000 7fe057006740 1 -- 172.21.15.90:0/2036949245 wait complete. 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.431+0000 7fe057006740 20 client.15792 trim_cache size 0 max 16384 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 32, 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 6, 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:05:57.433 INFO:teuthology.orchestra.run.smithi090.stdout: "36789a83": { 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 6, 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 140.5, 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:05:57.434 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:05:57.435 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:57.435 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:57.435 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:57.435 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:57.435 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:05:57.435 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:05:57.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 -- 172.21.15.90:0/2766239062 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0340b5500 msgr2=0x7fe0340b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0340b5500 0x7fe0340b79c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe04c102ed0 tx=0x7fe038009560 comp rx=0 tx=0).stop 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 -- 172.21.15.90:0/2766239062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe04c10b180 msgr2=0x7fe04c1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe04c10b180 0x7fe04c1a09a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe044009fd0 tx=0x7fe04400ee50 comp rx=0 tx=0).stop 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 -- 172.21.15.90:0/2766239062 shutdown_connections 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0340b5500 0x7fe0340b79c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe04c10b180 0x7fe04c1a09a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe04c10a800 0x7fe04c19b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 --2- 172.21.15.90:0/2766239062 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe04c104df0 0x7fe04c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 -- 172.21.15.90:0/2766239062 >> 172.21.15.90:0/2766239062 conn(0x7fe04c100840 msgr2=0x7fe04c108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 -- 172.21.15.90:0/2766239062 shutdown_connections 2024-09-10T22:05:57.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:05:57.432+0000 7fe052e00640 1 -- 172.21.15.90:0/2766239062 wait complete. 2024-09-10T22:05:57.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.39 MB/sec warmup 49 sec latency 59.407 ms 2024-09-10T22:05:58.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.34 MB/sec warmup 50 sec latency 1059.548 ms 2024-09-10T22:05:58.890 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '36789a83', quiesce took 6.5 seconds. Will release after: 14.0 2024-09-10T22:05:58.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:05:58 smithi171 ceph-mon[34138]: pgmap v480: 129 pgs: 129 active+clean; 32 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 791 KiB/s wr, 93 op/s 2024-09-10T22:05:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:05:58 smithi090 ceph-mon[30350]: pgmap v480: 129 pgs: 129 active+clean; 32 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 791 KiB/s wr, 93 op/s 2024-09-10T22:05:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:05:58 smithi154 ceph-mon[34495]: pgmap v480: 129 pgs: 129 active+clean; 32 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 791 KiB/s wr, 93 op/s 2024-09-10T22:05:59.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.29 MB/sec warmup 51 sec latency 2059.665 ms 2024-09-10T22:06:00.448 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.25 MB/sec warmup 52 sec latency 3059.744 ms 2024-09-10T22:06:00.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:00 smithi171 ceph-mon[34138]: pgmap v481: 129 pgs: 129 active+clean; 33 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1022 KiB/s wr, 131 op/s 2024-09-10T22:06:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:00 smithi090 ceph-mon[30350]: pgmap v481: 129 pgs: 129 active+clean; 33 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1022 KiB/s wr, 131 op/s 2024-09-10T22:06:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:00 smithi154 ceph-mon[34495]: pgmap v481: 129 pgs: 129 active+clean; 33 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1022 KiB/s wr, 131 op/s 2024-09-10T22:06:01.449 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.21 MB/sec warmup 53 sec latency 4059.871 ms 2024-09-10T22:06:02.449 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.16 MB/sec warmup 54 sec latency 5060.009 ms 2024-09-10T22:06:02.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:02 smithi171 ceph-mon[34138]: pgmap v482: 129 pgs: 129 active+clean; 33 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 740 KiB/s wr, 96 op/s 2024-09-10T22:06:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:02 smithi090 ceph-mon[30350]: pgmap v482: 129 pgs: 129 active+clean; 33 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 740 KiB/s wr, 96 op/s 2024-09-10T22:06:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:02 smithi154 ceph-mon[34495]: pgmap v482: 129 pgs: 129 active+clean; 33 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 740 KiB/s wr, 96 op/s 2024-09-10T22:06:03.449 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.13 MB/sec warmup 55 sec latency 6060.147 ms 2024-09-10T22:06:04.449 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.09 MB/sec warmup 56 sec latency 7060.261 ms 2024-09-10T22:06:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:04 smithi171 ceph-mon[34138]: pgmap v483: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 751 KiB/s wr, 107 op/s 2024-09-10T22:06:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:04 smithi090 ceph-mon[30350]: pgmap v483: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 751 KiB/s wr, 107 op/s 2024-09-10T22:06:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:04 smithi154 ceph-mon[34495]: pgmap v483: 129 pgs: 129 active+clean; 32 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 751 KiB/s wr, 107 op/s 2024-09-10T22:06:05.449 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.05 MB/sec warmup 57 sec latency 8060.410 ms 2024-09-10T22:06:06.449 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 2.02 MB/sec warmup 58 sec latency 9060.527 ms 2024-09-10T22:06:07.009 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:06 smithi090 ceph-mon[30350]: pgmap v484: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 532 KiB/s wr, 80 op/s 2024-09-10T22:06:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:06 smithi154 ceph-mon[34495]: pgmap v484: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 532 KiB/s wr, 80 op/s 2024-09-10T22:06:07.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:06 smithi171 ceph-mon[34138]: pgmap v484: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 532 KiB/s wr, 80 op/s 2024-09-10T22:06:07.449 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 1.98 MB/sec warmup 59 sec latency 10060.626 ms 2024-09-10T22:06:08.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 1.95 MB/sec warmup 60 sec latency 11060.731 ms 2024-09-10T22:06:09.050 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:08 smithi171 ceph-mon[34138]: pgmap v485: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 353 KiB/s wr, 70 op/s 2024-09-10T22:06:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:08 smithi090 ceph-mon[30350]: pgmap v485: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 353 KiB/s wr, 70 op/s 2024-09-10T22:06:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:08 smithi154 ceph-mon[34495]: pgmap v485: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 353 KiB/s wr, 70 op/s 2024-09-10T22:06:09.449 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 1.92 MB/sec warmup 61 sec latency 12060.858 ms 2024-09-10T22:06:10.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 1.89 MB/sec warmup 62 sec latency 13060.910 ms 2024-09-10T22:06:11.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:10 smithi090 ceph-mon[30350]: pgmap v486: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 303 KiB/s wr, 72 op/s 2024-09-10T22:06:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:10 smithi154 ceph-mon[34495]: pgmap v486: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 303 KiB/s wr, 72 op/s 2024-09-10T22:06:11.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:10 smithi171 ceph-mon[34138]: pgmap v486: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 303 KiB/s wr, 72 op/s 2024-09-10T22:06:11.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 1.86 MB/sec warmup 63 sec latency 14060.997 ms 2024-09-10T22:06:12.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 1.83 MB/sec warmup 64 sec latency 15061.059 ms 2024-09-10T22:06:12.891 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:06:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:12 smithi090 ceph-mon[30350]: pgmap v487: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 34 op/s 2024-09-10T22:06:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:12 smithi154 ceph-mon[34495]: pgmap v487: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 34 op/s 2024-09-10T22:06:13.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:12 smithi171 ceph-mon[34138]: pgmap v487: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 71 KiB/s wr, 34 op/s 2024-09-10T22:06:13.276 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:06:13.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 1.80 MB/sec warmup 65 sec latency 16061.146 ms 2024-09-10T22:06:14.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.019+0000 7f8314a00640 1 -- 172.21.15.90:0/975857694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 msgr2=0x7f8310111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:14.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.019+0000 7f8314a00640 1 --2- 172.21.15.90:0/975857694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 0x7f8310111610 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8308009900 tx=0x7f830802a410 comp rx=0 tx=0).stop 2024-09-10T22:06:14.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.019+0000 7f8314a00640 1 -- 172.21.15.90:0/975857694 shutdown_connections 2024-09-10T22:06:14.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.019+0000 7f8314a00640 1 --2- 172.21.15.90:0/975857694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 0x7f8310111610 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.019+0000 7f8314a00640 1 --2- 172.21.15.90:0/975857694 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8310075de0 0x7f831010ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.019+0000 7f8314a00640 1 --2- 172.21.15.90:0/975857694 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f83100754c0 0x7f83100758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.019+0000 7f8314a00640 1 -- 172.21.15.90:0/975857694 >> 172.21.15.90:0/975857694 conn(0x7f83100fe660 msgr2=0x7f8310100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:14.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.019+0000 7f8314a00640 1 -- 172.21.15.90:0/975857694 shutdown_connections 2024-09-10T22:06:14.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.019+0000 7f8314a00640 1 -- 172.21.15.90:0/975857694 wait complete. 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.020+0000 7f8314a00640 1 Processor -- start 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.020+0000 7f8314a00640 1 -- start start 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.020+0000 7f8314a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83100754c0 0x7f831019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.020+0000 7f8314a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8310075de0 0x7f831019b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.020+0000 7f8314a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 0x7f83101a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.020+0000 7f8314a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8310114d00 con 0x7f831010f220 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.020+0000 7f8314a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8310114b80 con 0x7f8310075de0 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.020+0000 7f8314a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8310114e80 con 0x7f83100754c0 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 0x7f83101a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83100754c0 0x7f831019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:14.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83100754c0 0x7f831019b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:42974/0 (socket says 172.21.15.90:42974) 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 0x7f83101a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58208/0 (socket says 172.21.15.90:58208) 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830e800640 1 -- 172.21.15.90:0/1606922131 learned_addr learned my addr 172.21.15.90:0/1606922131 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f8305e00640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8310075de0 0x7f831019b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830f200640 1 -- 172.21.15.90:0/1606922131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83100754c0 msgr2=0x7f831019b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830f200640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83100754c0 0x7f831019b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830f200640 1 -- 172.21.15.90:0/1606922131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8310075de0 msgr2=0x7f831019b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830f200640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8310075de0 0x7f831019b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.021+0000 7f830f200640 1 -- 172.21.15.90:0/1606922131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83101a1160 con 0x7f831010f220 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.022+0000 7f830f200640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 0x7f83101a0990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f83080057c0 tx=0x7f8308003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.022+0000 7f8307e00640 1 -- 172.21.15.90:0/1606922131 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830803f070 con 0x7f831010f220 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.022+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83101a13f0 con 0x7f831010f220 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.022+0000 7f8307e00640 1 -- 172.21.15.90:0/1606922131 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8308003e40 con 0x7f831010f220 2024-09-10T22:06:14.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.022+0000 7f8307e00640 1 -- 172.21.15.90:0/1606922131 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308035030 con 0x7f831010f220 2024-09-10T22:06:14.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.022+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83101a7b10 con 0x7f831010f220 2024-09-10T22:06:14.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.023+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83100758a0 con 0x7f831010f220 2024-09-10T22:06:14.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.025+0000 7f8307e00640 1 -- 172.21.15.90:0/1606922131 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8308031ad0 con 0x7f831010f220 2024-09-10T22:06:14.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.027+0000 7f8307e00640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82f40b5500 0x7f82f40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:14.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.027+0000 7f8307e00640 1 -- 172.21.15.90:0/1606922131 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f8308106f40 con 0x7f831010f220 2024-09-10T22:06:14.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.027+0000 7f830e800640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82f40b5500 0x7f82f40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:14.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.028+0000 7f830e800640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82f40b5500 0x7f82f40b79c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8300007f80 tx=0x7f8300007670 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:14.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.029+0000 7f8307e00640 1 -- 172.21.15.90:0/1606922131 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f83080cd0a0 con 0x7f831010f220 2024-09-10T22:06:14.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.250+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83101a7d70 con 0x7f831010f220 2024-09-10T22:06:14.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.251+0000 7f8307e00640 1 -- 172.21.15.90:0/1606922131 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11993 (secure 0 0 0) 0x7f83080d1ad0 con 0x7f831010f220 2024-09-10T22:06:14.253 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:06:14.253 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":32,"btime":"2024-09-10T22:05:28:775491+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:05:28.494576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:06:14.253 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 32 2024-09-10T22:06:14.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82f40b5500 msgr2=0x7f82f40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:14.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82f40b5500 0x7f82f40b79c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8300007f80 tx=0x7f8300007670 comp rx=0 tx=0).stop 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 msgr2=0x7f83101a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 0x7f83101a0990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f83080057c0 tx=0x7f8308003cd0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 shutdown_connections 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f82f40b5500 0x7f82f40b79c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f831010f220 0x7f83101a0990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8310075de0 0x7f831019b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 --2- 172.21.15.90:0/1606922131 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f83100754c0 0x7f831019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.253+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 >> 172.21.15.90:0/1606922131 conn(0x7f83100fe660 msgr2=0x7f83100feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.254+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 shutdown_connections 2024-09-10T22:06:14.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:14.254+0000 7f8314a00640 1 -- 172.21.15.90:0/1606922131 wait complete. 2024-09-10T22:06:14.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 1.77 MB/sec warmup 66 sec latency 17061.217 ms 2024-09-10T22:06:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:14 smithi090 ceph-mon[30350]: pgmap v488: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 KiB/s wr, 30 op/s 2024-09-10T22:06:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:14 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1606922131' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:06:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:14 smithi154 ceph-mon[34495]: pgmap v488: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 KiB/s wr, 30 op/s 2024-09-10T22:06:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:14 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1606922131' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:06:15.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:14 smithi171 ceph-mon[34138]: pgmap v488: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 37 KiB/s wr, 30 op/s 2024-09-10T22:06:15.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:14 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1606922131' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:06:15.327 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 36789a83 --release --await' 2024-09-10T22:06:15.328 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 36789a83 --release --await 2024-09-10T22:06:15.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18477 1.74 MB/sec warmup 67 sec latency 18061.280 ms 2024-09-10T22:06:15.697 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:06:16.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.368+0000 7f3623e00640 1 -- 172.21.15.90:0/1654244753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 msgr2=0x7f361c105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:16.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.368+0000 7f3623e00640 1 --2- 172.21.15.90:0/1654244753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 0x7f361c105380 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3610009900 tx=0x7f361002a410 comp rx=0 tx=0).stop 2024-09-10T22:06:16.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.368+0000 7f3623e00640 1 -- 172.21.15.90:0/1654244753 shutdown_connections 2024-09-10T22:06:16.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.368+0000 7f3623e00640 1 --2- 172.21.15.90:0/1654244753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f361c10b330 0x7f361c111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.368+0000 7f3623e00640 1 --2- 172.21.15.90:0/1654244753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c10a9b0 0x7f361c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.368+0000 7f3623e00640 1 --2- 172.21.15.90:0/1654244753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 0x7f361c105380 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.368+0000 7f3623e00640 1 -- 172.21.15.90:0/1654244753 >> 172.21.15.90:0/1654244753 conn(0x7f361c100a50 msgr2=0x7f361c102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:16.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.368+0000 7f3623e00640 1 -- 172.21.15.90:0/1654244753 shutdown_connections 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.368+0000 7f3623e00640 1 -- 172.21.15.90:0/1654244753 wait complete. 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.369+0000 7f3623e00640 1 Processor -- start 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.369+0000 7f3623e00640 1 -- start start 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.369+0000 7f3623e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 0x7f361c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.369+0000 7f3623e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c10a9b0 0x7f361c19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3623e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f361c10b330 0x7f361c1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3623e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f361c114e40 con 0x7f361c104fa0 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3623e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f361c114cc0 con 0x7f361c10a9b0 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3623e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f361c114fc0 con 0x7f361c10b330 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 0x7f361c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 0x7f361c19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58232/0 (socket says 172.21.15.90:58232) 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621400640 1 -- 172.21.15.90:0/99592196 learned_addr learned my addr 172.21.15.90:0/99592196 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3620a00640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c10a9b0 0x7f361c19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621e00640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f361c10b330 0x7f361c1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621400640 1 -- 172.21.15.90:0/99592196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f361c10b330 msgr2=0x7f361c1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621400640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f361c10b330 0x7f361c1a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621400640 1 -- 172.21.15.90:0/99592196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c10a9b0 msgr2=0x7f361c19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621400640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c10a9b0 0x7f361c19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621400640 1 -- 172.21.15.90:0/99592196 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c1a1440 con 0x7f361c104fa0 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.370+0000 7f3621e00640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f361c10b330 0x7f361c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.371+0000 7f3621400640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 0x7f361c19b390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f361002a930 tx=0x7f3610003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.371+0000 7f361a000640 1 -- 172.21.15.90:0/99592196 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361003f070 con 0x7f361c104fa0 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.371+0000 7f361a000640 1 -- 172.21.15.90:0/99592196 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f361003baf0 con 0x7f361c104fa0 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.371+0000 7f361a000640 1 -- 172.21.15.90:0/99592196 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610035030 con 0x7f361c104fa0 2024-09-10T22:06:16.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.371+0000 7f3623e00640 1 -- 172.21.15.90:0/99592196 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c1a16d0 con 0x7f361c104fa0 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.371+0000 7f3623e00640 1 -- 172.21.15.90:0/99592196 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c1a7df0 con 0x7f361c104fa0 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.374+0000 7f36269be740 1 Processor -- start 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.374+0000 7f361a000640 1 -- 172.21.15.90:0/99592196 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f361003b660 con 0x7f361c104fa0 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.374+0000 7f36269be740 1 -- start start 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f36269be740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad07ff0 0x5639cad083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f36269be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad14360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f36269be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad148a0 0x5639cad16c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f36269be740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5639cac7f290 con 0x5639cad07ff0 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f36269be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5639cac6e6c0 con 0x5639cad08910 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f36269be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5639cac724d0 con 0x5639cad148a0 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad07ff0 0x5639cad083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad07ff0 0x5639cad083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58248/0 (socket says 172.21.15.90:58248) 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621e00640 1 -- 172.21.15.90:0/874665845 learned_addr learned my addr 172.21.15.90:0/874665845 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621400640 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad14360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3620a00640 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad148a0 0x5639cad16c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621e00640 1 -- 172.21.15.90:0/874665845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad148a0 msgr2=0x5639cad16c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621e00640 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad148a0 0x5639cad16c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621e00640 1 -- 172.21.15.90:0/874665845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 msgr2=0x5639cad14360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621e00640 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad14360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621e00640 1 -- 172.21.15.90:0/874665845 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639cabfb460 con 0x5639cad07ff0 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.375+0000 7f3621400640 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad14360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.376+0000 7f3621e00640 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad07ff0 0x5639cad083d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3614010370 tx=0x7f3614010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493da18ab57be342 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.376+0000 7f360e600640 1 -- 172.21.15.90:0/874665845 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614016070 con 0x5639cad07ff0 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.376+0000 7f360e600640 1 -- 172.21.15.90:0/874665845 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3614012ba0 con 0x5639cad07ff0 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.376+0000 7f360e600640 1 -- 172.21.15.90:0/874665845 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361401bb10 con 0x5639cad07ff0 2024-09-10T22:06:16.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.376+0000 7f361a000640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f36040b5450 0x7f36040b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.376+0000 7f3621400640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f36040b5450 0x7f36040b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f361a000640 1 -- 172.21.15.90:0/99592196 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3610106dc0 con 0x7f361c104fa0 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f36269be740 1 -- 172.21.15.90:0/874665845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad07ff0 msgr2=0x5639cad083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f36269be740 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad07ff0 0x5639cad083d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3614010370 tx=0x7f3614010830 comp rx=0 tx=0).stop 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f3621400640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f36040b5450 0x7f36040b7910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5639cad0b170 tx=0x7f3610009260 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f36269be740 1 -- 172.21.15.90:0/874665845 shutdown_connections 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f36269be740 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad148a0 0x5639cad16c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f36269be740 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad14360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f36269be740 1 --2- 172.21.15.90:0/874665845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad07ff0 0x5639cad083d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f36269be740 1 -- 172.21.15.90:0/874665845 >> 172.21.15.90:0/874665845 conn(0x5639cad05e20 msgr2=0x5639cad06fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f36269be740 1 -- 172.21.15.90:0/874665845 shutdown_connections 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.377+0000 7f36269be740 1 -- 172.21.15.90:0/874665845 wait complete. 2024-09-10T22:06:16.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.378+0000 7f36269be740 1 -- start start 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.378+0000 7f36269be740 1 Processor -- start 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.378+0000 7f36269be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad07ff0 0x5639cad118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.378+0000 7f36269be740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad08910 0x5639cad11e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.378+0000 7f36269be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad148a0 0x5639cad12340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.378+0000 7f36269be740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5639caacfbb0 con 0x5639cad08910 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.378+0000 7f36269be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5639cac096c0 con 0x5639cad07ff0 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.378+0000 7f36269be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5639cad17550 con 0x5639cad148a0 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.378+0000 7f36269be740 10 client.0 _ll_register_callbacks cb 0x5639cac24490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3620a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad08910 0x5639cad11e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3620a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad08910 0x5639cad11e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58258/0 (socket says 172.21.15.90:58258) 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3620a00640 1 -- 172.21.15.90:0/3375325108 learned_addr learned my addr 172.21.15.90:0/3375325108 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3621e00640 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad07ff0 0x5639cad118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3620a00640 1 -- 172.21.15.90:0/3375325108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad148a0 msgr2=0x5639cad12340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:16.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3620a00640 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad148a0 0x5639cad12340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3620a00640 1 -- 172.21.15.90:0/3375325108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad07ff0 msgr2=0x5639cad118c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3620a00640 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad07ff0 0x5639cad118c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3620a00640 1 -- 172.21.15.90:0/3375325108 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ca9ccb00 con 0x5639cad08910 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3620a00640 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad08910 0x5639cad11e00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3608012700 tx=0x7f3608012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bc646e9ddce9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3602a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3602a00640 1 -- 172.21.15.90:0/3375325108 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360800f8c0 con 0x5639cad08910 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f3602a00640 1 -- 172.21.15.90:0/3375325108 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f360800fa60 con 0x5639cad08910 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.379+0000 7f36269be740 1 -- 172.21.15.90:0/3375325108 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3614012ba0 con 0x5639cad08910 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f3602a00640 1 -- 172.21.15.90:0/3375325108 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3608025050 con 0x5639cad08910 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f36269be740 10 client.15810 fetch_fsmap learned FSMap version 32 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f36269be740 1 -- 172.21.15.90:0/3375325108 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5639caa43960 con 0x5639cad08910 2024-09-10T22:06:16.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f3602a00640 1 -- 172.21.15.90:0/3375325108 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3614012ba0 con 0x5639cad08910 2024-09-10T22:06:16.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f3602a00640 1 -- 172.21.15.90:0/3375325108 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f360801cc80 con 0x5639cad08910 2024-09-10T22:06:16.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f36269be740 10 client.15810 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:06:16.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f36269be740 10 client.15810 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:06:16.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f36269be740 20 client.15810 populate_metadata read hostname 'smithi090' 2024-09-10T22:06:16.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f36269be740 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639cad244e0 0x5639cad26930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f36269be740 4 client.15810 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.380+0000 7f36269be740 1 -- 172.21.15.90:0/3375325108 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639cac811e0 con 0x5639cad244e0 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.381+0000 7f3621e00640 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639cad244e0 0x5639cad26930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.382+0000 7f3602a00640 10 client.15810 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.382+0000 7f3621e00640 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639cad244e0 0x5639cad26930 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=208f28452afd97a4 server_cookie=3eeba631c62a644b in_seq=0 out_seq=0 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.382+0000 7f3602a00640 1 -- 172.21.15.90:0/3375325108 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5639cac811e0 con 0x5639cad244e0 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.382+0000 7f3602a00640 10 client.15810 handle_command_reply: tid=0 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.382+0000 7f36269be740 1 client.15810 shutdown 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 -- 172.21.15.90:0/3375325108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad08910 msgr2=0x5639cad11e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad08910 0x5639cad11e00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3608012700 tx=0x7f3608012bc0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 -- 172.21.15.90:0/3375325108 shutdown_connections 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639cad244e0 0x5639cad26930 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad148a0 0x5639cad12340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad08910 0x5639cad11e00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f3602a00640 0 client.15810 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 --2- 172.21.15.90:0/3375325108 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad07ff0 0x5639cad118c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 -- 172.21.15.90:0/3375325108 >> 172.21.15.90:0/3375325108 conn(0x5639cad05e20 msgr2=0x5639cad0b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 -- 172.21.15.90:0/3375325108 shutdown_connections 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 1 -- 172.21.15.90:0/3375325108 wait complete. 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.383+0000 7f36269be740 20 client.15810 trim_cache size 0 max 16384 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.400+0000 7f36269be740 1 Processor -- start 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f36269be740 1 -- start start 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f36269be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 0x5639cad0c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f36269be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad04ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f36269be740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad0ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f36269be740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b1f00 con 0x5639cad148a0 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f36269be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b3b00 con 0x5639cad08910 2024-09-10T22:06:16.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f36269be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b25d0 con 0x5639cad07ff0 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f3621e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad0ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f3621e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad0ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58264/0 (socket says 172.21.15.90:58264) 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f3621e00640 1 -- 172.21.15.90:0/3074481010 learned_addr learned my addr 172.21.15.90:0/3074481010 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f3620a00640 1 --2- 172.21.15.90:0/3074481010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad04ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.401+0000 7f3621e00640 1 --2- 172.21.15.90:0/3074481010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 0x5639cad0c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.402+0000 7f3621e00640 1 -- 172.21.15.90:0/3074481010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 msgr2=0x5639cad0c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.402+0000 7f3621e00640 1 --2- 172.21.15.90:0/3074481010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 0x5639cad0c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.402+0000 7f3621e00640 1 -- 172.21.15.90:0/3074481010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 msgr2=0x5639cad04ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.402+0000 7f3621e00640 1 --2- 172.21.15.90:0/3074481010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad04ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.402+0000 7f3621e00640 1 -- 172.21.15.90:0/3074481010 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639cad068a0 con 0x5639cad148a0 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.402+0000 7f3621e00640 1 --2- 172.21.15.90:0/3074481010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad0ead0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3614000f80 tx=0x7f3614010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772cdd28682569b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.402+0000 7f360dc00640 1 -- 172.21.15.90:0/3074481010 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361402a070 con 0x5639cad148a0 2024-09-10T22:06:16.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.402+0000 7f360dc00640 1 -- 172.21.15.90:0/3074481010 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f361401bc50 con 0x5639cad148a0 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.402+0000 7f360dc00640 1 -- 172.21.15.90:0/3074481010 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614016040 con 0x5639cad148a0 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 -- 172.21.15.90:0/3074481010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 msgr2=0x5639cad0ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 --2- 172.21.15.90:0/3074481010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad0ead0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3614000f80 tx=0x7f3614010fa0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 -- 172.21.15.90:0/3074481010 shutdown_connections 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 --2- 172.21.15.90:0/3074481010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad0ead0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 --2- 172.21.15.90:0/3074481010 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad04ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 --2- 172.21.15.90:0/3074481010 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 0x5639cad0c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 -- 172.21.15.90:0/3074481010 >> 172.21.15.90:0/3074481010 conn(0x5639cad05e20 msgr2=0x5639cad04590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 -- 172.21.15.90:0/3074481010 shutdown_connections 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 -- 172.21.15.90:0/3074481010 wait complete. 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.403+0000 7f36269be740 1 -- start start 2024-09-10T22:06:16.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.404+0000 7f36269be740 1 Processor -- start 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.404+0000 7f36269be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 0x5639cad18c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.404+0000 7f36269be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad19150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.404+0000 7f36269be740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad90450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.404+0000 7f36269be740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b3370 con 0x5639cad148a0 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.404+0000 7f36269be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1b2ca0 con 0x5639cad08910 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.404+0000 7f36269be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1afee0 con 0x5639cad07ff0 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.404+0000 7f36269be740 10 client.0 _ll_register_callbacks cb 0x7f3608009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.404+0000 7f3621e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad90450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3621e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad90450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58280/0 (socket says 172.21.15.90:58280) 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3621e00640 1 -- 172.21.15.90:0/3933980955 learned_addr learned my addr 172.21.15.90:0/3933980955 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3620a00640 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad19150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3621e00640 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 0x5639cad18c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3621e00640 1 -- 172.21.15.90:0/3933980955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 msgr2=0x5639cad18c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3621e00640 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 0x5639cad18c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3621e00640 1 -- 172.21.15.90:0/3933980955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 msgr2=0x5639cad19150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3621e00640 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad19150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3621e00640 1 -- 172.21.15.90:0/3933980955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639cad90990 con 0x5639cad148a0 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3620a00640 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad19150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3621e00640 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad90450 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f361401b5c0 tx=0x7f3614009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e39d050127abd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3602a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3602a00640 1 -- 172.21.15.90:0/3933980955 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361402a070 con 0x5639cad148a0 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f3602a00640 1 -- 172.21.15.90:0/3933980955 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f3614004a00 con 0x5639cad148a0 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.405+0000 7f36269be740 1 -- 172.21.15.90:0/3933980955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f361401bc50 con 0x5639cad148a0 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f3602a00640 1 -- 172.21.15.90:0/3933980955 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f361403e050 con 0x5639cad148a0 2024-09-10T22:06:16.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f36269be740 10 client.15816 fetch_fsmap learned FSMap version 32 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f36269be740 1 -- 172.21.15.90:0/3933980955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5639cad91c20 con 0x5639cad148a0 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f3602a00640 1 -- 172.21.15.90:0/3933980955 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f361401bc50 con 0x5639cad148a0 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f3602a00640 1 -- 172.21.15.90:0/3933980955 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f361400f040 con 0x5639cad148a0 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f36269be740 10 client.15816 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f36269be740 10 client.15816 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f36269be740 20 client.15816 populate_metadata read hostname 'smithi090' 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f36269be740 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639cad921c0 0x5639cad945e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f36269be740 4 client.15816 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "36789a83", "release": true, "await": true}] 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.406+0000 7f36269be740 1 -- 172.21.15.90:0/3933980955 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "36789a83", "release": true, "await": true}) -- 0x5639cac7cc90 con 0x5639cad921c0 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.407+0000 7f3620a00640 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639cad921c0 0x5639cad945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.407+0000 7f3620a00640 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639cad921c0 0x5639cad945e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=314ebf088c8547d5 server_cookie=76a2bbff9142ec2 in_seq=0 out_seq=0 2024-09-10T22:06:16.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.407+0000 7f3602a00640 10 client.15816 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f3602a00640 1 -- 172.21.15.90:0/3933980955 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5639cac7cc90 con 0x5639cad921c0 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f3602a00640 10 client.15816 handle_command_reply: tid=0 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f36269be740 1 client.15816 shutdown 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f36269be740 1 -- 172.21.15.90:0/3933980955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 msgr2=0x5639cad90450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f36269be740 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad90450 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f361401b5c0 tx=0x7f3614009780 comp rx=0 tx=0).stop 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f36269be740 1 -- 172.21.15.90:0/3933980955 shutdown_connections 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f36269be740 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639cad921c0 0x5639cad945e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f36269be740 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5639cad148a0 0x5639cad90450 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f36269be740 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5639cad08910 0x5639cad19150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f36269be740 1 --2- 172.21.15.90:0/3933980955 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5639cad07ff0 0x5639cad18c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f3602a00640 0 client.15816 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.414+0000 7f36269be740 1 -- 172.21.15.90:0/3933980955 >> 172.21.15.90:0/3933980955 conn(0x5639cad05e20 msgr2=0x5639cad04db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.415+0000 7f36269be740 1 -- 172.21.15.90:0/3933980955 shutdown_connections 2024-09-10T22:06:16.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.415+0000 7f36269be740 1 -- 172.21.15.90:0/3933980955 wait complete. 2024-09-10T22:06:16.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.415+0000 7f36269be740 20 client.15816 trim_cache size 0 max 16384 2024-09-10T22:06:16.450 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:06:16.450 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 32, 2024-09-10T22:06:16.450 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:06:16.450 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 8, 2024-09-10T22:06:16.450 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:06:16.450 INFO:teuthology.orchestra.run.smithi090.stdout: "36789a83": { 2024-09-10T22:06:16.450 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 8, 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 140.5, 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:06:16.451 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:06:16.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:06:16.452 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:06:16.452 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:16.452 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:16.452 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:16.452 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:16.452 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:16.452 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:06:16.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 -- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f36040b5450 msgr2=0x7f36040b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f36040b5450 0x7f36040b7910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5639cad0b170 tx=0x7f3610009260 comp rx=0 tx=0).stop 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 -- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 msgr2=0x7f361c19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 0x7f361c19b390 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f361002a930 tx=0x7f3610003b90 comp rx=0 tx=0).stop 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 -- 172.21.15.90:0/99592196 shutdown_connections 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f36040b5450 0x7f36040b7910 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f361c10b330 0x7f361c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f361c10a9b0 0x7f361c19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 --2- 172.21.15.90:0/99592196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f361c104fa0 0x7f361c19b390 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.416+0000 7f3623e00640 1 -- 172.21.15.90:0/99592196 >> 172.21.15.90:0/99592196 conn(0x7f361c100a50 msgr2=0x7f361c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.417+0000 7f3623e00640 1 -- 172.21.15.90:0/99592196 shutdown_connections 2024-09-10T22:06:16.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:16.417+0000 7f3623e00640 1 -- 172.21.15.90:0/99592196 wait complete. 2024-09-10T22:06:16.454 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18495 1.72 MB/sec warmup 68 sec latency 19024.846 ms 2024-09-10T22:06:16.845 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:16 smithi090 ceph-mon[30350]: pgmap v489: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-09-10T22:06:16.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:16 smithi154 ceph-mon[34495]: pgmap v489: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-09-10T22:06:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:16 smithi171 ceph-mon[34138]: pgmap v489: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.1 KiB/s wr, 13 op/s 2024-09-10T22:06:17.359 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '36789a83', total seconds elapsed: 25.0 2024-09-10T22:06:17.360 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.5 2024-09-10T22:06:17.360 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:06:17.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18840 1.72 MB/sec warmup 69 sec latency 45.362 ms 2024-09-10T22:06:17.772 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:06:18.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.353+0000 7f65f9c00640 1 -- 172.21.15.90:0/169322819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410b1a0 msgr2=0x7f65f41119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:18.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.353+0000 7f65f9c00640 1 --2- 172.21.15.90:0/169322819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410b1a0 0x7f65f41119a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009ca0 tx=0x7f65ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.353+0000 7f65f9c00640 1 -- 172.21.15.90:0/169322819 shutdown_connections 2024-09-10T22:06:18.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.353+0000 7f65f9c00640 1 --2- 172.21.15.90:0/169322819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410b1a0 0x7f65f41119a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.353+0000 7f65f9c00640 1 --2- 172.21.15.90:0/169322819 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f410a820 0x7f65f410ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.353+0000 7f65f9c00640 1 --2- 172.21.15.90:0/169322819 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4104e10 0x7f65f41051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.353+0000 7f65f9c00640 1 -- 172.21.15.90:0/169322819 >> 172.21.15.90:0/169322819 conn(0x7f65f4100840 msgr2=0x7f65f4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:18.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.353+0000 7f65f9c00640 1 -- 172.21.15.90:0/169322819 shutdown_connections 2024-09-10T22:06:18.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.353+0000 7f65f9c00640 1 -- 172.21.15.90:0/169322819 wait complete. 2024-09-10T22:06:18.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.354+0000 7f65f9c00640 1 Processor -- start 2024-09-10T22:06:18.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f9c00640 1 -- start start 2024-09-10T22:06:18.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f9c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4104e10 0x7f65f419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410a820 0x7f65f419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f9c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f410b1a0 0x7f65f41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f9c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4114d60 con 0x7f65f410a820 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f9c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4114be0 con 0x7f65f410b1a0 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f9c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4114ee0 con 0x7f65f4104e10 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410a820 0x7f65f419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410a820 0x7f65f419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55224/0 (socket says 172.21.15.90:55224) 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f3400640 1 -- 172.21.15.90:0/3113552741 learned_addr learned my addr 172.21.15.90:0/3113552741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.355+0000 7f65f3e00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4104e10 0x7f65f419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.356+0000 7f65f9200640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f410b1a0 0x7f65f41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.356+0000 7f65f3400640 1 -- 172.21.15.90:0/3113552741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4104e10 msgr2=0x7f65f419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.356+0000 7f65f3400640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4104e10 0x7f65f419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.356+0000 7f65f3400640 1 -- 172.21.15.90:0/3113552741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f410b1a0 msgr2=0x7f65f41a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:18.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.356+0000 7f65f3400640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f410b1a0 0x7f65f41a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.356+0000 7f65f3400640 1 -- 172.21.15.90:0/3113552741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f41a1200 con 0x7f65f410a820 2024-09-10T22:06:18.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.356+0000 7f65f3400640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410a820 0x7f65f419b690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f65e000e9f0 tx=0x7f65e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:18.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.357+0000 7f65f0c00640 1 -- 172.21.15.90:0/3113552741 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e000cd80 con 0x7f65f410a820 2024-09-10T22:06:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.357+0000 7f65f0c00640 1 -- 172.21.15.90:0/3113552741 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f65e00108c0 con 0x7f65f410a820 2024-09-10T22:06:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.357+0000 7f65f0c00640 1 -- 172.21.15.90:0/3113552741 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e00187c0 con 0x7f65f410a820 2024-09-10T22:06:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.357+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f41a14f0 con 0x7f65f410a820 2024-09-10T22:06:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.357+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41a7d80 con 0x7f65f410a820 2024-09-10T22:06:18.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.358+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f41060c0 con 0x7f65f410a820 2024-09-10T22:06:18.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.361+0000 7f65f0c00640 1 -- 172.21.15.90:0/3113552741 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f65e0010430 con 0x7f65f410a820 2024-09-10T22:06:18.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.362+0000 7f65f0c00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b5580 0x7f65d80b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:18.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.362+0000 7f65f0c00640 1 -- 172.21.15.90:0/3113552741 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f65e00e66d0 con 0x7f65f410a820 2024-09-10T22:06:18.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.363+0000 7f65f3e00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b5580 0x7f65d80b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:18.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.363+0000 7f65f3e00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b5580 0x7f65d80b7a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65e4005fd0 tx=0x7f65e400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:18.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.367+0000 7f65f0c00640 1 -- 172.21.15.90:0/3113552741 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f65e00ace80 con 0x7f65f410a820 2024-09-10T22:06:18.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 19243 1.74 MB/sec warmup 70 sec latency 17.148 ms 2024-09-10T22:06:18.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.653+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65f41a7f50 con 0x7f65f410a820 2024-09-10T22:06:18.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.654+0000 7f65f0c00640 1 -- 172.21.15.90:0/3113552741 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11993 (secure 0 0 0) 0x7f65e00ac780 con 0x7f65f410a820 2024-09-10T22:06:18.656 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:06:18.656 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":32,"btime":"2024-09-10T22:05:28:775491+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:05:28.494576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:06:18.656 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 32 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b5580 msgr2=0x7f65d80b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b5580 0x7f65d80b7a40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65e4005fd0 tx=0x7f65e400b390 comp rx=0 tx=0).stop 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410a820 msgr2=0x7f65f419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410a820 0x7f65f419b690 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f65e000e9f0 tx=0x7f65e000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 shutdown_connections 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f65d80b5580 0x7f65d80b7a40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f65f410b1a0 0x7f65f41a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65f410a820 0x7f65f419b690 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 --2- 172.21.15.90:0/3113552741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f65f4104e10 0x7f65f419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:18.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 >> 172.21.15.90:0/3113552741 conn(0x7f65f4100840 msgr2=0x7f65f4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:18.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 shutdown_connections 2024-09-10T22:06:18.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:18.657+0000 7f65f9c00640 1 -- 172.21.15.90:0/3113552741 wait complete. 2024-09-10T22:06:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:18 smithi171 ceph-mon[34138]: pgmap v490: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-09-10T22:06:18.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:18 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3113552741' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:06:19.097 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:18 smithi090 ceph-mon[30350]: pgmap v490: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-09-10T22:06:19.097 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:18 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3113552741' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:06:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:18 smithi154 ceph-mon[34495]: pgmap v490: 129 pgs: 129 active+clean; 31 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-09-10T22:06:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:18 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3113552741' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:06:19.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 19649 1.72 MB/sec warmup 71 sec latency 32.673 ms 2024-09-10T22:06:19.513 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0' 2024-09-10T22:06:19.514 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 140.5 --await-for 45.0 2024-09-10T22:06:19.945 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:06:20.450 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20122 1.72 MB/sec warmup 72 sec latency 19.816 ms 2024-09-10T22:06:20.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.651+0000 7fdd39600640 1 -- 172.21.15.90:0/140137599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34108620 msgr2=0x7fdd34108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:20.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.651+0000 7fdd39600640 1 --2- 172.21.15.90:0/140137599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34108620 0x7fdd34108a60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009860 tx=0x7fdd2002a340 comp rx=0 tx=0).stop 2024-09-10T22:06:20.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.651+0000 7fdd39600640 1 -- 172.21.15.90:0/140137599 shutdown_connections 2024-09-10T22:06:20.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.651+0000 7fdd39600640 1 --2- 172.21.15.90:0/140137599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd34108fa0 0x7fdd3410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.651+0000 7fdd39600640 1 --2- 172.21.15.90:0/140137599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34108620 0x7fdd34108a60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.651+0000 7fdd39600640 1 --2- 172.21.15.90:0/140137599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd34102c10 0x7fdd34102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.651+0000 7fdd39600640 1 -- 172.21.15.90:0/140137599 >> 172.21.15.90:0/140137599 conn(0x7fdd340fe6c0 msgr2=0x7fdd34100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.651+0000 7fdd39600640 1 -- 172.21.15.90:0/140137599 shutdown_connections 2024-09-10T22:06:20.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.651+0000 7fdd39600640 1 -- 172.21.15.90:0/140137599 wait complete. 2024-09-10T22:06:20.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.652+0000 7fdd39600640 1 Processor -- start 2024-09-10T22:06:20.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd39600640 1 -- start start 2024-09-10T22:06:20.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd39600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd34102c10 0x7fdd3419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd39600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd34108620 0x7fdd3419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd39600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34108fa0 0x7fdd341a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd39600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34112aa0 con 0x7fdd34102c10 2024-09-10T22:06:20.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd39600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34112920 con 0x7fdd34108fa0 2024-09-10T22:06:20.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd39600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34112c20 con 0x7fdd34108620 2024-09-10T22:06:20.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd33e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd34102c10 0x7fdd3419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd38c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34108fa0 0x7fdd341a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd33e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd34102c10 0x7fdd3419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55240/0 (socket says 172.21.15.90:55240) 2024-09-10T22:06:20.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd33e00640 1 -- 172.21.15.90:0/285234625 learned_addr learned my addr 172.21.15.90:0/285234625 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:20.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.653+0000 7fdd33400640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd34108620 0x7fdd3419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.654+0000 7fdd33e00640 1 -- 172.21.15.90:0/285234625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd34108620 msgr2=0x7fdd3419b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.654+0000 7fdd33e00640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd34108620 0x7fdd3419b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.654+0000 7fdd33e00640 1 -- 172.21.15.90:0/285234625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34108fa0 msgr2=0x7fdd341a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.654+0000 7fdd33e00640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34108fa0 0x7fdd341a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.654+0000 7fdd33e00640 1 -- 172.21.15.90:0/285234625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd341a1230 con 0x7fdd34102c10 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.654+0000 7fdd38c00640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34108fa0 0x7fdd341a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.654+0000 7fdd33e00640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd34102c10 0x7fdd3419b180 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400e9c0 tx=0x7fdd2400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.654+0000 7fdd30c00640 1 -- 172.21.15.90:0/285234625 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2400cde0 con 0x7fdd34102c10 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.654+0000 7fdd39600640 1 -- 172.21.15.90:0/285234625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd341a1520 con 0x7fdd34102c10 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.655+0000 7fdd39600640 1 -- 172.21.15.90:0/285234625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd341a7db0 con 0x7fdd34102c10 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.655+0000 7fdd30c00640 1 -- 172.21.15.90:0/285234625 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd240035f0 con 0x7fdd34102c10 2024-09-10T22:06:20.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.655+0000 7fdd30c00640 1 -- 172.21.15.90:0/285234625 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24010690 con 0x7fdd34102c10 2024-09-10T22:06:20.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.658+0000 7fdd3d65d740 1 Processor -- start 2024-09-10T22:06:20.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.658+0000 7fdd3d65d740 1 -- start start 2024-09-10T22:06:20.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.658+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.658+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b01c10 0x5643e0b0d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.658+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.658+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5643e0a76ca0 con 0x5643e0b0dba0 2024-09-10T22:06:20.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.658+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5643e0a660d0 con 0x5643e0b012f0 2024-09-10T22:06:20.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.658+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5643e0a69ee0 con 0x5643e0b01c10 2024-09-10T22:06:20.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd30c00640 1 -- 172.21.15.90:0/285234625 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdd24004020 con 0x7fdd34102c10 2024-09-10T22:06:20.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd38c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd38c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:55850/0 (socket says 172.21.15.90:55850) 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd38c00640 1 -- 172.21.15.90:0/171626137 learned_addr learned my addr 172.21.15.90:0/171626137 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd33400640 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd38c00640 1 -- 172.21.15.90:0/171626137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b01c10 msgr2=0x5643e0b0d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd33e00640 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b01c10 0x5643e0b0d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd38c00640 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b01c10 0x5643e0b0d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd38c00640 1 -- 172.21.15.90:0/171626137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b0dba0 msgr2=0x5643e0b0ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.659+0000 7fdd38c00640 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.660+0000 7fdd38c00640 1 -- 172.21.15.90:0/171626137 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e0a18400 con 0x5643e0b012f0 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.660+0000 7fdd38c00640 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b016d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0117e0 tx=0x7fdd2c011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18576d1a461a73c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.660+0000 7fdd33400640 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:20.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.660+0000 7fdd33e00640 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b01c10 0x5643e0b0d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:06:20.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.660+0000 7fdd29200640 1 -- 172.21.15.90:0/171626137 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c011310 con 0x5643e0b012f0 2024-09-10T22:06:20.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.660+0000 7fdd29200640 1 -- 172.21.15.90:0/171626137 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd2c003e00 con 0x5643e0b012f0 2024-09-10T22:06:20.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.660+0000 7fdd29200640 1 -- 172.21.15.90:0/171626137 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c004490 con 0x5643e0b012f0 2024-09-10T22:06:20.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd30c00640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd180b5580 0x7fdd180b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd30c00640 1 -- 172.21.15.90:0/285234625 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fdd240e65b0 con 0x7fdd34102c10 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd33e00640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd180b5580 0x7fdd180b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd3d65d740 1 -- 172.21.15.90:0/171626137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 msgr2=0x5643e0b016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b016d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0117e0 tx=0x7fdd2c011ca0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd3d65d740 1 -- 172.21.15.90:0/171626137 shutdown_connections 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b01c10 0x5643e0b0d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/171626137 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b016d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.661+0000 7fdd3d65d740 1 -- 172.21.15.90:0/171626137 >> 172.21.15.90:0/171626137 conn(0x5643e0aff120 msgr2=0x5643e0b002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.662+0000 7fdd3d65d740 1 -- 172.21.15.90:0/171626137 shutdown_connections 2024-09-10T22:06:20.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.662+0000 7fdd3d65d740 1 -- 172.21.15.90:0/171626137 wait complete. 2024-09-10T22:06:20.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.662+0000 7fdd33e00640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd180b5580 0x7fdd180b7a40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdd240e3a40 tx=0x7fdd2400c850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:20.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.662+0000 7fdd3d65d740 1 -- start start 2024-09-10T22:06:20.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd3d65d740 1 Processor -- start 2024-09-10T22:06:20.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b0d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5643e08c77a0 con 0x5643e0b01c10 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5643e0a0ba10 con 0x5643e0b012f0 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5643e0b10820 con 0x5643e0b0dba0 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd3d65d740 10 client.0 _ll_register_callbacks cb 0x5643e0a1bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd33400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd38c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b0d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd33400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55256/0 (socket says 172.21.15.90:55256) 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.663+0000 7fdd33400640 1 -- 172.21.15.90:0/24356722 learned_addr learned my addr 172.21.15.90:0/24356722 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:20.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.664+0000 7fdd38c00640 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.664+0000 7fdd33400640 1 -- 172.21.15.90:0/24356722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 msgr2=0x5643e0b0ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:20.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.664+0000 7fdd33400640 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.664+0000 7fdd33400640 1 -- 172.21.15.90:0/24356722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 msgr2=0x5643e0b0d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.664+0000 7fdd33400640 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b0d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.664+0000 7fdd33400640 1 -- 172.21.15.90:0/24356722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e07d3b90 con 0x5643e0b01c10 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.664+0000 7fdd38c00640 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.664+0000 7fdd33400640 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a8a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009830 tx=0x7fdd200094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a787fdc104348de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.664+0000 7fdd1d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.665+0000 7fdd1d600640 1 -- 172.21.15.90:0/24356722 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20044070 con 0x5643e0b01c10 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.665+0000 7fdd1d600640 1 -- 172.21.15.90:0/24356722 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fdd20030340 con 0x5643e0b01c10 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.665+0000 7fdd3d65d740 1 -- 172.21.15.90:0/24356722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd2c003e00 con 0x5643e0b01c10 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.665+0000 7fdd1d600640 1 -- 172.21.15.90:0/24356722 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdd2004f050 con 0x5643e0b01c10 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.665+0000 7fdd3d65d740 10 client.15834 fetch_fsmap learned FSMap version 32 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.665+0000 7fdd3d65d740 1 -- 172.21.15.90:0/24356722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5643e0b1c0b0 con 0x5643e0b01c10 2024-09-10T22:06:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.665+0000 7fdd1d600640 1 -- 172.21.15.90:0/24356722 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd2c003e00 con 0x5643e0b01c10 2024-09-10T22:06:20.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.666+0000 7fdd1d600640 1 -- 172.21.15.90:0/24356722 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdd20032440 con 0x5643e0b01c10 2024-09-10T22:06:20.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.666+0000 7fdd3d65d740 10 client.15834 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:06:20.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.666+0000 7fdd3d65d740 10 client.15834 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:06:20.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.666+0000 7fdd3d65d740 20 client.15834 populate_metadata read hostname 'smithi090' 2024-09-10T22:06:20.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.666+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5643e0b1d560 0x5643e0b1f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.666+0000 7fdd3d65d740 4 client.15834 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:06:20.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.666+0000 7fdd3d65d740 1 -- 172.21.15.90:0/24356722 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643e0a78bf0 con 0x5643e0b1d560 2024-09-10T22:06:20.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.666+0000 7fdd38c00640 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5643e0b1d560 0x5643e0b1f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.667+0000 7fdd38c00640 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5643e0b1d560 0x5643e0b1f9a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75417f6469210112 server_cookie=9a9a30c59789d55d in_seq=0 out_seq=0 2024-09-10T22:06:20.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.667+0000 7fdd1d600640 10 client.15834 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:20.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd1d600640 1 -- 172.21.15.90:0/24356722 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643e0a78bf0 con 0x5643e0b1d560 2024-09-10T22:06:20.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd1d600640 10 client.15834 handle_command_reply: tid=0 2024-09-10T22:06:20.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 client.15834 shutdown 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 -- 172.21.15.90:0/24356722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 msgr2=0x5643e0b0a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a8a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdd20009830 tx=0x7fdd200094b0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 -- 172.21.15.90:0/24356722 shutdown_connections 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5643e0b1d560 0x5643e0b1f9a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd1d600640 0 client.15834 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a8a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/24356722 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b0d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 -- 172.21.15.90:0/24356722 >> 172.21.15.90:0/24356722 conn(0x5643e0aff120 msgr2=0x5643e0b04c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 -- 172.21.15.90:0/24356722 shutdown_connections 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.668+0000 7fdd3d65d740 1 -- 172.21.15.90:0/24356722 wait complete. 2024-09-10T22:06:20.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.669+0000 7fdd3d65d740 20 client.15834 trim_cache size 0 max 16384 2024-09-10T22:06:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.687+0000 7fdd3d65d740 1 Processor -- start 2024-09-10T22:06:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.687+0000 7fdd3d65d740 1 -- start start 2024-09-10T22:06:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.687+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b19a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341b1d80 con 0x5643e0b01c10 2024-09-10T22:06:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341b3980 con 0x5643e0b012f0 2024-09-10T22:06:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341b2450 con 0x5643e0b0dba0 2024-09-10T22:06:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd33400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd38c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b19a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd38c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b19a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:55860/0 (socket says 172.21.15.90:55860) 2024-09-10T22:06:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd38c00640 1 -- 172.21.15.90:0/2139748936 learned_addr learned my addr 172.21.15.90:0/2139748936 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd38c00640 1 --2- 172.21.15.90:0/2139748936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd33400640 1 -- 172.21.15.90:0/2139748936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 msgr2=0x5643e0b0ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd33400640 1 --2- 172.21.15.90:0/2139748936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd33400640 1 -- 172.21.15.90:0/2139748936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 msgr2=0x5643e0b19a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd33400640 1 --2- 172.21.15.90:0/2139748936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b19a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd33400640 1 -- 172.21.15.90:0/2139748936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643e0b00cb0 con 0x5643e0b01c10 2024-09-10T22:06:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.688+0000 7fdd38c00640 1 --2- 172.21.15.90:0/2139748936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b19a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:06:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.689+0000 7fdd33400640 1 --2- 172.21.15.90:0/2139748936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a130 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdd2002a860 tx=0x7fdd2002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83019432ad4d0005 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:20.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.689+0000 7fdd1fe00640 1 -- 172.21.15.90:0/2139748936 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20044070 con 0x5643e0b01c10 2024-09-10T22:06:20.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.689+0000 7fdd1fe00640 1 -- 172.21.15.90:0/2139748936 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd200307d0 con 0x5643e0b01c10 2024-09-10T22:06:20.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.689+0000 7fdd1fe00640 1 -- 172.21.15.90:0/2139748936 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20000e60 con 0x5643e0b01c10 2024-09-10T22:06:20.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 -- 172.21.15.90:0/2139748936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 msgr2=0x5643e0b0a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:20.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/2139748936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a130 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdd2002a860 tx=0x7fdd2002a340 comp rx=0 tx=0).stop 2024-09-10T22:06:20.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 -- 172.21.15.90:0/2139748936 shutdown_connections 2024-09-10T22:06:20.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/2139748936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b0dba0 0x5643e0b0ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/2139748936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b0a130 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/2139748936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b012f0 0x5643e0b19a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 -- 172.21.15.90:0/2139748936 >> 172.21.15.90:0/2139748936 conn(0x5643e0aff120 msgr2=0x5643e0b0c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:20.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 -- 172.21.15.90:0/2139748936 shutdown_connections 2024-09-10T22:06:20.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 -- 172.21.15.90:0/2139748936 wait complete. 2024-09-10T22:06:20.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 -- start start 2024-09-10T22:06:20.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.690+0000 7fdd3d65d740 1 Processor -- start 2024-09-10T22:06:20.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b012f0 0x5643e0b162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b10aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd3d65d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b0dba0 0x5643e0b11000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341b31f0 con 0x5643e0b01c10 2024-09-10T22:06:20.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341b2b20 con 0x5643e0b0dba0 2024-09-10T22:06:20.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd3d65d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341afd60 con 0x5643e0b012f0 2024-09-10T22:06:20.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd3d65d740 10 client.0 _ll_register_callbacks cb 0x7fdd24005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:06:20.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd33400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b10aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd33400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b10aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55282/0 (socket says 172.21.15.90:55282) 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd33400640 1 -- 172.21.15.90:0/4157407153 learned_addr learned my addr 172.21.15.90:0/4157407153 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd38c00640 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b012f0 0x5643e0b162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.691+0000 7fdd38c00640 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b0dba0 0x5643e0b11000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd33400640 1 -- 172.21.15.90:0/4157407153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b012f0 msgr2=0x5643e0b162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd33400640 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b012f0 0x5643e0b162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd33400640 1 -- 172.21.15.90:0/4157407153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b0dba0 msgr2=0x5643e0b11000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd33400640 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b0dba0 0x5643e0b11000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd33400640 1 -- 172.21.15.90:0/4157407153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643e0b11830 con 0x5643e0b01c10 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd38c00640 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b0dba0 0x5643e0b11000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd33400640 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b10aa0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdd2002a930 tx=0x7fdd20030f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af652294f3f0ff2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:20.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd1d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd1d600640 1 -- 172.21.15.90:0/4157407153 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20044040 con 0x5643e0b01c10 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd1d600640 1 -- 172.21.15.90:0/4157407153 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fdd20030540 con 0x5643e0b01c10 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.692+0000 7fdd3d65d740 1 -- 172.21.15.90:0/4157407153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd200307d0 con 0x5643e0b01c10 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd1d600640 1 -- 172.21.15.90:0/4157407153 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdd20064050 con 0x5643e0b01c10 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd3d65d740 10 client.15840 fetch_fsmap learned FSMap version 32 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd3d65d740 1 -- 172.21.15.90:0/4157407153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5643e0b8b520 con 0x5643e0b01c10 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd1d600640 1 -- 172.21.15.90:0/4157407153 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd200307d0 con 0x5643e0b01c10 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd1d600640 1 -- 172.21.15.90:0/4157407153 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdd20003a80 con 0x5643e0b01c10 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd3d65d740 10 client.15840 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd3d65d740 10 client.15840 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd3d65d740 20 client.15840 populate_metadata read hostname 'smithi090' 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5643e0b8bd10 0x5643e0b8e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd3d65d740 4 client.15840 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}] 2024-09-10T22:06:20.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.693+0000 7fdd3d65d740 1 -- 172.21.15.90:0/4157407153 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.5, "await_for": 45.0}) -- 0x5643e0a746a0 con 0x5643e0b8bd10 2024-09-10T22:06:20.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.694+0000 7fdd38c00640 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5643e0b8bd10 0x5643e0b8e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:20.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.694+0000 7fdd38c00640 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5643e0b8bd10 0x5643e0b8e130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa16a69ac1d0b8d server_cookie=ca56ba95de901166 in_seq=0 out_seq=0 2024-09-10T22:06:20.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.694+0000 7fdd1d600640 10 client.15840 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:20.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd1d600640 1 -- 172.21.15.90:0/4157407153 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5643e0a746a0 con 0x5643e0b8bd10 2024-09-10T22:06:20.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd1d600640 10 client.15840 handle_command_reply: tid=0 2024-09-10T22:06:20.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 client.15840 shutdown 2024-09-10T22:06:20.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 -- 172.21.15.90:0/4157407153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 msgr2=0x5643e0b10aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:20.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b10aa0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdd2002a930 tx=0x7fdd20030f60 comp rx=0 tx=0).stop 2024-09-10T22:06:20.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 -- 172.21.15.90:0/4157407153 shutdown_connections 2024-09-10T22:06:20.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5643e0b8bd10 0x5643e0b8e130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5643e0b0dba0 0x5643e0b11000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5643e0b01c10 0x5643e0b10aa0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 --2- 172.21.15.90:0/4157407153 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5643e0b012f0 0x5643e0b162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd1d600640 0 client.15840 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:20.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 -- 172.21.15.90:0/4157407153 >> 172.21.15.90:0/4157407153 conn(0x5643e0aff120 msgr2=0x5643e0b09120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:20.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 -- 172.21.15.90:0/4157407153 shutdown_connections 2024-09-10T22:06:20.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 1 -- 172.21.15.90:0/4157407153 wait complete. 2024-09-10T22:06:20.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.759+0000 7fdd3d65d740 20 client.15840 trim_cache size 0 max 16384 2024-09-10T22:06:20.762 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:06:20.762 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 32, 2024-09-10T22:06:20.762 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 10, 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "9b5b926": { 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 10, 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 140.5, 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:06:20.763 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:06:20.764 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:06:20.764 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:06:20.764 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:06:20.764 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:20.764 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:20.764 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:20.764 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:20.764 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:20.764 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 -- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd180b5580 msgr2=0x7fdd180b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd180b5580 0x7fdd180b7a40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdd240e3a40 tx=0x7fdd2400c850 comp rx=0 tx=0).stop 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 -- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd34102c10 msgr2=0x7fdd3419b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd34102c10 0x7fdd3419b180 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdd2400e9c0 tx=0x7fdd2400ee80 comp rx=0 tx=0).stop 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 -- 172.21.15.90:0/285234625 shutdown_connections 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd180b5580 0x7fdd180b7a40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd34108fa0 0x7fdd341a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd34108620 0x7fdd3419b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 --2- 172.21.15.90:0/285234625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd34102c10 0x7fdd3419b180 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 -- 172.21.15.90:0/285234625 >> 172.21.15.90:0/285234625 conn(0x7fdd340fe6c0 msgr2=0x7fdd34105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:20.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 -- 172.21.15.90:0/285234625 shutdown_connections 2024-09-10T22:06:20.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:20.760+0000 7fdd39600640 1 -- 172.21.15.90:0/285234625 wait complete. 2024-09-10T22:06:20.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:20 smithi090 ceph-mon[30350]: pgmap v491: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 24 op/s 2024-09-10T22:06:20.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:20 smithi171 ceph-mon[34138]: pgmap v491: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 24 op/s 2024-09-10T22:06:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:20 smithi154 ceph-mon[34495]: pgmap v491: 129 pgs: 129 active+clean; 32 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 24 op/s 2024-09-10T22:06:21.454 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.69 MB/sec warmup 73 sec latency 760.221 ms 2024-09-10T22:06:21.958 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9b5b926', quiesce took 4.6 seconds. Will release after: 15.9 2024-09-10T22:06:22.454 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.67 MB/sec warmup 74 sec latency 1760.339 ms 2024-09-10T22:06:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:22 smithi171 ceph-mon[34138]: pgmap v492: 129 pgs: 129 active+clean; 33 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 144 KiB/s wr, 24 op/s 2024-09-10T22:06:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:22 smithi090 ceph-mon[30350]: pgmap v492: 129 pgs: 129 active+clean; 33 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 144 KiB/s wr, 24 op/s 2024-09-10T22:06:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:22 smithi154 ceph-mon[34495]: pgmap v492: 129 pgs: 129 active+clean; 33 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 144 KiB/s wr, 24 op/s 2024-09-10T22:06:23.454 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.65 MB/sec warmup 75 sec latency 2760.493 ms 2024-09-10T22:06:24.455 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.63 MB/sec warmup 76 sec latency 3760.592 ms 2024-09-10T22:06:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:24 smithi171 ceph-mon[34138]: pgmap v493: 129 pgs: 129 active+clean; 33 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 249 KiB/s wr, 43 op/s 2024-09-10T22:06:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:24 smithi090 ceph-mon[30350]: pgmap v493: 129 pgs: 129 active+clean; 33 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 249 KiB/s wr, 43 op/s 2024-09-10T22:06:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:24 smithi154 ceph-mon[34495]: pgmap v493: 129 pgs: 129 active+clean; 33 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 249 KiB/s wr, 43 op/s 2024-09-10T22:06:25.454 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.61 MB/sec warmup 77 sec latency 4760.727 ms 2024-09-10T22:06:26.454 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.58 MB/sec warmup 78 sec latency 5760.816 ms 2024-09-10T22:06:26.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:26 smithi171 ceph-mon[34138]: pgmap v494: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 299 KiB/s wr, 49 op/s 2024-09-10T22:06:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:26 smithi090 ceph-mon[30350]: pgmap v494: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 299 KiB/s wr, 49 op/s 2024-09-10T22:06:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:26 smithi154 ceph-mon[34495]: pgmap v494: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 299 KiB/s wr, 49 op/s 2024-09-10T22:06:27.454 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.56 MB/sec warmup 79 sec latency 6760.881 ms 2024-09-10T22:06:28.455 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.55 MB/sec warmup 80 sec latency 7760.973 ms 2024-09-10T22:06:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:28 smithi090 ceph-mon[30350]: pgmap v495: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 300 KiB/s wr, 47 op/s 2024-09-10T22:06:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:28 smithi154 ceph-mon[34495]: pgmap v495: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 300 KiB/s wr, 47 op/s 2024-09-10T22:06:29.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:28 smithi171 ceph-mon[34138]: pgmap v495: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 300 KiB/s wr, 47 op/s 2024-09-10T22:06:29.455 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.53 MB/sec warmup 81 sec latency 8761.306 ms 2024-09-10T22:06:30.455 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.51 MB/sec warmup 82 sec latency 9761.391 ms 2024-09-10T22:06:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:30 smithi090 ceph-mon[30350]: pgmap v496: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 305 KiB/s wr, 51 op/s 2024-09-10T22:06:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:30 smithi154 ceph-mon[34495]: pgmap v496: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 305 KiB/s wr, 51 op/s 2024-09-10T22:06:31.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:30 smithi171 ceph-mon[34138]: pgmap v496: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 305 KiB/s wr, 51 op/s 2024-09-10T22:06:31.455 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.49 MB/sec warmup 83 sec latency 10761.513 ms 2024-09-10T22:06:32.455 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.47 MB/sec warmup 84 sec latency 11761.678 ms 2024-09-10T22:06:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:32 smithi090 ceph-mon[30350]: pgmap v497: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 178 KiB/s wr, 38 op/s 2024-09-10T22:06:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:32 smithi154 ceph-mon[34495]: pgmap v497: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 178 KiB/s wr, 38 op/s 2024-09-10T22:06:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:32 smithi171 ceph-mon[34138]: pgmap v497: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 178 KiB/s wr, 38 op/s 2024-09-10T22:06:33.456 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.45 MB/sec warmup 85 sec latency 12761.823 ms 2024-09-10T22:06:34.456 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.44 MB/sec warmup 86 sec latency 13761.965 ms 2024-09-10T22:06:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:34 smithi090 ceph-mon[30350]: pgmap v498: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 163 KiB/s wr, 36 op/s 2024-09-10T22:06:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:34 smithi154 ceph-mon[34495]: pgmap v498: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 163 KiB/s wr, 36 op/s 2024-09-10T22:06:35.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:34 smithi171 ceph-mon[34138]: pgmap v498: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 163 KiB/s wr, 36 op/s 2024-09-10T22:06:35.456 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.42 MB/sec warmup 87 sec latency 14762.096 ms 2024-09-10T22:06:36.456 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.40 MB/sec warmup 88 sec latency 15762.241 ms 2024-09-10T22:06:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:36 smithi090 ceph-mon[30350]: pgmap v499: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-10T22:06:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:36 smithi154 ceph-mon[34495]: pgmap v499: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-10T22:06:37.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:36 smithi171 ceph-mon[34138]: pgmap v499: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 57 KiB/s wr, 18 op/s 2024-09-10T22:06:37.456 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.39 MB/sec warmup 89 sec latency 16762.397 ms 2024-09-10T22:06:37.859 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:06:38.236 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:06:38.456 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.37 MB/sec warmup 90 sec latency 17762.488 ms 2024-09-10T22:06:38.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.856+0000 7fb20e600640 1 -- 172.21.15.90:0/1364636549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810b180 msgr2=0x7fb208111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.856+0000 7fb20e600640 1 --2- 172.21.15.90:0/1364636549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810b180 0x7fb208111980 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb200009ca0 tx=0x7fb20002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.856+0000 7fb20e600640 1 -- 172.21.15.90:0/1364636549 shutdown_connections 2024-09-10T22:06:38.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.856+0000 7fb20e600640 1 --2- 172.21.15.90:0/1364636549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810b180 0x7fb208111980 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.856+0000 7fb20e600640 1 --2- 172.21.15.90:0/1364636549 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20810a800 0x7fb20810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.856+0000 7fb20e600640 1 --2- 172.21.15.90:0/1364636549 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb208104df0 0x7fb2081051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.856+0000 7fb20e600640 1 -- 172.21.15.90:0/1364636549 >> 172.21.15.90:0/1364636549 conn(0x7fb208100840 msgr2=0x7fb208102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:38.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.857+0000 7fb20e600640 1 -- 172.21.15.90:0/1364636549 shutdown_connections 2024-09-10T22:06:38.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.857+0000 7fb20e600640 1 -- 172.21.15.90:0/1364636549 wait complete. 2024-09-10T22:06:38.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.857+0000 7fb20e600640 1 Processor -- start 2024-09-10T22:06:38.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb20e600640 1 -- start start 2024-09-10T22:06:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb20e600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb208104df0 0x7fb20819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb20e600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810a800 0x7fb20819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb20e600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20810b180 0x7fb2081a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb20e600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb208114d40 con 0x7fb20810a800 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb20e600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb208114bc0 con 0x7fb20810b180 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb20e600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb208114ec0 con 0x7fb208104df0 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb1ffe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810a800 0x7fb20819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb1ffe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810a800 0x7fb20819b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52578/0 (socket says 172.21.15.90:52578) 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb1ffe00640 1 -- 172.21.15.90:0/3811896454 learned_addr learned my addr 172.21.15.90:0/3811896454 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.858+0000 7fb20d200640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb208104df0 0x7fb20819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb20dc00640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20810b180 0x7fb2081a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb1ffe00640 1 -- 172.21.15.90:0/3811896454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb208104df0 msgr2=0x7fb20819b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb1ffe00640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb208104df0 0x7fb20819b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb1ffe00640 1 -- 172.21.15.90:0/3811896454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20810b180 msgr2=0x7fb2081a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb1ffe00640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20810b180 0x7fb2081a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb1ffe00640 1 -- 172.21.15.90:0/3811896454 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2081a1160 con 0x7fb20810a800 2024-09-10T22:06:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb20dc00640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20810b180 0x7fb2081a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:06:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb1ffe00640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810a800 0x7fb20819b5f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000d950 tx=0x7fb1f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb206000640 1 -- 172.21.15.90:0/3811896454 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0017070 con 0x7fb20810a800 2024-09-10T22:06:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.859+0000 7fb206000640 1 -- 172.21.15.90:0/3811896454 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1f0009c30 con 0x7fb20810a800 2024-09-10T22:06:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.860+0000 7fb206000640 1 -- 172.21.15.90:0/3811896454 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f000f620 con 0x7fb20810a800 2024-09-10T22:06:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.860+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2081a1450 con 0x7fb20810a800 2024-09-10T22:06:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.860+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081a7b10 con 0x7fb20810a800 2024-09-10T22:06:38.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.861+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2081051d0 con 0x7fb20810a800 2024-09-10T22:06:38.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.863+0000 7fb206000640 1 -- 172.21.15.90:0/3811896454 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb1f0010460 con 0x7fb20810a800 2024-09-10T22:06:38.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.864+0000 7fb206000640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1ec0b5500 0x7fb1ec0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:38.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.864+0000 7fb206000640 1 -- 172.21.15.90:0/3811896454 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb1f00e5ff0 con 0x7fb20810a800 2024-09-10T22:06:38.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.864+0000 7fb20d200640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1ec0b5500 0x7fb1ec0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:38.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.865+0000 7fb20d200640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1ec0b5500 0x7fb1ec0b79c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8005fd0 tx=0x7fb1f800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:38.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:38.868+0000 7fb206000640 1 -- 172.21.15.90:0/3811896454 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb1f00ac7a0 con 0x7fb20810a800 2024-09-10T22:06:39.050 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:38 smithi171 ceph-mon[34138]: pgmap v500: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-09-10T22:06:39.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.090+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2081a7e80 con 0x7fb20810a800 2024-09-10T22:06:39.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:38 smithi090 ceph-mon[30350]: pgmap v500: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-09-10T22:06:39.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.091+0000 7fb206000640 1 -- 172.21.15.90:0/3811896454 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11993 (secure 0 0 0) 0x7fb1f0009de0 con 0x7fb20810a800 2024-09-10T22:06:39.093 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:06:39.093 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":32,"btime":"2024-09-10T22:05:28:775491+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:05:28.494576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:06:39.093 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 32 2024-09-10T22:06:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1ec0b5500 msgr2=0x7fb1ec0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1ec0b5500 0x7fb1ec0b79c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8005fd0 tx=0x7fb1f800b390 comp rx=0 tx=0).stop 2024-09-10T22:06:39.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810a800 msgr2=0x7fb20819b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810a800 0x7fb20819b5f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000d950 tx=0x7fb1f000de10 comp rx=0 tx=0).stop 2024-09-10T22:06:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 shutdown_connections 2024-09-10T22:06:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1ec0b5500 0x7fb1ec0b79c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20810b180 0x7fb2081a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20810a800 0x7fb20819b5f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 --2- 172.21.15.90:0/3811896454 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb208104df0 0x7fb20819b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.094+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 >> 172.21.15.90:0/3811896454 conn(0x7fb208100840 msgr2=0x7fb208103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.095+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 shutdown_connections 2024-09-10T22:06:39.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:39.095+0000 7fb20e600640 1 -- 172.21.15.90:0/3811896454 wait complete. 2024-09-10T22:06:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:38 smithi154 ceph-mon[34495]: pgmap v500: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-09-10T22:06:39.456 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.36 MB/sec warmup 91 sec latency 18762.655 ms 2024-09-10T22:06:40.018 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 9b5b926 --release --await' 2024-09-10T22:06:40.019 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 9b5b926 --release --await 2024-09-10T22:06:40.069 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:39 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3811896454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:06:40.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:39 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3811896454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:06:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:39 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3811896454' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:06:40.389 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:06:40.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20184 1.34 MB/sec warmup 92 sec latency 19762.726 ms 2024-09-10T22:06:40.842 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:40 smithi090 ceph-mon[30350]: pgmap v501: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-09-10T22:06:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.959+0000 7f9969c00640 1 -- 172.21.15.90:0/1810719247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 msgr2=0x7f996410ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.959+0000 7f9969c00640 1 --2- 172.21.15.90:0/1810719247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 0x7f996410ae10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f994c009900 tx=0x7f994c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.959+0000 7f9969c00640 1 -- 172.21.15.90:0/1810719247 shutdown_connections 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.959+0000 7f9969c00640 1 --2- 172.21.15.90:0/1810719247 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996410b350 0x7f9964111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.959+0000 7f9969c00640 1 --2- 172.21.15.90:0/1810719247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 0x7f996410ae10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.959+0000 7f9969c00640 1 --2- 172.21.15.90:0/1810719247 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9964104fc0 0x7f99641053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.959+0000 7f9969c00640 1 -- 172.21.15.90:0/1810719247 >> 172.21.15.90:0/1810719247 conn(0x7f99641009f0 msgr2=0x7f9964102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.959+0000 7f9969c00640 1 -- 172.21.15.90:0/1810719247 shutdown_connections 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.959+0000 7f9969c00640 1 -- 172.21.15.90:0/1810719247 wait complete. 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.960+0000 7f9969c00640 1 Processor -- start 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f9969c00640 1 -- start start 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f9969c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9964104fc0 0x7f996419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f9969c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 0x7f996419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f9969c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996410b350 0x7f99641a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f9969c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9964114e70 con 0x7f996410a9d0 2024-09-10T22:06:41.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f9969c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9964114cf0 con 0x7f9964104fc0 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f9969c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9964114ff0 con 0x7f996410b350 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f995a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 0x7f996419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f995a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 0x7f996419b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52594/0 (socket says 172.21.15.90:52594) 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f995a000640 1 -- 172.21.15.90:0/1123074493 learned_addr learned my addr 172.21.15.90:0/1123074493 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.961+0000 7f9962a00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9964104fc0 0x7f996419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f9963400640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996410b350 0x7f99641a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f995a000640 1 -- 172.21.15.90:0/1123074493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996410b350 msgr2=0x7f99641a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f995a000640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996410b350 0x7f99641a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f995a000640 1 -- 172.21.15.90:0/1123074493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9964104fc0 msgr2=0x7f996419b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f995a000640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9964104fc0 0x7f996419b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f995a000640 1 -- 172.21.15.90:0/1123074493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99641a1410 con 0x7f996410a9d0 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f9963400640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996410b350 0x7f99641a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f995a000640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 0x7f996419b8a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f994c02a910 tx=0x7f994c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f995be00640 1 -- 172.21.15.90:0/1123074493 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c03f070 con 0x7f996410a9d0 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f995be00640 1 -- 172.21.15.90:0/1123074493 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f994c003e40 con 0x7f996410a9d0 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f9969c00640 1 -- 172.21.15.90:0/1123074493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99641a16a0 con 0x7f996410a9d0 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.962+0000 7f995be00640 1 -- 172.21.15.90:0/1123074493 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c035030 con 0x7f996410a9d0 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.963+0000 7f9969c00640 1 -- 172.21.15.90:0/1123074493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99641a7dc0 con 0x7f996410a9d0 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.966+0000 7f996c613740 1 Processor -- start 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f996c613740 1 -- start start 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f996c613740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2ba5ff0 0x55ede2ba63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f996c613740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 0x55ede2bb2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f996c613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2bb28a0 0x55ede2bb4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f996c613740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ede2b1d290 con 0x55ede2ba5ff0 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f996c613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ede2b0c6c0 con 0x55ede2bb28a0 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f996c613740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ede2b104d0 con 0x55ede2ba6910 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f995be00640 1 -- 172.21.15.90:0/1123074493 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f994c031400 con 0x7f996410a9d0 2024-09-10T22:06:41.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f9963400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2ba5ff0 0x55ede2ba63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f9963400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2ba5ff0 0x55ede2ba63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52600/0 (socket says 172.21.15.90:52600) 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.967+0000 7f9963400640 1 -- 172.21.15.90:0/2070995690 learned_addr learned my addr 172.21.15.90:0/2070995690 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.968+0000 7f9962a00640 1 --2- 172.21.15.90:0/2070995690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 0x55ede2bb2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.968+0000 7f995a000640 1 --2- 172.21.15.90:0/2070995690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2bb28a0 0x55ede2bb4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.968+0000 7f9963400640 1 -- 172.21.15.90:0/2070995690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 msgr2=0x55ede2bb2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.968+0000 7f9963400640 1 --2- 172.21.15.90:0/2070995690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 0x55ede2bb2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.968+0000 7f9963400640 1 -- 172.21.15.90:0/2070995690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2bb28a0 msgr2=0x55ede2bb4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.968+0000 7f9963400640 1 --2- 172.21.15.90:0/2070995690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2bb28a0 0x55ede2bb4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.968+0000 7f9963400640 1 -- 172.21.15.90:0/2070995690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede2a99460 con 0x55ede2ba5ff0 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.968+0000 7f9963400640 1 --2- 172.21.15.90:0/2070995690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2ba5ff0 0x55ede2ba63d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f995c008e90 tx=0x7f995c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c95e23596167b97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.968+0000 7f9953e00640 1 -- 172.21.15.90:0/2070995690 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00f830 con 0x55ede2ba5ff0 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f9953e00640 1 -- 172.21.15.90:0/2070995690 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f995c018480 con 0x55ede2ba5ff0 2024-09-10T22:06:41.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f9953e00640 1 -- 172.21.15.90:0/2070995690 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c00fe60 con 0x55ede2ba5ff0 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f995be00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99480b5450 0x7f99480b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f995be00640 1 -- 172.21.15.90:0/1123074493 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f994c106ea0 con 0x7f996410a9d0 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f9962a00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99480b5450 0x7f99480b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f996c613740 1 -- 172.21.15.90:0/2070995690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2ba5ff0 msgr2=0x55ede2ba63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f996c613740 1 --2- 172.21.15.90:0/2070995690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2ba5ff0 0x55ede2ba63d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f995c008e90 tx=0x7f995c00d710 comp rx=0 tx=0).stop 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f996c613740 1 -- 172.21.15.90:0/2070995690 shutdown_connections 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f996c613740 1 --2- 172.21.15.90:0/2070995690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2bb28a0 0x55ede2bb4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f996c613740 1 --2- 172.21.15.90:0/2070995690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 0x55ede2bb2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.969+0000 7f996c613740 1 --2- 172.21.15.90:0/2070995690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2ba5ff0 0x55ede2ba63d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.970+0000 7f996c613740 1 -- 172.21.15.90:0/2070995690 >> 172.21.15.90:0/2070995690 conn(0x55ede2ba3e20 msgr2=0x55ede2ba4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.970+0000 7f996c613740 1 -- 172.21.15.90:0/2070995690 shutdown_connections 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.970+0000 7f996c613740 1 -- 172.21.15.90:0/2070995690 wait complete. 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.970+0000 7f9962a00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99480b5450 0x7f99480b7910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55ede2ba9170 tx=0x7f995400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.970+0000 7f996c613740 1 -- start start 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f996c613740 1 Processor -- start 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f996c613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba5ff0 0x55ede2baf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f996c613740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 0x55ede2bafd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f996c613740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f996c613740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ede296dbb0 con 0x55ede2bb28a0 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f996c613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ede2aa76c0 con 0x55ede2ba5ff0 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f996c613740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ede2bb5550 con 0x55ede2ba6910 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f996c613740 10 client.0 _ll_register_callbacks cb 0x55ede2ac2490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f9963400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f995a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 0x55ede2bafd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f9963400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52606/0 (socket says 172.21.15.90:52606) 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f995a000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 0x55ede2bafd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:55592/0 (socket says 172.21.15.90:55592) 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.971+0000 7f9963400640 1 -- 172.21.15.90:0/2418360441 learned_addr learned my addr 172.21.15.90:0/2418360441 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:41.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9962a00640 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba5ff0 0x55ede2baf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9963400640 1 -- 172.21.15.90:0/2418360441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 msgr2=0x55ede2bafd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9963400640 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 0x55ede2bafd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9963400640 1 -- 172.21.15.90:0/2418360441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba5ff0 msgr2=0x55ede2baf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9963400640 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba5ff0 0x55ede2baf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9963400640 1 -- 172.21.15.90:0/2418360441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede286ab00 con 0x55ede2bb28a0 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9963400640 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb0290 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f995c018e20 tx=0x7f995c008ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd305d410ff4266 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9950c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9950c00640 1 -- 172.21.15.90:0/2418360441 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c028060 con 0x55ede2bb28a0 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f9950c00640 1 -- 172.21.15.90:0/2418360441 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f995c004c50 con 0x55ede2bb28a0 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.972+0000 7f996c613740 1 -- 172.21.15.90:0/2418360441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995c018480 con 0x55ede2bb28a0 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f9950c00640 1 -- 172.21.15.90:0/2418360441 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f995c02f050 con 0x55ede2bb28a0 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f996c613740 10 client.15858 fetch_fsmap learned FSMap version 32 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f996c613740 1 -- 172.21.15.90:0/2418360441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ede28e1960 con 0x55ede2bb28a0 2024-09-10T22:06:41.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f9950c00640 1 -- 172.21.15.90:0/2418360441 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f995c018480 con 0x55ede2bb28a0 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f9950c00640 1 -- 172.21.15.90:0/2418360441 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f995c0050f0 con 0x55ede2bb28a0 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f996c613740 10 client.15858 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f996c613740 10 client.15858 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f996c613740 20 client.15858 populate_metadata read hostname 'smithi090' 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f996c613740 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ede2bc2370 0x55ede2bc4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f996c613740 4 client.15858 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.973+0000 7f996c613740 1 -- 172.21.15.90:0/2418360441 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ede2b1f1e0 con 0x55ede2bc2370 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.974+0000 7f9962a00640 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ede2bc2370 0x55ede2bc4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.974+0000 7f9962a00640 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ede2bc2370 0x55ede2bc4760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be36ae3e76fb29de server_cookie=e694b68bc069e57a in_seq=0 out_seq=0 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.974+0000 7f9950c00640 10 client.15858 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.975+0000 7f9950c00640 1 -- 172.21.15.90:0/2418360441 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ede2b1f1e0 con 0x55ede2bc2370 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.975+0000 7f9950c00640 10 client.15858 handle_command_reply: tid=0 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.975+0000 7f996c613740 1 client.15858 shutdown 2024-09-10T22:06:41.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.975+0000 7f996c613740 1 -- 172.21.15.90:0/2418360441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 msgr2=0x55ede2bb0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.975+0000 7f996c613740 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb0290 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f995c018e20 tx=0x7f995c008ba0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.975+0000 7f996c613740 1 -- 172.21.15.90:0/2418360441 shutdown_connections 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.975+0000 7f996c613740 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ede2bc2370 0x55ede2bc4760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.976+0000 7f996c613740 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb0290 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.976+0000 7f996c613740 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba6910 0x55ede2bafd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.976+0000 7f996c613740 1 --2- 172.21.15.90:0/2418360441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba5ff0 0x55ede2baf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.976+0000 7f996c613740 1 -- 172.21.15.90:0/2418360441 >> 172.21.15.90:0/2418360441 conn(0x55ede2ba3e20 msgr2=0x55ede2ba99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.976+0000 7f9950c00640 0 client.15858 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.976+0000 7f996c613740 1 -- 172.21.15.90:0/2418360441 shutdown_connections 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.976+0000 7f996c613740 1 -- 172.21.15.90:0/2418360441 wait complete. 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.976+0000 7f996c613740 20 client.15858 trim_cache size 0 max 16384 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.993+0000 7f996c613740 1 Processor -- start 2024-09-10T22:06:41.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f996c613740 1 -- start start 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f996c613740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 0x55ede2bc47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f996c613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 0x55ede2bb1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f996c613740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2baca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f996c613740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99641b1e90 con 0x55ede2bb28a0 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f996c613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f99641b3a90 con 0x55ede2ba6910 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f996c613740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99641b2560 con 0x55ede2ba5ff0 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f9963400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2baca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f9963400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2baca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52618/0 (socket says 172.21.15.90:52618) 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f9963400640 1 -- 172.21.15.90:0/2015867989 learned_addr learned my addr 172.21.15.90:0/2015867989 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f9962a00640 1 --2- 172.21.15.90:0/2015867989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 0x55ede2bc47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f995a000640 1 --2- 172.21.15.90:0/2015867989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 0x55ede2bb1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f9963400640 1 -- 172.21.15.90:0/2015867989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 msgr2=0x55ede2bc47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f9963400640 1 --2- 172.21.15.90:0/2015867989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 0x55ede2bc47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.994+0000 7f9963400640 1 -- 172.21.15.90:0/2015867989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 msgr2=0x55ede2bb1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.995+0000 7f9963400640 1 --2- 172.21.15.90:0/2015867989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 0x55ede2bb1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.995+0000 7f9963400640 1 -- 172.21.15.90:0/2015867989 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ede2ba6740 con 0x55ede2bb28a0 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.995+0000 7f9963400640 1 --2- 172.21.15.90:0/2015867989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2baca20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f995c008d30 tx=0x7f995c005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a838a57b9d10c1f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.995+0000 7f9953400640 1 -- 172.21.15.90:0/2015867989 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c028040 con 0x55ede2bb28a0 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.995+0000 7f9953400640 1 -- 172.21.15.90:0/2015867989 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f995c018e50 con 0x55ede2bb28a0 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.995+0000 7f9953400640 1 -- 172.21.15.90:0/2015867989 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c0189d0 con 0x55ede2bb28a0 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 -- 172.21.15.90:0/2015867989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 msgr2=0x55ede2baca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 --2- 172.21.15.90:0/2015867989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2baca20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f995c008d30 tx=0x7f995c005680 comp rx=0 tx=0).stop 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 -- 172.21.15.90:0/2015867989 shutdown_connections 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 --2- 172.21.15.90:0/2015867989 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2baca20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 --2- 172.21.15.90:0/2015867989 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 0x55ede2bb1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 --2- 172.21.15.90:0/2015867989 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 0x55ede2bc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 -- 172.21.15.90:0/2015867989 >> 172.21.15.90:0/2015867989 conn(0x55ede2ba3e20 msgr2=0x55ede2bb0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 -- 172.21.15.90:0/2015867989 shutdown_connections 2024-09-10T22:06:41.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 -- 172.21.15.90:0/2015867989 wait complete. 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 -- start start 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.996+0000 7f996c613740 1 Processor -- start 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f996c613740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 0x55ede2bbada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f996c613740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 0x55ede2bb7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f996c613740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f996c613740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99641b3300 con 0x55ede2bb28a0 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f996c613740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f99641b2c30 con 0x55ede2ba6910 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f996c613740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99641afe70 con 0x55ede2ba5ff0 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f996c613740 10 client.0 _ll_register_callbacks cb 0x7f9954004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f9963400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f9963400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52634/0 (socket says 172.21.15.90:52634) 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f9963400640 1 -- 172.21.15.90:0/1616033388 learned_addr learned my addr 172.21.15.90:0/1616033388 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:06:41.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f9962a00640 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 0x55ede2bbada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f9963400640 1 -- 172.21.15.90:0/1616033388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 msgr2=0x55ede2bbada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f995a000640 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 0x55ede2bb7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f9963400640 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 0x55ede2bbada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f9963400640 1 -- 172.21.15.90:0/1616033388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 msgr2=0x55ede2bb7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f9963400640 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 0x55ede2bb7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.997+0000 7f9963400640 1 -- 172.21.15.90:0/1616033388 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ede2bb7dd0 con 0x55ede2bb28a0 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.998+0000 7f9963400640 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb77c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f995c0185c0 tx=0x7f995c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610244fc33f69098 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.998+0000 7f9950c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.998+0000 7f9950c00640 1 -- 172.21.15.90:0/1616033388 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c028070 con 0x55ede2bb28a0 2024-09-10T22:06:41.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.998+0000 7f9950c00640 1 -- 172.21.15.90:0/1616033388 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f995c0189d0 con 0x55ede2bb28a0 2024-09-10T22:06:41.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.998+0000 7f996c613740 1 -- 172.21.15.90:0/1616033388 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995c018e50 con 0x55ede2bb28a0 2024-09-10T22:06:41.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.998+0000 7f9950c00640 1 -- 172.21.15.90:0/1616033388 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f995c043050 con 0x55ede2bb28a0 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.998+0000 7f996c613740 10 client.15864 fetch_fsmap learned FSMap version 32 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.998+0000 7f996c613740 1 -- 172.21.15.90:0/1616033388 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ede2c2fec0 con 0x55ede2bb28a0 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.999+0000 7f9950c00640 1 -- 172.21.15.90:0/1616033388 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f995c018e50 con 0x55ede2bb28a0 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.999+0000 7f9950c00640 1 -- 172.21.15.90:0/1616033388 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f995c00fb30 con 0x55ede2bb28a0 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.999+0000 7f996c613740 10 client.15864 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.999+0000 7f996c613740 10 client.15864 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.999+0000 7f996c613740 20 client.15864 populate_metadata read hostname 'smithi090' 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.999+0000 7f996c613740 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ede2c30600 0x55ede2c32a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.999+0000 7f996c613740 4 client.15864 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9b5b926", "release": true, "await": true}] 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.999+0000 7f996c613740 1 -- 172.21.15.90:0/1616033388 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9b5b926", "release": true, "await": true}) -- 0x55ede2b1ac90 con 0x55ede2c30600 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:40.999+0000 7f9962a00640 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ede2c30600 0x55ede2c32a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.000+0000 7f9962a00640 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ede2c30600 0x55ede2c32a20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7051eabde2ced061 server_cookie=6e9995cf9010362c in_seq=0 out_seq=0 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.000+0000 7f9950c00640 10 client.15864 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:41.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.006+0000 7f9950c00640 1 -- 172.21.15.90:0/1616033388 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ede2b1ac90 con 0x55ede2c30600 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.006+0000 7f9950c00640 10 client.15864 handle_command_reply: tid=0 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.006+0000 7f996c613740 1 client.15864 shutdown 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 -- 172.21.15.90:0/1616033388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 msgr2=0x55ede2bb77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb77c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f995c0185c0 tx=0x7f995c00dc10 comp rx=0 tx=0).stop 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 -- 172.21.15.90:0/1616033388 shutdown_connections 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ede2c30600 0x55ede2c32a20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ede2bb28a0 0x55ede2bb77c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ede2ba6910 0x55ede2bb7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 --2- 172.21.15.90:0/1616033388 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ede2ba5ff0 0x55ede2bbada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 -- 172.21.15.90:0/1616033388 >> 172.21.15.90:0/1616033388 conn(0x55ede2ba3e20 msgr2=0x55ede2ba2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f9950c00640 0 client.15864 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:06:41.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 -- 172.21.15.90:0/1616033388 shutdown_connections 2024-09-10T22:06:41.030 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:06:41.030 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 32, 2024-09-10T22:06:41.030 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:06:41.030 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 12, 2024-09-10T22:06:41.030 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:06:41.030 INFO:teuthology.orchestra.run.smithi090.stdout: "9b5b926": { 2024-09-10T22:06:41.030 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 12, 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 140.5, 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:06:41.031 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:41.032 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:41.032 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:41.032 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:41.032 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:06:41.032 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:06:41.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 1 -- 172.21.15.90:0/1616033388 wait complete. 2024-09-10T22:06:41.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.007+0000 7f996c613740 20 client.15864 trim_cache size 0 max 16384 2024-09-10T22:06:41.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 -- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99480b5450 msgr2=0x7f99480b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99480b5450 0x7f99480b7910 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55ede2ba9170 tx=0x7f995400b390 comp rx=0 tx=0).stop 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 -- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 msgr2=0x7f996419b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 0x7f996419b8a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f994c02a910 tx=0x7f994c003c20 comp rx=0 tx=0).stop 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 -- 172.21.15.90:0/1123074493 shutdown_connections 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99480b5450 0x7f99480b7910 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996410b350 0x7f99641a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996410a9d0 0x7f996419b8a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 --2- 172.21.15.90:0/1123074493 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9964104fc0 0x7f996419b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 -- 172.21.15.90:0/1123074493 >> 172.21.15.90:0/1123074493 conn(0x7f99641009f0 msgr2=0x7f9964102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 -- 172.21.15.90:0/1123074493 shutdown_connections 2024-09-10T22:06:41.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:06:41.008+0000 7f9969c00640 1 -- 172.21.15.90:0/1123074493 wait complete. 2024-09-10T22:06:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:40 smithi154 ceph-mon[34495]: pgmap v501: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-09-10T22:06:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:40 smithi171 ceph-mon[34138]: pgmap v501: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.1 KiB/s wr, 12 op/s 2024-09-10T22:06:41.456 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20332 1.34 MB/sec warmup 93 sec latency 20317.119 ms 2024-09-10T22:06:41.997 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9b5b926', total seconds elapsed: 24.6 2024-09-10T22:06:42.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 20793 1.37 MB/sec warmup 94 sec latency 18.186 ms 2024-09-10T22:06:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:42 smithi090 ceph-mon[30350]: pgmap v502: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-10T22:06:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:42 smithi154 ceph-mon[34495]: pgmap v502: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-10T22:06:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:42 smithi171 ceph-mon[34138]: pgmap v502: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-09-10T22:06:43.456 INFO:tasks.workunit.client.0.smithi090.stdout: 1 21122 1.38 MB/sec warmup 95 sec latency 20.139 ms 2024-09-10T22:06:44.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 21537 1.41 MB/sec warmup 96 sec latency 21.537 ms 2024-09-10T22:06:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:44 smithi090 ceph-mon[30350]: pgmap v503: 129 pgs: 129 active+clean; 35 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 144 KiB/s wr, 22 op/s 2024-09-10T22:06:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:44 smithi154 ceph-mon[34495]: pgmap v503: 129 pgs: 129 active+clean; 35 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 144 KiB/s wr, 22 op/s 2024-09-10T22:06:45.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:44 smithi171 ceph-mon[34138]: pgmap v503: 129 pgs: 129 active+clean; 35 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 144 KiB/s wr, 22 op/s 2024-09-10T22:06:45.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 21710 1.40 MB/sec warmup 97 sec latency 18.612 ms 2024-09-10T22:06:46.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 21899 1.39 MB/sec warmup 98 sec latency 21.038 ms 2024-09-10T22:06:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:46 smithi090 ceph-mon[30350]: pgmap v504: 129 pgs: 129 active+clean; 36 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 319 KiB/s wr, 24 op/s 2024-09-10T22:06:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:46 smithi154 ceph-mon[34495]: pgmap v504: 129 pgs: 129 active+clean; 36 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 319 KiB/s wr, 24 op/s 2024-09-10T22:06:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:46 smithi171 ceph-mon[34138]: pgmap v504: 129 pgs: 129 active+clean; 36 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 319 KiB/s wr, 24 op/s 2024-09-10T22:06:47.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 22233 1.38 MB/sec warmup 99 sec latency 17.241 ms 2024-09-10T22:06:48.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 22575 1.38 MB/sec warmup 100 sec latency 17.322 ms 2024-09-10T22:06:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:48 smithi090 ceph-mon[30350]: pgmap v505: 129 pgs: 129 active+clean; 36 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 328 KiB/s wr, 23 op/s 2024-09-10T22:06:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:48 smithi154 ceph-mon[34495]: pgmap v505: 129 pgs: 129 active+clean; 36 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 328 KiB/s wr, 23 op/s 2024-09-10T22:06:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:48 smithi171 ceph-mon[34138]: pgmap v505: 129 pgs: 129 active+clean; 36 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 328 KiB/s wr, 23 op/s 2024-09-10T22:06:49.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 22974 1.40 MB/sec warmup 101 sec latency 18.060 ms 2024-09-10T22:06:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:06:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:06:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:06:50.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:06:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:06:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:06:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:06:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:06:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:06:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:06:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:06:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:06:50.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 23408 1.39 MB/sec warmup 102 sec latency 21.335 ms 2024-09-10T22:06:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:50 smithi090 ceph-mon[30350]: pgmap v506: 129 pgs: 129 active+clean; 37 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 618 KiB/s wr, 62 op/s 2024-09-10T22:06:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:50 smithi154 ceph-mon[34495]: pgmap v506: 129 pgs: 129 active+clean; 37 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 618 KiB/s wr, 62 op/s 2024-09-10T22:06:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:50 smithi171 ceph-mon[34138]: pgmap v506: 129 pgs: 129 active+clean; 37 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 618 KiB/s wr, 62 op/s 2024-09-10T22:06:51.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 23767 1.39 MB/sec warmup 103 sec latency 21.753 ms 2024-09-10T22:06:52.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 24254 1.43 MB/sec warmup 104 sec latency 20.312 ms 2024-09-10T22:06:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:52 smithi090 ceph-mon[30350]: pgmap v507: 129 pgs: 129 active+clean; 37 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 632 KiB/s wr, 68 op/s 2024-09-10T22:06:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:52 smithi154 ceph-mon[34495]: pgmap v507: 129 pgs: 129 active+clean; 37 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 632 KiB/s wr, 68 op/s 2024-09-10T22:06:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:52 smithi171 ceph-mon[34138]: pgmap v507: 129 pgs: 129 active+clean; 37 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 632 KiB/s wr, 68 op/s 2024-09-10T22:06:53.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 24640 1.43 MB/sec warmup 105 sec latency 20.254 ms 2024-09-10T22:06:54.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 24981 1.45 MB/sec warmup 106 sec latency 19.791 ms 2024-09-10T22:06:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:54 smithi090 ceph-mon[30350]: pgmap v508: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 884 KiB/s wr, 98 op/s 2024-09-10T22:06:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:54 smithi154 ceph-mon[34495]: pgmap v508: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 884 KiB/s wr, 98 op/s 2024-09-10T22:06:55.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:54 smithi171 ceph-mon[34138]: pgmap v508: 129 pgs: 129 active+clean; 37 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 884 KiB/s wr, 98 op/s 2024-09-10T22:06:55.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 25198 1.44 MB/sec warmup 107 sec latency 16.462 ms 2024-09-10T22:06:56.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 25375 1.43 MB/sec warmup 108 sec latency 18.332 ms 2024-09-10T22:06:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:56 smithi090 ceph-mon[30350]: pgmap v509: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 952 KiB/s wr, 95 op/s 2024-09-10T22:06:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:56 smithi154 ceph-mon[34495]: pgmap v509: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 952 KiB/s wr, 95 op/s 2024-09-10T22:06:57.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:56 smithi171 ceph-mon[34138]: pgmap v509: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 952 KiB/s wr, 95 op/s 2024-09-10T22:06:57.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 25664 1.42 MB/sec warmup 109 sec latency 19.618 ms 2024-09-10T22:06:58.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 26018 1.43 MB/sec warmup 110 sec latency 18.186 ms 2024-09-10T22:06:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:06:58 smithi090 ceph-mon[30350]: pgmap v510: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 784 KiB/s wr, 92 op/s 2024-09-10T22:06:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:06:58 smithi154 ceph-mon[34495]: pgmap v510: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 784 KiB/s wr, 92 op/s 2024-09-10T22:06:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:06:58 smithi171 ceph-mon[34138]: pgmap v510: 129 pgs: 129 active+clean; 38 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 784 KiB/s wr, 92 op/s 2024-09-10T22:06:59.457 INFO:tasks.workunit.client.0.smithi090.stdout: 1 26406 1.44 MB/sec warmup 111 sec latency 25.363 ms 2024-09-10T22:07:00.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 26793 1.43 MB/sec warmup 112 sec latency 24.475 ms 2024-09-10T22:07:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:00 smithi090 ceph-mon[30350]: pgmap v511: 129 pgs: 129 active+clean; 39 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T22:07:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:00 smithi154 ceph-mon[34495]: pgmap v511: 129 pgs: 129 active+clean; 39 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T22:07:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:00 smithi171 ceph-mon[34138]: pgmap v511: 129 pgs: 129 active+clean; 39 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T22:07:01.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 27237 1.43 MB/sec warmup 113 sec latency 20.131 ms 2024-09-10T22:07:02.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 27495 1.43 MB/sec warmup 114 sec latency 21.760 ms 2024-09-10T22:07:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:02 smithi090 ceph-mon[30350]: pgmap v512: 129 pgs: 129 active+clean; 39 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 848 KiB/s wr, 99 op/s 2024-09-10T22:07:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:02 smithi154 ceph-mon[34495]: pgmap v512: 129 pgs: 129 active+clean; 39 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 848 KiB/s wr, 99 op/s 2024-09-10T22:07:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:02 smithi171 ceph-mon[34138]: pgmap v512: 129 pgs: 129 active+clean; 39 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 848 KiB/s wr, 99 op/s 2024-09-10T22:07:03.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 28151 1.47 MB/sec warmup 115 sec latency 18.426 ms 2024-09-10T22:07:04.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 28374 1.48 MB/sec warmup 116 sec latency 24.567 ms 2024-09-10T22:07:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:04 smithi090 ceph-mon[30350]: pgmap v513: 129 pgs: 129 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T22:07:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:04 smithi154 ceph-mon[34495]: pgmap v513: 129 pgs: 129 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T22:07:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:04 smithi171 ceph-mon[34138]: pgmap v513: 129 pgs: 129 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T22:07:05.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 28672 1.48 MB/sec warmup 117 sec latency 19.107 ms 2024-09-10T22:07:06.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 28854 1.47 MB/sec warmup 118 sec latency 19.949 ms 2024-09-10T22:07:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:06 smithi090 ceph-mon[30350]: pgmap v514: 129 pgs: 129 active+clean; 40 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1014 KiB/s wr, 113 op/s 2024-09-10T22:07:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:06 smithi154 ceph-mon[34495]: pgmap v514: 129 pgs: 129 active+clean; 40 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1014 KiB/s wr, 113 op/s 2024-09-10T22:07:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:06 smithi171 ceph-mon[34138]: pgmap v514: 129 pgs: 129 active+clean; 40 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 1014 KiB/s wr, 113 op/s 2024-09-10T22:07:07.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 29081 1.46 MB/sec warmup 119 sec latency 18.654 ms 2024-09-10T22:07:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:08 smithi090 ceph-mon[30350]: pgmap v515: 129 pgs: 129 active+clean; 41 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 854 KiB/s wr, 115 op/s 2024-09-10T22:07:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:08 smithi154 ceph-mon[34495]: pgmap v515: 129 pgs: 129 active+clean; 41 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 854 KiB/s wr, 115 op/s 2024-09-10T22:07:09.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:08 smithi171 ceph-mon[34138]: pgmap v515: 129 pgs: 129 active+clean; 41 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 854 KiB/s wr, 115 op/s 2024-09-10T22:07:09.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 29834 3.69 MB/sec execute 1 sec latency 22.711 ms 2024-09-10T22:07:10.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 30243 2.01 MB/sec execute 2 sec latency 20.224 ms 2024-09-10T22:07:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:10 smithi090 ceph-mon[30350]: pgmap v516: 129 pgs: 129 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-09-10T22:07:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:10 smithi154 ceph-mon[34495]: pgmap v516: 129 pgs: 129 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-09-10T22:07:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:10 smithi171 ceph-mon[34138]: pgmap v516: 129 pgs: 129 active+clean; 42 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-09-10T22:07:11.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 30682 1.49 MB/sec execute 3 sec latency 19.811 ms 2024-09-10T22:07:12.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 30994 1.66 MB/sec execute 4 sec latency 18.506 ms 2024-09-10T22:07:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:12 smithi090 ceph-mon[30350]: pgmap v517: 129 pgs: 129 active+clean; 43 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 906 KiB/s wr, 121 op/s 2024-09-10T22:07:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:12 smithi154 ceph-mon[34495]: pgmap v517: 129 pgs: 129 active+clean; 43 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 906 KiB/s wr, 121 op/s 2024-09-10T22:07:13.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:12 smithi171 ceph-mon[34138]: pgmap v517: 129 pgs: 129 active+clean; 43 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 906 KiB/s wr, 121 op/s 2024-09-10T22:07:13.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 31353 2.23 MB/sec execute 5 sec latency 357.925 ms 2024-09-10T22:07:14.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 31738 2.17 MB/sec execute 6 sec latency 370.218 ms 2024-09-10T22:07:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:14 smithi090 ceph-mon[30350]: pgmap v518: 129 pgs: 129 active+clean; 43 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-10T22:07:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:14 smithi154 ceph-mon[34495]: pgmap v518: 129 pgs: 129 active+clean; 43 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-10T22:07:15.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:14 smithi171 ceph-mon[34138]: pgmap v518: 129 pgs: 129 active+clean; 43 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-09-10T22:07:15.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 32093 2.35 MB/sec execute 7 sec latency 20.226 ms 2024-09-10T22:07:16.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 32320 2.22 MB/sec execute 8 sec latency 21.438 ms 2024-09-10T22:07:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:16 smithi090 ceph-mon[30350]: pgmap v519: 129 pgs: 129 active+clean; 43 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.0 MiB/s wr, 110 op/s 2024-09-10T22:07:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:16 smithi154 ceph-mon[34495]: pgmap v519: 129 pgs: 129 active+clean; 43 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.0 MiB/s wr, 110 op/s 2024-09-10T22:07:17.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:16 smithi171 ceph-mon[34138]: pgmap v519: 129 pgs: 129 active+clean; 43 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.0 MiB/s wr, 110 op/s 2024-09-10T22:07:17.458 INFO:tasks.workunit.client.0.smithi090.stdout: 1 32508 1.97 MB/sec execute 9 sec latency 17.208 ms 2024-09-10T22:07:18.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 32798 1.83 MB/sec execute 10 sec latency 21.015 ms 2024-09-10T22:07:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:18 smithi090 ceph-mon[30350]: pgmap v520: 129 pgs: 129 active+clean; 44 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 917 KiB/s wr, 109 op/s 2024-09-10T22:07:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:18 smithi154 ceph-mon[34495]: pgmap v520: 129 pgs: 129 active+clean; 44 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 917 KiB/s wr, 109 op/s 2024-09-10T22:07:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:18 smithi171 ceph-mon[34138]: pgmap v520: 129 pgs: 129 active+clean; 44 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 917 KiB/s wr, 109 op/s 2024-09-10T22:07:19.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 33144 1.80 MB/sec execute 11 sec latency 22.722 ms 2024-09-10T22:07:20.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 33559 1.91 MB/sec execute 12 sec latency 20.501 ms 2024-09-10T22:07:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:20 smithi090 ceph-mon[30350]: pgmap v521: 129 pgs: 129 active+clean; 45 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-09-10T22:07:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:20 smithi154 ceph-mon[34495]: pgmap v521: 129 pgs: 129 active+clean; 45 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-09-10T22:07:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:20 smithi171 ceph-mon[34138]: pgmap v521: 129 pgs: 129 active+clean; 45 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-09-10T22:07:21.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 33968 1.79 MB/sec execute 13 sec latency 20.760 ms 2024-09-10T22:07:22.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 34364 1.77 MB/sec execute 14 sec latency 20.007 ms 2024-09-10T22:07:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:22 smithi090 ceph-mon[30350]: pgmap v522: 129 pgs: 129 active+clean; 46 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 970 KiB/s wr, 122 op/s 2024-09-10T22:07:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:22 smithi154 ceph-mon[34495]: pgmap v522: 129 pgs: 129 active+clean; 46 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 970 KiB/s wr, 122 op/s 2024-09-10T22:07:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:22 smithi171 ceph-mon[34138]: pgmap v522: 129 pgs: 129 active+clean; 46 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 970 KiB/s wr, 122 op/s 2024-09-10T22:07:23.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 34836 1.93 MB/sec execute 15 sec latency 18.721 ms 2024-09-10T22:07:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:23 smithi154 ceph-mon[34495]: pgmap v523: 129 pgs: 129 active+clean; 46 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1010 KiB/s wr, 109 op/s 2024-09-10T22:07:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:23 smithi171 ceph-mon[34138]: pgmap v523: 129 pgs: 129 active+clean; 46 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1010 KiB/s wr, 109 op/s 2024-09-10T22:07:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:23 smithi090 ceph-mon[30350]: pgmap v523: 129 pgs: 129 active+clean; 46 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1010 KiB/s wr, 109 op/s 2024-09-10T22:07:24.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 35244 2.01 MB/sec execute 16 sec latency 193.024 ms 2024-09-10T22:07:25.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 35492 2.07 MB/sec execute 17 sec latency 20.023 ms 2024-09-10T22:07:26.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 35775 2.03 MB/sec execute 18 sec latency 18.720 ms 2024-09-10T22:07:27.058 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:26 smithi090 ceph-mon[30350]: pgmap v524: 129 pgs: 129 active+clean; 45 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1015 KiB/s wr, 113 op/s 2024-09-10T22:07:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:26 smithi154 ceph-mon[34495]: pgmap v524: 129 pgs: 129 active+clean; 45 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1015 KiB/s wr, 113 op/s 2024-09-10T22:07:27.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:26 smithi171 ceph-mon[34138]: pgmap v524: 129 pgs: 129 active+clean; 45 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1015 KiB/s wr, 113 op/s 2024-09-10T22:07:27.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 35950 1.94 MB/sec execute 19 sec latency 17.843 ms 2024-09-10T22:07:28.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 36172 1.85 MB/sec execute 20 sec latency 16.979 ms 2024-09-10T22:07:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:28 smithi090 ceph-mon[30350]: pgmap v525: 129 pgs: 129 active+clean; 46 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 845 KiB/s wr, 102 op/s 2024-09-10T22:07:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:28 smithi154 ceph-mon[34495]: pgmap v525: 129 pgs: 129 active+clean; 46 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 845 KiB/s wr, 102 op/s 2024-09-10T22:07:29.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:28 smithi171 ceph-mon[34138]: pgmap v525: 129 pgs: 129 active+clean; 46 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 845 KiB/s wr, 102 op/s 2024-09-10T22:07:29.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 36546 1.85 MB/sec execute 21 sec latency 23.255 ms 2024-09-10T22:07:30.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 36912 1.91 MB/sec execute 22 sec latency 22.550 ms 2024-09-10T22:07:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:30 smithi090 ceph-mon[30350]: pgmap v526: 129 pgs: 129 active+clean; 47 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-09-10T22:07:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:30 smithi154 ceph-mon[34495]: pgmap v526: 129 pgs: 129 active+clean; 47 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-09-10T22:07:31.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:30 smithi171 ceph-mon[34138]: pgmap v526: 129 pgs: 129 active+clean; 47 MiB data, 577 MiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-09-10T22:07:31.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 37316 1.84 MB/sec execute 23 sec latency 21.209 ms 2024-09-10T22:07:32.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 37728 1.78 MB/sec execute 24 sec latency 18.786 ms 2024-09-10T22:07:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:32 smithi090 ceph-mon[30350]: pgmap v527: 129 pgs: 129 active+clean; 48 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 931 KiB/s wr, 123 op/s 2024-09-10T22:07:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:32 smithi154 ceph-mon[34495]: pgmap v527: 129 pgs: 129 active+clean; 48 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 931 KiB/s wr, 123 op/s 2024-09-10T22:07:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:32 smithi171 ceph-mon[34138]: pgmap v527: 129 pgs: 129 active+clean; 48 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 931 KiB/s wr, 123 op/s 2024-09-10T22:07:33.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 38066 1.79 MB/sec execute 25 sec latency 21.172 ms 2024-09-10T22:07:34.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 38525 1.90 MB/sec execute 26 sec latency 21.304 ms 2024-09-10T22:07:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:34 smithi090 ceph-mon[30350]: pgmap v528: 129 pgs: 129 active+clean; 47 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 950 KiB/s wr, 110 op/s 2024-09-10T22:07:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:34 smithi154 ceph-mon[34495]: pgmap v528: 129 pgs: 129 active+clean; 47 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 950 KiB/s wr, 110 op/s 2024-09-10T22:07:35.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:34 smithi171 ceph-mon[34138]: pgmap v528: 129 pgs: 129 active+clean; 47 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 950 KiB/s wr, 110 op/s 2024-09-10T22:07:35.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 38863 1.90 MB/sec execute 27 sec latency 16.020 ms 2024-09-10T22:07:36.459 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39271 1.99 MB/sec execute 28 sec latency 27.978 ms 2024-09-10T22:07:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:36 smithi090 ceph-mon[30350]: pgmap v529: 129 pgs: 129 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-09-10T22:07:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:36 smithi154 ceph-mon[34495]: pgmap v529: 129 pgs: 129 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-09-10T22:07:37.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:36 smithi171 ceph-mon[34138]: pgmap v529: 129 pgs: 129 active+clean; 48 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-09-10T22:07:37.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39449 1.93 MB/sec execute 29 sec latency 17.172 ms 2024-09-10T22:07:38.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39651 1.87 MB/sec execute 30 sec latency 20.727 ms 2024-09-10T22:07:39.050 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:38 smithi171 ceph-mon[34138]: pgmap v530: 129 pgs: 129 active+clean; 49 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 945 KiB/s wr, 103 op/s 2024-09-10T22:07:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:38 smithi090 ceph-mon[30350]: pgmap v530: 129 pgs: 129 active+clean; 49 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 945 KiB/s wr, 103 op/s 2024-09-10T22:07:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:38 smithi154 ceph-mon[34495]: pgmap v530: 129 pgs: 129 active+clean; 49 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 945 KiB/s wr, 103 op/s 2024-09-10T22:07:39.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39875 1.82 MB/sec execute 31 sec latency 355.372 ms 2024-09-10T22:07:40.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 40138 1.81 MB/sec execute 32 sec latency 572.148 ms 2024-09-10T22:07:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:40 smithi090 ceph-mon[30350]: pgmap v531: 129 pgs: 129 active+clean; 50 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-09-10T22:07:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:40 smithi154 ceph-mon[34495]: pgmap v531: 129 pgs: 129 active+clean; 50 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-09-10T22:07:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:40 smithi171 ceph-mon[34138]: pgmap v531: 129 pgs: 129 active+clean; 50 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.1 MiB/s wr, 114 op/s 2024-09-10T22:07:41.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 40526 1.85 MB/sec execute 33 sec latency 21.680 ms 2024-09-10T22:07:42.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 40921 1.81 MB/sec execute 34 sec latency 25.109 ms 2024-09-10T22:07:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:42 smithi090 ceph-mon[30350]: pgmap v532: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 939 KiB/s wr, 120 op/s 2024-09-10T22:07:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:42 smithi154 ceph-mon[34495]: pgmap v532: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 939 KiB/s wr, 120 op/s 2024-09-10T22:07:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:42 smithi171 ceph-mon[34138]: pgmap v532: 129 pgs: 129 active+clean; 50 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 939 KiB/s wr, 120 op/s 2024-09-10T22:07:43.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 41386 1.80 MB/sec execute 35 sec latency 17.495 ms 2024-09-10T22:07:44.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 41660 1.78 MB/sec execute 36 sec latency 19.797 ms 2024-09-10T22:07:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:44 smithi090 ceph-mon[30350]: pgmap v533: 129 pgs: 129 active+clean; 50 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 975 KiB/s wr, 116 op/s 2024-09-10T22:07:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:44 smithi154 ceph-mon[34495]: pgmap v533: 129 pgs: 129 active+clean; 50 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 975 KiB/s wr, 116 op/s 2024-09-10T22:07:45.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:44 smithi171 ceph-mon[34138]: pgmap v533: 129 pgs: 129 active+clean; 50 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 975 KiB/s wr, 116 op/s 2024-09-10T22:07:45.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 42266 1.87 MB/sec execute 37 sec latency 15.787 ms 2024-09-10T22:07:46.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 42547 1.93 MB/sec execute 38 sec latency 16.762 ms 2024-09-10T22:07:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:46 smithi090 ceph-mon[30350]: pgmap v534: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 953 KiB/s wr, 110 op/s 2024-09-10T22:07:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:46 smithi154 ceph-mon[34495]: pgmap v534: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 953 KiB/s wr, 110 op/s 2024-09-10T22:07:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:46 smithi171 ceph-mon[34138]: pgmap v534: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 953 KiB/s wr, 110 op/s 2024-09-10T22:07:47.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 42852 1.92 MB/sec execute 39 sec latency 17.817 ms 2024-09-10T22:07:48.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 43029 1.88 MB/sec execute 40 sec latency 18.539 ms 2024-09-10T22:07:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:48 smithi090 ceph-mon[30350]: pgmap v535: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 668 KiB/s wr, 94 op/s 2024-09-10T22:07:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:48 smithi154 ceph-mon[34495]: pgmap v535: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 668 KiB/s wr, 94 op/s 2024-09-10T22:07:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:48 smithi171 ceph-mon[34138]: pgmap v535: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 668 KiB/s wr, 94 op/s 2024-09-10T22:07:49.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 43254 1.83 MB/sec execute 41 sec latency 19.790 ms 2024-09-10T22:07:50.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 43578 1.81 MB/sec execute 42 sec latency 17.837 ms 2024-09-10T22:07:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:50 smithi090 ceph-mon[30350]: pgmap v536: 129 pgs: 129 active+clean; 52 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-10T22:07:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:07:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:07:51.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:07:51.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:50 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:07:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:50 smithi154 ceph-mon[34495]: pgmap v536: 129 pgs: 129 active+clean; 52 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-10T22:07:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:07:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:07:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:07:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:50 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:07:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:50 smithi171 ceph-mon[34138]: pgmap v536: 129 pgs: 129 active+clean; 52 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-10T22:07:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:07:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:07:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:07:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:50 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:07:51.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 43986 1.86 MB/sec execute 43 sec latency 23.589 ms 2024-09-10T22:07:52.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 44375 1.83 MB/sec execute 44 sec latency 17.297 ms 2024-09-10T22:07:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:52 smithi090 ceph-mon[30350]: pgmap v537: 129 pgs: 129 active+clean; 53 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 906 KiB/s wr, 107 op/s 2024-09-10T22:07:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:52 smithi154 ceph-mon[34495]: pgmap v537: 129 pgs: 129 active+clean; 53 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 906 KiB/s wr, 107 op/s 2024-09-10T22:07:53.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:52 smithi171 ceph-mon[34138]: pgmap v537: 129 pgs: 129 active+clean; 53 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 906 KiB/s wr, 107 op/s 2024-09-10T22:07:53.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 44782 1.80 MB/sec execute 45 sec latency 20.521 ms 2024-09-10T22:07:54.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 45099 1.80 MB/sec execute 46 sec latency 19.467 ms 2024-09-10T22:07:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:54 smithi090 ceph-mon[30350]: pgmap v538: 129 pgs: 129 active+clean; 51 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1001 KiB/s wr, 120 op/s 2024-09-10T22:07:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:54 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:07:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:54 smithi154 ceph-mon[34495]: pgmap v538: 129 pgs: 129 active+clean; 51 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1001 KiB/s wr, 120 op/s 2024-09-10T22:07:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:54 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:07:55.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:54 smithi171 ceph-mon[34138]: pgmap v538: 129 pgs: 129 active+clean; 51 MiB data, 603 MiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 1001 KiB/s wr, 120 op/s 2024-09-10T22:07:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:54 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:07:55.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 45603 1.87 MB/sec execute 47 sec latency 22.417 ms 2024-09-10T22:07:56.460 INFO:tasks.workunit.client.0.smithi090.stdout: 1 45931 1.87 MB/sec execute 48 sec latency 21.101 ms 2024-09-10T22:07:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:56 smithi090 ceph-mon[30350]: pgmap v539: 129 pgs: 129 active+clean; 52 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-09-10T22:07:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:56 smithi154 ceph-mon[34495]: pgmap v539: 129 pgs: 129 active+clean; 52 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-09-10T22:07:57.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:56 smithi171 ceph-mon[34138]: pgmap v539: 129 pgs: 129 active+clean; 52 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-09-10T22:07:57.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 46307 1.90 MB/sec execute 49 sec latency 20.798 ms 2024-09-10T22:07:58.141 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T22:07:58.141 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T22:07:58.142 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:07:58.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 46519 1.88 MB/sec execute 50 sec latency 17.363 ms 2024-09-10T22:07:58.573 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:07:58.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:58 smithi090 ceph-mon[30350]: pgmap v540: 129 pgs: 129 active+clean; 52 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 910 KiB/s wr, 96 op/s 2024-09-10T22:07:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:58 smithi154 ceph-mon[34495]: pgmap v540: 129 pgs: 129 active+clean; 52 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 910 KiB/s wr, 96 op/s 2024-09-10T22:07:59.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.199+0000 7f97b2a00640 1 -- 172.21.15.90:0/4102062921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac106d40 msgr2=0x7f97ac107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.199+0000 7f97b2a00640 1 --2- 172.21.15.90:0/4102062921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac106d40 0x7f97ac107190 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f97a4009ca0 tx=0x7f97a402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.199+0000 7f97b2a00640 1 -- 172.21.15.90:0/4102062921 shutdown_connections 2024-09-10T22:07:59.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.199+0000 7f97b2a00640 1 --2- 172.21.15.90:0/4102062921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac106d40 0x7f97ac107190 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.199+0000 7f97b2a00640 1 --2- 172.21.15.90:0/4102062921 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97ac102220 0x7f97ac1066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.199+0000 7f97b2a00640 1 --2- 172.21.15.90:0/4102062921 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac101900 0x7f97ac101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.199+0000 7f97b2a00640 1 -- 172.21.15.90:0/4102062921 >> 172.21.15.90:0/4102062921 conn(0x7f97ac0fd5b0 msgr2=0x7f97ac0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:59.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.199+0000 7f97b2a00640 1 -- 172.21.15.90:0/4102062921 shutdown_connections 2024-09-10T22:07:59.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.199+0000 7f97b2a00640 1 -- 172.21.15.90:0/4102062921 wait complete. 2024-09-10T22:07:59.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.200+0000 7f97b2a00640 1 Processor -- start 2024-09-10T22:07:59.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b2a00640 1 -- start start 2024-09-10T22:07:59.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac101900 0x7f97ac19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:59.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b2a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97ac102220 0x7f97ac19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:59.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b2a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac106d40 0x7f97ac1a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:59.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b2a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac1190c0 con 0x7f97ac101900 2024-09-10T22:07:59.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b2a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac118f40 con 0x7f97ac106d40 2024-09-10T22:07:59.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b2a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f97ac119240 con 0x7f97ac102220 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b1600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac101900 0x7f97ac19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b1600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac101900 0x7f97ac19f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57778/0 (socket says 172.21.15.90:57778) 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b1600640 1 -- 172.21.15.90:0/1663968237 learned_addr learned my addr 172.21.15.90:0/1663968237 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.201+0000 7f97b2000640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac106d40 0x7f97ac1a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97b0c00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97ac102220 0x7f97ac19fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97b1600640 1 -- 172.21.15.90:0/1663968237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97ac102220 msgr2=0x7f97ac19fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97b1600640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97ac102220 0x7f97ac19fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97b1600640 1 -- 172.21.15.90:0/1663968237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac106d40 msgr2=0x7f97ac1a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97b1600640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac106d40 0x7f97ac1a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97b1600640 1 -- 172.21.15.90:0/1663968237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ac1a55c0 con 0x7f97ac101900 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97b0c00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97ac102220 0x7f97ac19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97b2000640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac106d40 0x7f97ac1a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:07:59.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97b1600640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac101900 0x7f97ac19f510 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f97a000e9f0 tx=0x7f97a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:59.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97aa000640 1 -- 172.21.15.90:0/1663968237 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a000cd80 con 0x7f97ac101900 2024-09-10T22:07:59.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.203+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ac1a58b0 con 0x7f97ac101900 2024-09-10T22:07:59.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.202+0000 7f97aa000640 1 -- 172.21.15.90:0/1663968237 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97a00108c0 con 0x7f97ac101900 2024-09-10T22:07:59.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.203+0000 7f97aa000640 1 -- 172.21.15.90:0/1663968237 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a00187c0 con 0x7f97ac101900 2024-09-10T22:07:59.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.203+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ac1abf70 con 0x7f97ac101900 2024-09-10T22:07:59.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.204+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97ac1066f0 con 0x7f97ac101900 2024-09-10T22:07:59.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.207+0000 7f97aa000640 1 -- 172.21.15.90:0/1663968237 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f97a0010430 con 0x7f97ac101900 2024-09-10T22:07:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.208+0000 7f97aa000640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f97940b5500 0x7f97940b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:07:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.208+0000 7f97aa000640 1 -- 172.21.15.90:0/1663968237 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f97a00e6ca0 con 0x7f97ac101900 2024-09-10T22:07:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.208+0000 7f97b0c00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f97940b5500 0x7f97940b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:07:59.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.209+0000 7f97b0c00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f97940b5500 0x7f97940b79c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9798005fd0 tx=0x7f9798005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:07:59.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.210+0000 7f97aa000640 1 -- 172.21.15.90:0/1663968237 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f97a00ad450 con 0x7f97ac101900 2024-09-10T22:07:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:58 smithi171 ceph-mon[34138]: pgmap v540: 129 pgs: 129 active+clean; 52 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 910 KiB/s wr, 96 op/s 2024-09-10T22:07:59.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 46705 1.85 MB/sec execute 51 sec latency 18.272 ms 2024-09-10T22:07:59.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.469+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97ac1a0640 con 0x7f97ac101900 2024-09-10T22:07:59.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.470+0000 7f97aa000640 1 -- 172.21.15.90:0/1663968237 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11987 (secure 0 0 0) 0x7f97a00acd50 con 0x7f97ac101900 2024-09-10T22:07:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:07:59.473 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":34,"btime":"2024-09-10T22:07:58:800401+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:07:58.499883+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:07:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 34 2024-09-10T22:07:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f97940b5500 msgr2=0x7f97940b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f97940b5500 0x7f97940b79c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f9798005fd0 tx=0x7f9798005850 comp rx=0 tx=0).stop 2024-09-10T22:07:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac101900 msgr2=0x7f97ac19f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:07:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac101900 0x7f97ac19f510 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f97a000e9f0 tx=0x7f97a000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 shutdown_connections 2024-09-10T22:07:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f97940b5500 0x7f97940b79c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f97ac106d40 0x7f97ac1a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f97ac102220 0x7f97ac19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 --2- 172.21.15.90:0/1663968237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97ac101900 0x7f97ac19f510 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:07:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 >> 172.21.15.90:0/1663968237 conn(0x7f97ac0fd5b0 msgr2=0x7f97ac104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:07:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 shutdown_connections 2024-09-10T22:07:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:07:59.473+0000 7f97b2a00640 1 -- 172.21.15.90:0/1663968237 wait complete. 2024-09-10T22:08:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:59 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:08:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:07:59 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1663968237' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:08:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:59 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:08:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:07:59 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1663968237' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:08:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:59 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:08:00.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:07:59 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1663968237' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:08:00.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47021 1.82 MB/sec execute 52 sec latency 20.798 ms 2024-09-10T22:08:00.771 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T22:08:00.771 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T22:08:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:00 smithi090 ceph-mon[30350]: pgmap v541: 129 pgs: 129 active+clean; 54 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-10T22:08:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:00 smithi154 ceph-mon[34495]: pgmap v541: 129 pgs: 129 active+clean; 54 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-10T22:08:01.219 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:08:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:00 smithi171 ceph-mon[34138]: pgmap v541: 129 pgs: 129 active+clean; 54 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-10T22:08:01.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47355 1.82 MB/sec execute 53 sec latency 21.304 ms 2024-09-10T22:08:02.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.380+0000 7f89c4e00640 1 -- 172.21.15.90:0/2100053848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108620 msgr2=0x7f89c0108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.380+0000 7f89c4e00640 1 --2- 172.21.15.90:0/2100053848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108620 0x7f89c0108a60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009900 tx=0x7f89ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.380+0000 7f89c4e00640 1 -- 172.21.15.90:0/2100053848 shutdown_connections 2024-09-10T22:08:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.380+0000 7f89c4e00640 1 --2- 172.21.15.90:0/2100053848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c0108fa0 0x7f89c010f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.380+0000 7f89c4e00640 1 --2- 172.21.15.90:0/2100053848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108620 0x7f89c0108a60 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.380+0000 7f89c4e00640 1 --2- 172.21.15.90:0/2100053848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89c0102c10 0x7f89c0102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.380+0000 7f89c4e00640 1 -- 172.21.15.90:0/2100053848 >> 172.21.15.90:0/2100053848 conn(0x7f89c00fe660 msgr2=0x7f89c0100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.380+0000 7f89c4e00640 1 -- 172.21.15.90:0/2100053848 shutdown_connections 2024-09-10T22:08:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.380+0000 7f89c4e00640 1 -- 172.21.15.90:0/2100053848 wait complete. 2024-09-10T22:08:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.381+0000 7f89c4e00640 1 Processor -- start 2024-09-10T22:08:02.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89c4e00640 1 -- start start 2024-09-10T22:08:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89c4e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89c0102c10 0x7f89c0198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89c4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c0108620 0x7f89c01994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89c4e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108fa0 0x7f89c019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89c4e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0112b60 con 0x7f89c0108fa0 2024-09-10T22:08:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89c4e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01129e0 con 0x7f89c0108620 2024-09-10T22:08:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89c4e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f89c0112ce0 con 0x7f89c0102c10 2024-09-10T22:08:02.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89bf400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89c0102c10 0x7f89c0198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108fa0 0x7f89c019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108fa0 0x7f89c019e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57794/0 (socket says 172.21.15.90:57794) 2024-09-10T22:08:02.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89bea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c0108620 0x7f89c01994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.382+0000 7f89bfe00640 1 -- 172.21.15.90:0/4008245649 learned_addr learned my addr 172.21.15.90:0/4008245649 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:02.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.383+0000 7f89bfe00640 1 -- 172.21.15.90:0/4008245649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89c0102c10 msgr2=0x7f89c0198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.383+0000 7f89bfe00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89c0102c10 0x7f89c0198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.383+0000 7f89bfe00640 1 -- 172.21.15.90:0/4008245649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c0108620 msgr2=0x7f89c01994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.383+0000 7f89bfe00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c0108620 0x7f89c01994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.383+0000 7f89bfe00640 1 -- 172.21.15.90:0/4008245649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c019f020 con 0x7f89c0108fa0 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.383+0000 7f89bea00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c0108620 0x7f89c01994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.383+0000 7f89bf400640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89c0102c10 0x7f89c0198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.383+0000 7f89bfe00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108fa0 0x7f89c019e850 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f89b800eb10 tx=0x7f89b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.384+0000 7f89b7e00640 1 -- 172.21.15.90:0/4008245649 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8004b20 con 0x7f89c0108fa0 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.384+0000 7f89b7e00640 1 -- 172.21.15.90:0/4008245649 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89b8004cc0 con 0x7f89c0108fa0 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.384+0000 7f89c4e00640 1 -- 172.21.15.90:0/4008245649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c019f310 con 0x7f89c0108fa0 2024-09-10T22:08:02.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.384+0000 7f89b7e00640 1 -- 172.21.15.90:0/4008245649 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8010690 con 0x7f89c0108fa0 2024-09-10T22:08:02.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.384+0000 7f89c4e00640 1 -- 172.21.15.90:0/4008245649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c01a59d0 con 0x7f89c0108fa0 2024-09-10T22:08:02.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.388+0000 7f89c8ff9740 1 Processor -- start 2024-09-10T22:08:02.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.389+0000 7f89b7e00640 1 -- 172.21.15.90:0/4008245649 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f89b8005020 con 0x7f89c0108fa0 2024-09-10T22:08:02.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.389+0000 7f89c8ff9740 1 -- start start 2024-09-10T22:08:02.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.389+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de4a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.389+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4ac10 0x56213de5aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.389+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de5d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.389+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56213ddbfca0 con 0x56213de4a2f0 2024-09-10T22:08:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.389+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56213ddaf0d0 con 0x56213de5afb0 2024-09-10T22:08:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.389+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56213ddb2ee0 con 0x56213de4ac10 2024-09-10T22:08:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.390+0000 7f89bf400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4ac10 0x56213de5aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.390+0000 7f89bf400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4ac10 0x56213de5aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:38632/0 (socket says 172.21.15.90:38632) 2024-09-10T22:08:02.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.390+0000 7f89bf400640 1 -- 172.21.15.90:0/3487889432 learned_addr learned my addr 172.21.15.90:0/3487889432 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.390+0000 7f89bea00640 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de5d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.390+0000 7f89bea00640 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de4a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89bea00640 1 -- 172.21.15.90:0/3487889432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4ac10 msgr2=0x56213de5aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89bea00640 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4ac10 0x56213de5aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89bea00640 1 -- 172.21.15.90:0/3487889432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 msgr2=0x56213de4a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89bea00640 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de4a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89bea00640 1 -- 172.21.15.90:0/3487889432 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56213dd61400 con 0x56213de5afb0 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89bf400640 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4ac10 0x56213de5aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89b7e00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f89a40b5580 0x7f89a40b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89bea00640 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de5d3a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89ac02a910 tx=0x7f89ac005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2a94781048916dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89b7e00640 1 -- 172.21.15.90:0/4008245649 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f89b80e6870 con 0x7f89c0108fa0 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89b4c00640 1 -- 172.21.15.90:0/3487889432 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac035030 con 0x56213de5afb0 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89b4c00640 1 -- 172.21.15.90:0/3487889432 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89ac003cb0 con 0x56213de5afb0 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.391+0000 7f89b4c00640 1 -- 172.21.15.90:0/3487889432 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac03c440 con 0x56213de5afb0 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.392+0000 7f89bf400640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f89a40b5580 0x7f89a40b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.392+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3487889432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 msgr2=0x56213de5d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.392+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de5d3a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f89ac02a910 tx=0x7f89ac005e80 comp rx=0 tx=0).stop 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.393+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3487889432 shutdown_connections 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.393+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de5d3a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.393+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4ac10 0x56213de5aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.393+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3487889432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.393+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3487889432 >> 172.21.15.90:0/3487889432 conn(0x56213de48120 msgr2=0x56213de492c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.393+0000 7f89bf400640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f89a40b5580 0x7f89a40b7a40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56213de49000 tx=0x7f89b0005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.393+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3487889432 shutdown_connections 2024-09-10T22:08:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.393+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3487889432 wait complete. 2024-09-10T22:08:02.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.393+0000 7f89c8ff9740 1 -- start start 2024-09-10T22:08:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.394+0000 7f89c8ff9740 1 Processor -- start 2024-09-10T22:08:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.394+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 0x56213de5a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.394+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de57d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.394+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de5afb0 0x56213de58460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.394+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56213dc107a0 con 0x56213de5afb0 2024-09-10T22:08:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.394+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56213dd54a10 con 0x56213de4ac10 2024-09-10T22:08:02.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.394+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56213de5dbd0 con 0x56213de4a2f0 2024-09-10T22:08:02.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.394+0000 7f89c8ff9740 10 client.0 _ll_register_callbacks cb 0x56213dd64be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:08:02.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de57d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de5afb0 0x56213de58460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de57d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:54960/0 (socket says 172.21.15.90:54960) 2024-09-10T22:08:02.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bea00640 1 -- 172.21.15.90:0/4657597 learned_addr learned my addr 172.21.15.90:0/4657597 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:02.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bf400640 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 0x56213de5a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bfe00640 1 -- 172.21.15.90:0/4657597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 msgr2=0x56213de5a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:02.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bfe00640 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 0x56213de5a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bfe00640 1 -- 172.21.15.90:0/4657597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 msgr2=0x56213de57d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bfe00640 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de57d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bfe00640 1 -- 172.21.15.90:0/4657597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56213db1cb90 con 0x56213de5afb0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bea00640 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de57d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.395+0000 7f89bfe00640 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de5afb0 0x56213de58460 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f89b80e3c50 tx=0x7f89b80ac750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577b58ef682ba410 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.396+0000 7f89a9600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.396+0000 7f89a9600640 1 -- 172.21.15.90:0/4657597 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b80a80f0 con 0x56213de5afb0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.396+0000 7f89a9600640 1 -- 172.21.15.90:0/4657597 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f89b80b5a80 con 0x56213de5afb0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.396+0000 7f89c8ff9740 1 -- 172.21.15.90:0/4657597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac003cb0 con 0x56213de5afb0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.396+0000 7f89a9600640 1 -- 172.21.15.90:0/4657597 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f89b8005540 con 0x56213de5afb0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.396+0000 7f89c8ff9740 10 client.15882 fetch_fsmap learned FSMap version 34 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.396+0000 7f89c8ff9740 1 -- 172.21.15.90:0/4657597 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56213de69600 con 0x56213de5afb0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.396+0000 7f89a9600640 1 -- 172.21.15.90:0/4657597 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89ac003cb0 con 0x56213de5afb0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.397+0000 7f89a9600640 1 -- 172.21.15.90:0/4657597 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f89b80a82d0 con 0x56213de5afb0 2024-09-10T22:08:02.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.397+0000 7f89c8ff9740 10 client.15882 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T22:08:02.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.397+0000 7f89c8ff9740 10 client.15882 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:08:02.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.397+0000 7f89c8ff9740 20 client.15882 populate_metadata read hostname 'smithi090' 2024-09-10T22:08:02.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.397+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56213de6ab60 0x56213de6cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.397+0000 7f89c8ff9740 4 client.15882 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:08:02.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.397+0000 7f89c8ff9740 1 -- 172.21.15.90:0/4657597 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56213ddc1bf0 con 0x56213de6ab60 2024-09-10T22:08:02.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.397+0000 7f89bf400640 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56213de6ab60 0x56213de6cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.398+0000 7f89bf400640 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56213de6ab60 0x56213de6cf90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=382ac2121dac0c1d server_cookie=5d88fc4b22b4f6de in_seq=0 out_seq=0 2024-09-10T22:08:02.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.398+0000 7f89a9600640 10 client.15882 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:02.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.399+0000 7f89a9600640 1 -- 172.21.15.90:0/4657597 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56213ddc1bf0 con 0x56213de6ab60 2024-09-10T22:08:02.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.399+0000 7f89a9600640 10 client.15882 handle_command_reply: tid=0 2024-09-10T22:08:02.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.399+0000 7f89c8ff9740 1 client.15882 shutdown 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.399+0000 7f89c8ff9740 1 -- 172.21.15.90:0/4657597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de5afb0 msgr2=0x56213de58460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.399+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de5afb0 0x56213de58460 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f89b80e3c50 tx=0x7f89b80ac750 comp rx=0 tx=0).stop 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89c8ff9740 1 -- 172.21.15.90:0/4657597 shutdown_connections 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56213de6ab60 0x56213de6cf90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de5afb0 0x56213de58460 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de57d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/4657597 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 0x56213de5a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89c8ff9740 1 -- 172.21.15.90:0/4657597 >> 172.21.15.90:0/4657597 conn(0x56213de48120 msgr2=0x56213de4dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89a9600640 0 client.15882 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89c8ff9740 1 -- 172.21.15.90:0/4657597 shutdown_connections 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89c8ff9740 1 -- 172.21.15.90:0/4657597 wait complete. 2024-09-10T22:08:02.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.400+0000 7f89c8ff9740 20 client.15882 trim_cache size 0 max 16384 2024-09-10T22:08:02.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.421+0000 7f89c8ff9740 1 Processor -- start 2024-09-10T22:08:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.421+0000 7f89c8ff9740 1 -- start start 2024-09-10T22:08:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.421+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 0x56213de66e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.421+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4ac10 0x56213de59ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.421+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.421+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01ad900 con 0x56213de4ac10 2024-09-10T22:08:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.421+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01af470 con 0x56213de5afb0 2024-09-10T22:08:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.421+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01adfd0 con 0x56213de4a2f0 2024-09-10T22:08:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.421+0000 7f89bea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4ac10 0x56213de59ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4ac10 0x56213de59ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57806/0 (socket says 172.21.15.90:57806) 2024-09-10T22:08:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bea00640 1 -- 172.21.15.90:0/604284123 learned_addr learned my addr 172.21.15.90:0/604284123 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bfe00640 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bf400640 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 0x56213de66e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bea00640 1 -- 172.21.15.90:0/604284123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 msgr2=0x56213de66e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bea00640 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 0x56213de66e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bea00640 1 -- 172.21.15.90:0/604284123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 msgr2=0x56213de572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bea00640 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bea00640 1 -- 172.21.15.90:0/604284123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56213de5c520 con 0x56213de4ac10 2024-09-10T22:08:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.422+0000 7f89bfe00640 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:02.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.423+0000 7f89bea00640 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4ac10 0x56213de59ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0099f0 tx=0x7f89ac005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c65206db4d571a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.423+0000 7f89abe00640 1 -- 172.21.15.90:0/604284123 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac035040 con 0x56213de4ac10 2024-09-10T22:08:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.423+0000 7f89abe00640 1 -- 172.21.15.90:0/604284123 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89ac04e070 con 0x56213de4ac10 2024-09-10T22:08:02.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.423+0000 7f89abe00640 1 -- 172.21.15.90:0/604284123 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac053cb0 con 0x56213de4ac10 2024-09-10T22:08:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.424+0000 7f89c8ff9740 1 -- 172.21.15.90:0/604284123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4ac10 msgr2=0x56213de59ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.424+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4ac10 0x56213de59ad0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89ac0099f0 tx=0x7f89ac005890 comp rx=0 tx=0).stop 2024-09-10T22:08:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.424+0000 7f89c8ff9740 1 -- 172.21.15.90:0/604284123 shutdown_connections 2024-09-10T22:08:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.424+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de5afb0 0x56213de572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.424+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4ac10 0x56213de59ad0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.424+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/604284123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de4a2f0 0x56213de66e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.424+0000 7f89c8ff9740 1 -- 172.21.15.90:0/604284123 >> 172.21.15.90:0/604284123 conn(0x56213de48120 msgr2=0x56213de593c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.424+0000 7f89c8ff9740 1 -- 172.21.15.90:0/604284123 shutdown_connections 2024-09-10T22:08:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.424+0000 7f89c8ff9740 1 -- 172.21.15.90:0/604284123 wait complete. 2024-09-10T22:08:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.425+0000 7f89c8ff9740 1 -- start start 2024-09-10T22:08:02.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.425+0000 7f89c8ff9740 1 Processor -- start 2024-09-10T22:08:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.425+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de62e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89c8ff9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de5afb0 0x56213de5fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01aed70 con 0x56213de4a2f0 2024-09-10T22:08:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01ae6a0 con 0x56213de4ac10 2024-09-10T22:08:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89c8ff9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01ab8e0 con 0x56213de5afb0 2024-09-10T22:08:02.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89c8ff9740 10 client.0 _ll_register_callbacks cb 0x7f89b80a8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:08:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89bf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de62e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89bf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de62e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57820/0 (socket says 172.21.15.90:57820) 2024-09-10T22:08:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89bf400640 1 -- 172.21.15.90:0/3040448695 learned_addr learned my addr 172.21.15.90:0/3040448695 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.426+0000 7f89bea00640 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de633a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.427+0000 7f89bf400640 1 -- 172.21.15.90:0/3040448695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de5afb0 msgr2=0x56213de5fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:02.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.427+0000 7f89bf400640 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de5afb0 0x56213de5fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.427+0000 7f89bf400640 1 -- 172.21.15.90:0/3040448695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 msgr2=0x56213de633a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.427+0000 7f89bf400640 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de633a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.427+0000 7f89bf400640 1 -- 172.21.15.90:0/3040448695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56213de5b930 con 0x56213de4a2f0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.427+0000 7f89bf400640 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de62e60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x56213de6d800 tx=0x7f89b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d02b8432715e88d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.427+0000 7f89a9600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.428+0000 7f89a9600640 1 -- 172.21.15.90:0/3040448695 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b00035f0 con 0x56213de4a2f0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.428+0000 7f89a9600640 1 -- 172.21.15.90:0/3040448695 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f89b0003790 con 0x56213de4a2f0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.428+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3040448695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89ac04e070 con 0x56213de4a2f0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.428+0000 7f89a9600640 1 -- 172.21.15.90:0/3040448695 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f89b003c050 con 0x56213de4a2f0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.428+0000 7f89c8ff9740 10 client.15888 fetch_fsmap learned FSMap version 34 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.428+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3040448695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56213ded8130 con 0x56213de4a2f0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.428+0000 7f89a9600640 1 -- 172.21.15.90:0/3040448695 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f89ac04e070 con 0x56213de4a2f0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.429+0000 7f89a9600640 1 -- 172.21.15.90:0/3040448695 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f89b0031880 con 0x56213de4a2f0 2024-09-10T22:08:02.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.429+0000 7f89c8ff9740 10 client.15888 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T22:08:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.429+0000 7f89c8ff9740 10 client.15888 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:08:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.429+0000 7f89c8ff9740 20 client.15888 populate_metadata read hostname 'smithi090' 2024-09-10T22:08:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.429+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56213ded8980 0x56213dedada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.429+0000 7f89c8ff9740 4 client.15888 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T22:08:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.429+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3040448695 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x56213ddbd6a0 con 0x56213ded8980 2024-09-10T22:08:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.429+0000 7f89bea00640 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56213ded8980 0x56213dedada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.430+0000 7f89bea00640 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56213ded8980 0x56213dedada0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e8f2c8a192f2721 server_cookie=84b5650f2f283701 in_seq=0 out_seq=0 2024-09-10T22:08:02.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.430+0000 7f89a9600640 10 client.15888 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:02.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47744 1.84 MB/sec execute 54 sec latency 25.171 ms 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.501+0000 7f89a9600640 1 -- 172.21.15.90:0/3040448695 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56213ddbd6a0 con 0x56213ded8980 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.501+0000 7f89a9600640 10 client.15888 handle_command_reply: tid=0 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.501+0000 7f89c8ff9740 1 client.15888 shutdown 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.501+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3040448695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 msgr2=0x56213de62e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.501+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de62e60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x56213de6d800 tx=0x7f89b00046c0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.502+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3040448695 shutdown_connections 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.502+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56213ded8980 0x56213dedada0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.502+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56213de5afb0 0x56213de5fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.502+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56213de4ac10 0x56213de633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.502+0000 7f89c8ff9740 1 --2- 172.21.15.90:0/3040448695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56213de4a2f0 0x56213de62e60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.502+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3040448695 >> 172.21.15.90:0/3040448695 conn(0x56213de48120 msgr2=0x56213de59be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.502+0000 7f89a9600640 0 client.15888 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:02.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.502+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3040448695 shutdown_connections 2024-09-10T22:08:02.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.503+0000 7f89c8ff9740 1 -- 172.21.15.90:0/3040448695 wait complete. 2024-09-10T22:08:02.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.503+0000 7f89c8ff9740 20 client.15888 trim_cache size 0 max 16384 2024-09-10T22:08:02.505 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:08:02.505 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 34, 2024-09-10T22:08:02.505 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:08:02.505 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 14, 2024-09-10T22:08:02.505 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:08:02.505 INFO:teuthology.orchestra.run.smithi090.stdout: "b34f2bd6": { 2024-09-10T22:08:02.505 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 14, 2024-09-10T22:08:02.505 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:02.506 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:02.507 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:02.507 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 -- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f89a40b5580 msgr2=0x7f89a40b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f89a40b5580 0x7f89a40b7a40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56213de49000 tx=0x7f89b0005f90 comp rx=0 tx=0).stop 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 -- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108fa0 msgr2=0x7f89c019e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108fa0 0x7f89c019e850 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f89b800eb10 tx=0x7f89b800c2d0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 -- 172.21.15.90:0/4008245649 shutdown_connections 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f89a40b5580 0x7f89a40b7a40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89c0108fa0 0x7f89c019e850 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f89c0108620 0x7f89c01994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 --2- 172.21.15.90:0/4008245649 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f89c0102c10 0x7f89c0198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.504+0000 7f89c4e00640 1 -- 172.21.15.90:0/4008245649 >> 172.21.15.90:0/4008245649 conn(0x7f89c00fe660 msgr2=0x7f89c0100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:02.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.505+0000 7f89c4e00640 1 -- 172.21.15.90:0/4008245649 shutdown_connections 2024-09-10T22:08:02.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:02.506+0000 7f89c4e00640 1 -- 172.21.15.90:0/4008245649 wait complete. 2024-09-10T22:08:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:02 smithi090 ceph-mon[30350]: pgmap v542: 129 pgs: 129 active+clean; 55 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 911 KiB/s wr, 110 op/s 2024-09-10T22:08:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:02 smithi154 ceph-mon[34495]: pgmap v542: 129 pgs: 129 active+clean; 55 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 911 KiB/s wr, 110 op/s 2024-09-10T22:08:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:02 smithi171 ceph-mon[34138]: pgmap v542: 129 pgs: 129 active+clean; 55 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 911 KiB/s wr, 110 op/s 2024-09-10T22:08:03.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.81 MB/sec execute 55 sec latency 959.450 ms 2024-09-10T22:08:03.537 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b34f2bd6', quiesce took 5.4 seconds. Will release after: 24.6 2024-09-10T22:08:04.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.78 MB/sec execute 56 sec latency 1959.561 ms 2024-09-10T22:08:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:04 smithi090 ceph-mon[30350]: pgmap v543: 129 pgs: 129 active+clean; 55 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-09-10T22:08:05.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:04 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:08:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:04 smithi154 ceph-mon[34495]: pgmap v543: 129 pgs: 129 active+clean; 55 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-09-10T22:08:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:04 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:08:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:04 smithi171 ceph-mon[34138]: pgmap v543: 129 pgs: 129 active+clean; 55 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-09-10T22:08:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:04 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:08:05.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.74 MB/sec execute 57 sec latency 2959.688 ms 2024-09-10T22:08:06.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.71 MB/sec execute 58 sec latency 3959.793 ms 2024-09-10T22:08:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:06 smithi090 ceph-mon[30350]: pgmap v544: 129 pgs: 129 active+clean; 55 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 979 KiB/s wr, 122 op/s 2024-09-10T22:08:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:06 smithi154 ceph-mon[34495]: pgmap v544: 129 pgs: 129 active+clean; 55 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 979 KiB/s wr, 122 op/s 2024-09-10T22:08:07.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:06 smithi171 ceph-mon[34138]: pgmap v544: 129 pgs: 129 active+clean; 55 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 979 KiB/s wr, 122 op/s 2024-09-10T22:08:07.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.69 MB/sec execute 59 sec latency 4959.918 ms 2024-09-10T22:08:08.461 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.66 MB/sec execute 60 sec latency 5959.999 ms 2024-09-10T22:08:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:08 smithi090 ceph-mon[30350]: pgmap v545: 129 pgs: 129 active+clean; 55 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 771 KiB/s wr, 104 op/s 2024-09-10T22:08:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:08 smithi154 ceph-mon[34495]: pgmap v545: 129 pgs: 129 active+clean; 55 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 771 KiB/s wr, 104 op/s 2024-09-10T22:08:09.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:08 smithi171 ceph-mon[34138]: pgmap v545: 129 pgs: 129 active+clean; 55 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 771 KiB/s wr, 104 op/s 2024-09-10T22:08:09.462 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.63 MB/sec execute 61 sec latency 6960.116 ms 2024-09-10T22:08:10.462 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.60 MB/sec execute 62 sec latency 7960.179 ms 2024-09-10T22:08:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:10 smithi090 ceph-mon[30350]: pgmap v546: 129 pgs: 129 active+clean; 54 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 767 KiB/s wr, 107 op/s 2024-09-10T22:08:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:10 smithi154 ceph-mon[34495]: pgmap v546: 129 pgs: 129 active+clean; 54 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 767 KiB/s wr, 107 op/s 2024-09-10T22:08:11.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:10 smithi171 ceph-mon[34138]: pgmap v546: 129 pgs: 129 active+clean; 54 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 767 KiB/s wr, 107 op/s 2024-09-10T22:08:11.462 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.58 MB/sec execute 63 sec latency 8960.314 ms 2024-09-10T22:08:12.462 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.55 MB/sec execute 64 sec latency 9960.446 ms 2024-09-10T22:08:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:12 smithi090 ceph-mon[30350]: pgmap v547: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 396 KiB/s wr, 83 op/s 2024-09-10T22:08:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:12 smithi154 ceph-mon[34495]: pgmap v547: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 396 KiB/s wr, 83 op/s 2024-09-10T22:08:13.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:12 smithi171 ceph-mon[34138]: pgmap v547: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 396 KiB/s wr, 83 op/s 2024-09-10T22:08:13.462 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.53 MB/sec execute 65 sec latency 10960.523 ms 2024-09-10T22:08:14.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:13 smithi154 ceph-mon[34495]: pgmap v548: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 281 KiB/s wr, 68 op/s 2024-09-10T22:08:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:13 smithi171 ceph-mon[34138]: pgmap v548: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 281 KiB/s wr, 68 op/s 2024-09-10T22:08:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:13 smithi090 ceph-mon[30350]: pgmap v548: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 281 KiB/s wr, 68 op/s 2024-09-10T22:08:14.462 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.51 MB/sec execute 66 sec latency 11960.680 ms 2024-09-10T22:08:15.462 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.48 MB/sec execute 67 sec latency 12960.820 ms 2024-09-10T22:08:16.463 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.46 MB/sec execute 68 sec latency 13960.973 ms 2024-09-10T22:08:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:16 smithi090 ceph-mon[30350]: pgmap v549: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 87 KiB/s wr, 27 op/s 2024-09-10T22:08:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:16 smithi154 ceph-mon[34495]: pgmap v549: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 87 KiB/s wr, 27 op/s 2024-09-10T22:08:17.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:16 smithi171 ceph-mon[34138]: pgmap v549: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 87 KiB/s wr, 27 op/s 2024-09-10T22:08:17.463 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.44 MB/sec execute 69 sec latency 14961.124 ms 2024-09-10T22:08:18.463 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.42 MB/sec execute 70 sec latency 15961.245 ms 2024-09-10T22:08:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:18 smithi090 ceph-mon[30350]: pgmap v550: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-10T22:08:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:18 smithi154 ceph-mon[34495]: pgmap v550: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-10T22:08:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:18 smithi171 ceph-mon[34138]: pgmap v550: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-09-10T22:08:19.463 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.40 MB/sec execute 71 sec latency 16961.388 ms 2024-09-10T22:08:20.463 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.38 MB/sec execute 72 sec latency 17961.513 ms 2024-09-10T22:08:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:20 smithi090 ceph-mon[30350]: pgmap v551: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-10T22:08:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:20 smithi154 ceph-mon[34495]: pgmap v551: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-10T22:08:21.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:20 smithi171 ceph-mon[34138]: pgmap v551: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-09-10T22:08:21.463 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.36 MB/sec execute 73 sec latency 18961.654 ms 2024-09-10T22:08:22.463 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.34 MB/sec execute 74 sec latency 19961.828 ms 2024-09-10T22:08:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:22 smithi090 ceph-mon[30350]: pgmap v552: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-10T22:08:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:22 smithi154 ceph-mon[34495]: pgmap v552: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-10T22:08:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:22 smithi171 ceph-mon[34138]: pgmap v552: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-09-10T22:08:23.463 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.33 MB/sec execute 75 sec latency 20961.899 ms 2024-09-10T22:08:24.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.31 MB/sec execute 76 sec latency 21962.028 ms 2024-09-10T22:08:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:24 smithi090 ceph-mon[30350]: pgmap v553: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-09-10T22:08:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:24 smithi154 ceph-mon[34495]: pgmap v553: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-09-10T22:08:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:24 smithi171 ceph-mon[34138]: pgmap v553: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.7 KiB/s wr, 12 op/s 2024-09-10T22:08:25.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.29 MB/sec execute 77 sec latency 22962.168 ms 2024-09-10T22:08:26.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.27 MB/sec execute 78 sec latency 23962.320 ms 2024-09-10T22:08:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:26 smithi090 ceph-mon[30350]: pgmap v554: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-10T22:08:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:26 smithi154 ceph-mon[34495]: pgmap v554: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-10T22:08:27.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:26 smithi171 ceph-mon[34138]: pgmap v554: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.3 KiB/s wr, 12 op/s 2024-09-10T22:08:27.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.26 MB/sec execute 79 sec latency 24962.455 ms 2024-09-10T22:08:28.138 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:08:28.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.24 MB/sec execute 80 sec latency 25962.494 ms 2024-09-10T22:08:28.522 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:08:28.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:28 smithi090 ceph-mon[30350]: pgmap v555: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-10T22:08:29.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:28 smithi154 ceph-mon[34495]: pgmap v555: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-10T22:08:29.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.160+0000 7f58e0a00640 1 -- 172.21.15.90:0/3545488848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc102c10 msgr2=0x7f58dc102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.160+0000 7f58e0a00640 1 --2- 172.21.15.90:0/3545488848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc102c10 0x7f58dc102ff0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009900 tx=0x7f58cc02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.160+0000 7f58e0a00640 1 -- 172.21.15.90:0/3545488848 shutdown_connections 2024-09-10T22:08:29.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.160+0000 7f58e0a00640 1 --2- 172.21.15.90:0/3545488848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58dc108fa0 0x7f58dc10f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.160+0000 7f58e0a00640 1 --2- 172.21.15.90:0/3545488848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58dc108620 0x7f58dc108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.160+0000 7f58e0a00640 1 --2- 172.21.15.90:0/3545488848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc102c10 0x7f58dc102ff0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.160+0000 7f58e0a00640 1 -- 172.21.15.90:0/3545488848 >> 172.21.15.90:0/3545488848 conn(0x7f58dc0fe660 msgr2=0x7f58dc100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.161+0000 7f58e0a00640 1 -- 172.21.15.90:0/3545488848 shutdown_connections 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.161+0000 7f58e0a00640 1 -- 172.21.15.90:0/3545488848 wait complete. 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.162+0000 7f58e0a00640 1 Processor -- start 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.162+0000 7f58e0a00640 1 -- start start 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.162+0000 7f58e0a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58dc102c10 0x7f58dc19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.162+0000 7f58e0a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc108620 0x7f58dc19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.162+0000 7f58e0a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58dc108fa0 0x7f58dc1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.162+0000 7f58e0a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc112b60 con 0x7f58dc108620 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.162+0000 7f58e0a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc1129e0 con 0x7f58dc102c10 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.162+0000 7f58e0a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc112ce0 con 0x7f58dc108fa0 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58d9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc108620 0x7f58dc19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:29.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58d9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc108620 0x7f58dc19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34578/0 (socket says 172.21.15.90:34578) 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58d9e00640 1 -- 172.21.15.90:0/2933189996 learned_addr learned my addr 172.21.15.90:0/2933189996 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58da800640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58dc102c10 0x7f58dc19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58db200640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58dc108fa0 0x7f58dc1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58d9e00640 1 -- 172.21.15.90:0/2933189996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58dc108fa0 msgr2=0x7f58dc1a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58d9e00640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58dc108fa0 0x7f58dc1a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58d9e00640 1 -- 172.21.15.90:0/2933189996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58dc102c10 msgr2=0x7f58dc19b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58d9e00640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58dc102c10 0x7f58dc19b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.163+0000 7f58d9e00640 1 -- 172.21.15.90:0/2933189996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58dc1a1250 con 0x7f58dc108620 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.164+0000 7f58d9e00640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc108620 0x7f58dc19b6e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f58c800d770 tx=0x7f58c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.164+0000 7f58d3400640 1 -- 172.21.15.90:0/2933189996 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8017070 con 0x7f58dc108620 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.164+0000 7f58d3400640 1 -- 172.21.15.90:0/2933189996 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f58c80054b0 con 0x7f58dc108620 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.164+0000 7f58d3400640 1 -- 172.21.15.90:0/2933189996 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8004c60 con 0x7f58dc108620 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.164+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc1a1540 con 0x7f58dc108620 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.164+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc1a7c00 con 0x7f58dc108620 2024-09-10T22:08:29.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.166+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc102ff0 con 0x7f58dc108620 2024-09-10T22:08:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.167+0000 7f58d3400640 1 -- 172.21.15.90:0/2933189996 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f58c8004e00 con 0x7f58dc108620 2024-09-10T22:08:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.169+0000 7f58d3400640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58c00b5500 0x7f58c00b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.169+0000 7f58d3400640 1 -- 172.21.15.90:0/2933189996 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7f58c80e5ae0 con 0x7f58dc108620 2024-09-10T22:08:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.169+0000 7f58da800640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58c00b5500 0x7f58c00b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.170+0000 7f58da800640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58c00b5500 0x7f58c00b79c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58cc006010 tx=0x7f58cc005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:29.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.173+0000 7f58d3400640 1 -- 172.21.15.90:0/2933189996 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f58c80ac290 con 0x7f58dc108620 2024-09-10T22:08:29.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:28 smithi171 ceph-mon[34138]: pgmap v555: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-09-10T22:08:29.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.415+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58dc19c2d0 con 0x7f58dc108620 2024-09-10T22:08:29.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.416+0000 7f58d3400640 1 -- 172.21.15.90:0/2933189996 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11986 (secure 0 0 0) 0x7f58c80abb90 con 0x7f58dc108620 2024-09-10T22:08:29.419 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:08:29.419 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"btime":"2024-09-10T22:08:03:817761+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:08:03.499417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:08:29.419 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-09-10T22:08:29.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.419+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58c00b5500 msgr2=0x7f58c00b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.419+0000 7f58e0a00640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58c00b5500 0x7f58c00b79c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f58cc006010 tx=0x7f58cc005e80 comp rx=0 tx=0).stop 2024-09-10T22:08:29.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.419+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc108620 msgr2=0x7f58dc19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:29.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.419+0000 7f58e0a00640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc108620 0x7f58dc19b6e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f58c800d770 tx=0x7f58c800dc30 comp rx=0 tx=0).stop 2024-09-10T22:08:29.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.419+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 shutdown_connections 2024-09-10T22:08:29.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.419+0000 7f58e0a00640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58c00b5500 0x7f58c00b79c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.419+0000 7f58e0a00640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f58dc108fa0 0x7f58dc1a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.419+0000 7f58e0a00640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58dc108620 0x7f58dc19b6e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.420+0000 7f58e0a00640 1 --2- 172.21.15.90:0/2933189996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f58dc102c10 0x7f58dc19b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:29.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.420+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 >> 172.21.15.90:0/2933189996 conn(0x7f58dc0fe660 msgr2=0x7f58dc075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:29.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.420+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 shutdown_connections 2024-09-10T22:08:29.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:29.420+0000 7f58e0a00640 1 -- 172.21.15.90:0/2933189996 wait complete. 2024-09-10T22:08:29.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.23 MB/sec execute 81 sec latency 26962.541 ms 2024-09-10T22:08:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:29 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2933189996' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:08:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:29 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2933189996' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:08:30.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:29 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2933189996' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:08:30.317 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id b34f2bd6 --release --await' 2024-09-10T22:08:30.317 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id b34f2bd6 --release --await 2024-09-10T22:08:30.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47746 1.21 MB/sec execute 82 sec latency 27962.616 ms 2024-09-10T22:08:30.689 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:08:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:30 smithi090 ceph-mon[30350]: pgmap v556: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-10T22:08:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:30 smithi154 ceph-mon[34495]: pgmap v556: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-10T22:08:31.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:30 smithi171 ceph-mon[34138]: pgmap v556: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-10T22:08:31.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.235+0000 7fba05e00640 1 -- 172.21.15.90:0/1667344007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 msgr2=0x7fba00115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.235+0000 7fba05e00640 1 --2- 172.21.15.90:0/1667344007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 0x7fba00115d80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009900 tx=0x7fb9f802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.235+0000 7fba05e00640 1 -- 172.21.15.90:0/1667344007 shutdown_connections 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.235+0000 7fba05e00640 1 --2- 172.21.15.90:0/1667344007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 0x7fba00115d80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.235+0000 7fba05e00640 1 --2- 172.21.15.90:0/1667344007 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba00077ff0 0x7fba00113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.235+0000 7fba05e00640 1 --2- 172.21.15.90:0/1667344007 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba000776d0 0x7fba00077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.235+0000 7fba05e00640 1 -- 172.21.15.90:0/1667344007 >> 172.21.15.90:0/1667344007 conn(0x7fba00100a50 msgr2=0x7fba00102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.235+0000 7fba05e00640 1 -- 172.21.15.90:0/1667344007 shutdown_connections 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.235+0000 7fba05e00640 1 -- 172.21.15.90:0/1667344007 wait complete. 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.236+0000 7fba05e00640 1 Processor -- start 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.236+0000 7fba05e00640 1 -- start start 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fba05e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba000776d0 0x7fba0019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fba05e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00077ff0 0x7fba0019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fba05e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 0x7fba001a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fba05e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fba00119210 con 0x7fba00113990 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fba05e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fba00119090 con 0x7fba000776d0 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fba05e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba00119390 con 0x7fba00077ff0 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9ff400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 0x7fba001a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9ff400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 0x7fba001a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34584/0 (socket says 172.21.15.90:34584) 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9ff400640 1 -- 172.21.15.90:0/231889787 learned_addr learned my addr 172.21.15.90:0/231889787 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9fea00640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba000776d0 0x7fba0019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9f6000640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00077ff0 0x7fba0019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9ff400640 1 -- 172.21.15.90:0/231889787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00077ff0 msgr2=0x7fba0019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9ff400640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00077ff0 0x7fba0019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9ff400640 1 -- 172.21.15.90:0/231889787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba000776d0 msgr2=0x7fba0019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9ff400640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba000776d0 0x7fba0019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.237+0000 7fb9ff400640 1 -- 172.21.15.90:0/231889787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba001a5850 con 0x7fba00113990 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.238+0000 7fb9ff400640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 0x7fba001a5080 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80057c0 tx=0x7fb9f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.238+0000 7fba04a00640 1 -- 172.21.15.90:0/231889787 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f803f030 con 0x7fba00113990 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.238+0000 7fba04a00640 1 -- 172.21.15.90:0/231889787 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9f8003a80 con 0x7fba00113990 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.238+0000 7fba05e00640 1 -- 172.21.15.90:0/231889787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba001a5ae0 con 0x7fba00113990 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.238+0000 7fba04a00640 1 -- 172.21.15.90:0/231889787 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8035070 con 0x7fba00113990 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.238+0000 7fba05e00640 1 -- 172.21.15.90:0/231889787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001ac200 con 0x7fba00113990 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.241+0000 7fba0880d740 1 Processor -- start 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.241+0000 7fba04a00640 1 -- 172.21.15.90:0/231889787 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb9f803bc00 con 0x7fba00113990 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.242+0000 7fba0880d740 1 -- start start 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.242+0000 7fba0880d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.242+0000 7fba0880d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e06910 0x5582b5e12360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.242+0000 7fba0880d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 0x5582b5e14c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.242+0000 7fba0880d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5582b5d7d290 con 0x5582b5e05ff0 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.242+0000 7fba0880d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5582b5d6c6c0 con 0x5582b5e06910 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.242+0000 7fba0880d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5582b5d704d0 con 0x5582b5e128a0 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.242+0000 7fb9f6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9f6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e063d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34592/0 (socket says 172.21.15.90:34592) 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9f6000640 1 -- 172.21.15.90:0/2624498004 learned_addr learned my addr 172.21.15.90:0/2624498004 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9fea00640 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e06910 0x5582b5e12360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9f6000640 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 0x5582b5e14c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9fea00640 1 -- 172.21.15.90:0/2624498004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 msgr2=0x5582b5e14c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9fea00640 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 0x5582b5e14c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9fea00640 1 -- 172.21.15.90:0/2624498004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 msgr2=0x5582b5e063d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:31.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9fea00640 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e063d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9fea00640 1 -- 172.21.15.90:0/2624498004 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b5cf9460 con 0x5582b5e06910 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9f6000640 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fba04a00640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb9e40b5450 0x7fb9e40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fb9fea00640 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e06910 0x5582b5e12360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009fb0 tx=0x7fb9f0003e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=658dc0116443e853 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.243+0000 7fba04a00640 1 -- 172.21.15.90:0/231889787 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb9f8106df0 con 0x7fba00113990 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fb9f6000640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb9e40b5450 0x7fb9e40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fb9efe00640 1 -- 172.21.15.90:0/2624498004 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0004b20 con 0x5582b5e06910 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fb9efe00640 1 -- 172.21.15.90:0/2624498004 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9f0004cc0 con 0x5582b5e06910 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fb9efe00640 1 -- 172.21.15.90:0/2624498004 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f00126b0 con 0x5582b5e06910 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fba0880d740 1 -- 172.21.15.90:0/2624498004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e06910 msgr2=0x5582b5e12360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fb9f6000640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb9e40b5450 0x7fb9e40b7910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8005fd0 tx=0x7fb9e8015040 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fba0880d740 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e06910 0x5582b5e12360 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009fb0 tx=0x7fb9f0003e00 comp rx=0 tx=0).stop 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fba0880d740 1 -- 172.21.15.90:0/2624498004 shutdown_connections 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fba0880d740 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 0x5582b5e14c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fba0880d740 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e06910 0x5582b5e12360 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.244+0000 7fba0880d740 1 --2- 172.21.15.90:0/2624498004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.245+0000 7fba0880d740 1 -- 172.21.15.90:0/2624498004 >> 172.21.15.90:0/2624498004 conn(0x5582b5e03e20 msgr2=0x5582b5e04fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.245+0000 7fba0880d740 1 -- 172.21.15.90:0/2624498004 shutdown_connections 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.245+0000 7fba0880d740 1 -- 172.21.15.90:0/2624498004 wait complete. 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.245+0000 7fba0880d740 1 -- start start 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.245+0000 7fba0880d740 1 Processor -- start 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fba0880d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e05ff0 0x5582b5e0f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fba0880d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e0fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fba0880d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 0x5582b5e102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fba0880d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5582b5bcdbb0 con 0x5582b5e06910 2024-09-10T22:08:31.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fba0880d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5582b5d076c0 con 0x5582b5e128a0 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fba0880d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5582b5e15550 con 0x5582b5e05ff0 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fba0880d740 10 client.0 _ll_register_callbacks cb 0x5582b5d22490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fb9f6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e0fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fb9f6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e0fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34596/0 (socket says 172.21.15.90:34596) 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fb9f6000640 1 -- 172.21.15.90:0/4059217227 learned_addr learned my addr 172.21.15.90:0/4059217227 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fb9fea00640 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e05ff0 0x5582b5e0f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fb9ff400640 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 0x5582b5e102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fb9f6000640 1 -- 172.21.15.90:0/4059217227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e05ff0 msgr2=0x5582b5e0f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fb9f6000640 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e05ff0 0x5582b5e0f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fb9f6000640 1 -- 172.21.15.90:0/4059217227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 msgr2=0x5582b5e102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:31.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.246+0000 7fb9f6000640 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 0x5582b5e102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.247+0000 7fb9f6000640 1 -- 172.21.15.90:0/4059217227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b5acab00 con 0x5582b5e06910 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.247+0000 7fb9f6000640 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e0fdb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80042a0 tx=0x7fb9e800b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2439b136004a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.247+0000 7fb9ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.247+0000 7fb9ecc00640 1 -- 172.21.15.90:0/4059217227 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e800e7b0 con 0x5582b5e06910 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.247+0000 7fba0880d740 1 -- 172.21.15.90:0/4059217227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f0004cc0 con 0x5582b5e06910 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.247+0000 7fb9ecc00640 1 -- 172.21.15.90:0/4059217227 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb9e800e950 con 0x5582b5e06910 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.247+0000 7fb9ecc00640 1 -- 172.21.15.90:0/4059217227 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb9e8030050 con 0x5582b5e06910 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.247+0000 7fba0880d740 10 client.15906 fetch_fsmap learned FSMap version 35 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.247+0000 7fba0880d740 1 -- 172.21.15.90:0/4059217227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5582b5b41960 con 0x5582b5e06910 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.248+0000 7fb9ecc00640 1 -- 172.21.15.90:0/4059217227 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9f0004cc0 con 0x5582b5e06910 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.248+0000 7fb9ecc00640 1 -- 172.21.15.90:0/4059217227 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fb9e8022c30 con 0x5582b5e06910 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.248+0000 7fba0880d740 10 client.15906 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:08:31.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.248+0000 7fba0880d740 10 client.15906 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.248+0000 7fba0880d740 20 client.15906 populate_metadata read hostname 'smithi090' 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.248+0000 7fba0880d740 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5582b5e22460 0x5582b5e24890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.248+0000 7fba0880d740 4 client.15906 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.248+0000 7fba0880d740 1 -- 172.21.15.90:0/4059217227 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582b5d7f1e0 con 0x5582b5e22460 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.249+0000 7fb9fea00640 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5582b5e22460 0x5582b5e24890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.249+0000 7fb9fea00640 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5582b5e22460 0x5582b5e24890 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45001c77009a6cc6 server_cookie=d89bf6436c1d28b0 in_seq=0 out_seq=0 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.249+0000 7fb9ecc00640 10 client.15906 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fb9ecc00640 1 -- 172.21.15.90:0/4059217227 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582b5d7f1e0 con 0x5582b5e22460 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fb9ecc00640 10 client.15906 handle_command_reply: tid=0 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fba0880d740 1 client.15906 shutdown 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fba0880d740 1 -- 172.21.15.90:0/4059217227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 msgr2=0x5582b5e0fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fba0880d740 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e0fdb0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80042a0 tx=0x7fb9e800b6b0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fba0880d740 1 -- 172.21.15.90:0/4059217227 shutdown_connections 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fba0880d740 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5582b5e22460 0x5582b5e24890 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fba0880d740 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 0x5582b5e102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fba0880d740 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e0fdb0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fb9ecc00640 0 client.15906 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fba0880d740 1 --2- 172.21.15.90:0/4059217227 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e05ff0 0x5582b5e0f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.250+0000 7fba0880d740 1 -- 172.21.15.90:0/4059217227 >> 172.21.15.90:0/4059217227 conn(0x5582b5e03e20 msgr2=0x5582b5e099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.251+0000 7fba0880d740 1 -- 172.21.15.90:0/4059217227 shutdown_connections 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.251+0000 7fba0880d740 1 -- 172.21.15.90:0/4059217227 wait complete. 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.251+0000 7fba0880d740 20 client.15906 trim_cache size 0 max 16384 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.268+0000 7fba0880d740 1 Processor -- start 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fba0880d740 1 -- start start 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fba0880d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e05ff0 0x5582b5e0acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fba0880d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e02c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fba0880d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 0x5582b5e0ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fba0880d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b6310 con 0x5582b5e06910 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fba0880d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b7e50 con 0x5582b5e05ff0 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fba0880d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b69e0 con 0x5582b5e128a0 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fb9f6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e02c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fb9f6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e02c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34606/0 (socket says 172.21.15.90:34606) 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fb9f6000640 1 -- 172.21.15.90:0/3435330595 learned_addr learned my addr 172.21.15.90:0/3435330595 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fb9fea00640 1 --2- 172.21.15.90:0/3435330595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e05ff0 0x5582b5e0acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.269+0000 7fb9ff400640 1 --2- 172.21.15.90:0/3435330595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 0x5582b5e0ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.270+0000 7fb9f6000640 1 -- 172.21.15.90:0/3435330595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 msgr2=0x5582b5e0ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.270+0000 7fb9f6000640 1 --2- 172.21.15.90:0/3435330595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 0x5582b5e0ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.270+0000 7fb9f6000640 1 -- 172.21.15.90:0/3435330595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e05ff0 msgr2=0x5582b5e0acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.270+0000 7fb9f6000640 1 --2- 172.21.15.90:0/3435330595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e05ff0 0x5582b5e0acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.270+0000 7fb9f6000640 1 -- 172.21.15.90:0/3435330595 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b5e048a0 con 0x5582b5e06910 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.270+0000 7fb9f6000640 1 --2- 172.21.15.90:0/3435330595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e02c00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800b860 tx=0x7fb9e800e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430efcd50db75ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.270+0000 7fb9ef400640 1 -- 172.21.15.90:0/3435330595 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e80107c0 con 0x5582b5e06910 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.270+0000 7fb9ef400640 1 -- 172.21.15.90:0/3435330595 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9e801b070 con 0x5582b5e06910 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.270+0000 7fb9ef400640 1 -- 172.21.15.90:0/3435330595 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8004730 con 0x5582b5e06910 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 -- 172.21.15.90:0/3435330595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 msgr2=0x5582b5e02c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 --2- 172.21.15.90:0/3435330595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e02c00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800b860 tx=0x7fb9e800e3a0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 -- 172.21.15.90:0/3435330595 shutdown_connections 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 --2- 172.21.15.90:0/3435330595 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e128a0 0x5582b5e0ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 --2- 172.21.15.90:0/3435330595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e06910 0x5582b5e02c00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 --2- 172.21.15.90:0/3435330595 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e05ff0 0x5582b5e0acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 -- 172.21.15.90:0/3435330595 >> 172.21.15.90:0/3435330595 conn(0x5582b5e03e20 msgr2=0x5582b5e116c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 -- 172.21.15.90:0/3435330595 shutdown_connections 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 -- 172.21.15.90:0/3435330595 wait complete. 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.271+0000 7fba0880d740 1 -- start start 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fba0880d740 1 Processor -- start 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fba0880d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e17790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fba0880d740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e06910 0x5582b5e17cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fba0880d740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 0x5582b5e18210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fba0880d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b7780 con 0x5582b5e05ff0 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fba0880d740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b70b0 con 0x5582b5e128a0 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fba0880d740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b42f0 con 0x5582b5e06910 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fba0880d740 10 client.0 _ll_register_callbacks cb 0x7fb9e800beb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fb9fea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e17790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fb9fea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e17790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34612/0 (socket says 172.21.15.90:34612) 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fb9fea00640 1 -- 172.21.15.90:0/1128238477 learned_addr learned my addr 172.21.15.90:0/1128238477 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fb9f6000640 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e06910 0x5582b5e17cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.272+0000 7fb9ff400640 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 0x5582b5e18210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fb9fea00640 1 -- 172.21.15.90:0/1128238477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e06910 msgr2=0x5582b5e17cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fb9fea00640 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e06910 0x5582b5e17cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fb9fea00640 1 -- 172.21.15.90:0/1128238477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 msgr2=0x5582b5e18210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fb9fea00640 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 0x5582b5e18210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fb9fea00640 1 -- 172.21.15.90:0/1128238477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b5e8f3f0 con 0x5582b5e05ff0 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fb9fea00640 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e17790 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009fb0 tx=0x7fb9f00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ffbd57bbd0a9cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fb9ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fb9ecc00640 1 -- 172.21.15.90:0/1128238477 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0033060 con 0x5582b5e05ff0 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fb9ecc00640 1 -- 172.21.15.90:0/1128238477 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fb9f0005260 con 0x5582b5e05ff0 2024-09-10T22:08:31.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.273+0000 7fba0880d740 1 -- 172.21.15.90:0/1128238477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e801b070 con 0x5582b5e05ff0 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fb9ecc00640 1 -- 172.21.15.90:0/1128238477 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb9f003a050 con 0x5582b5e05ff0 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fba0880d740 10 client.15912 fetch_fsmap learned FSMap version 35 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fba0880d740 1 -- 172.21.15.90:0/1128238477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5582b5e90730 con 0x5582b5e05ff0 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fb9ecc00640 1 -- 172.21.15.90:0/1128238477 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9e801b070 con 0x5582b5e05ff0 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fb9ecc00640 1 -- 172.21.15.90:0/1128238477 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fb9f000f040 con 0x5582b5e05ff0 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fba0880d740 10 client.15912 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fba0880d740 10 client.15912 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fba0880d740 20 client.15912 populate_metadata read hostname 'smithi090' 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fba0880d740 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5582b5e90d50 0x5582b5e93170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fba0880d740 4 client.15912 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b34f2bd6", "release": true, "await": true}] 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.274+0000 7fba0880d740 1 -- 172.21.15.90:0/1128238477 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b34f2bd6", "release": true, "await": true}) -- 0x5582b5d7ac90 con 0x5582b5e90d50 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.275+0000 7fb9f6000640 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5582b5e90d50 0x5582b5e93170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:31.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.275+0000 7fb9f6000640 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5582b5e90d50 0x5582b5e93170 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bd7c9229887705 server_cookie=72bc5a678cb9ea3 in_seq=0 out_seq=0 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.275+0000 7fb9ecc00640 10 client.15912 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fb9ecc00640 1 -- 172.21.15.90:0/1128238477 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5582b5d7ac90 con 0x5582b5e90d50 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fb9ecc00640 10 client.15912 handle_command_reply: tid=0 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fba0880d740 1 client.15912 shutdown 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fba0880d740 1 -- 172.21.15.90:0/1128238477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 msgr2=0x5582b5e17790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fba0880d740 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e17790 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009fb0 tx=0x7fb9f00044c0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fba0880d740 1 -- 172.21.15.90:0/1128238477 shutdown_connections 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fba0880d740 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5582b5e90d50 0x5582b5e93170 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fba0880d740 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5582b5e128a0 0x5582b5e18210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fba0880d740 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5582b5e06910 0x5582b5e17cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fb9ecc00640 0 client.15912 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fba0880d740 1 --2- 172.21.15.90:0/1128238477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582b5e05ff0 0x5582b5e17790 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.282+0000 7fba0880d740 1 -- 172.21.15.90:0/1128238477 >> 172.21.15.90:0/1128238477 conn(0x5582b5e03e20 msgr2=0x5582b5e1d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.283+0000 7fba0880d740 1 -- 172.21.15.90:0/1128238477 shutdown_connections 2024-09-10T22:08:31.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.283+0000 7fba0880d740 1 -- 172.21.15.90:0/1128238477 wait complete. 2024-09-10T22:08:31.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.283+0000 7fba0880d740 20 client.15912 trim_cache size 0 max 16384 2024-09-10T22:08:31.336 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:08:31.336 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 35, 2024-09-10T22:08:31.336 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:08:31.336 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 16, 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "b34f2bd6": { 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 16, 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:08:31.337 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 -- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb9e40b5450 msgr2=0x7fb9e40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb9e40b5450 0x7fb9e40b7910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8005fd0 tx=0x7fb9e8015040 comp rx=0 tx=0).stop 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 -- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 msgr2=0x7fba001a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:31.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 0x7fba001a5080 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80057c0 tx=0x7fb9f8004270 comp rx=0 tx=0).stop 2024-09-10T22:08:31.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 -- 172.21.15.90:0/231889787 shutdown_connections 2024-09-10T22:08:31.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb9e40b5450 0x7fb9e40b7910 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba00113990 0x7fba001a5080 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fba00077ff0 0x7fba0019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 --2- 172.21.15.90:0/231889787 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fba000776d0 0x7fba0019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:31.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 -- 172.21.15.90:0/231889787 >> 172.21.15.90:0/231889787 conn(0x7fba00100a50 msgr2=0x7fba00102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:31.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 -- 172.21.15.90:0/231889787 shutdown_connections 2024-09-10T22:08:31.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:31.284+0000 7fba05e00640 1 -- 172.21.15.90:0/231889787 wait complete. 2024-09-10T22:08:31.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 47804 1.20 MB/sec execute 83 sec latency 28781.124 ms 2024-09-10T22:08:32.332 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b34f2bd6', total seconds elapsed: 34.2 2024-09-10T22:08:32.333 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T22:08:32.333 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:08:32.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 48231 1.19 MB/sec execute 84 sec latency 21.416 ms 2024-09-10T22:08:32.738 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:08:33.067 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:32 smithi090 ceph-mon[30350]: pgmap v557: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-10T22:08:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:32 smithi154 ceph-mon[34495]: pgmap v557: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-10T22:08:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:32 smithi171 ceph-mon[34138]: pgmap v557: 129 pgs: 129 active+clean; 54 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-10T22:08:33.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.354+0000 7fed8f200640 1 -- 172.21.15.90:0/1990793745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 msgr2=0x7fed90111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:33.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.354+0000 7fed8f200640 1 --2- 172.21.15.90:0/1990793745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 0x7fed90111610 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fed88009ca0 tx=0x7fed8802a8b0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.354+0000 7fed8f200640 1 -- 172.21.15.90:0/1990793745 shutdown_connections 2024-09-10T22:08:33.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.354+0000 7fed8f200640 1 --2- 172.21.15.90:0/1990793745 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 0x7fed90111610 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.354+0000 7fed8f200640 1 --2- 172.21.15.90:0/1990793745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed90075f40 0x7fed9010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.354+0000 7fed8f200640 1 --2- 172.21.15.90:0/1990793745 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed90075540 0x7fed90075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.354+0000 7fed8f200640 1 -- 172.21.15.90:0/1990793745 >> 172.21.15.90:0/1990793745 conn(0x7fed900fe6d0 msgr2=0x7fed90100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:33.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.354+0000 7fed8f200640 1 -- 172.21.15.90:0/1990793745 shutdown_connections 2024-09-10T22:08:33.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.354+0000 7fed8f200640 1 -- 172.21.15.90:0/1990793745 wait complete. 2024-09-10T22:08:33.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.355+0000 7fed8f200640 1 Processor -- start 2024-09-10T22:08:33.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.355+0000 7fed8f200640 1 -- start start 2024-09-10T22:08:33.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed90075540 0x7fed9019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:33.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed90075f40 0x7fed9019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:33.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 0x7fed901a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:33.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fed90114c80 con 0x7fed90075540 2024-09-10T22:08:33.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8f200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fed90114b00 con 0x7fed9010f220 2024-09-10T22:08:33.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fed90114e00 con 0x7fed90075f40 2024-09-10T22:08:33.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed90075540 0x7fed9019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:33.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 0x7fed901a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:33.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8d400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed90075f40 0x7fed9019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:33.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed90075540 0x7fed9019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34632/0 (socket says 172.21.15.90:34632) 2024-09-10T22:08:33.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8de00640 1 -- 172.21.15.90:0/3298765327 learned_addr learned my addr 172.21.15.90:0/3298765327 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:33.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.356+0000 7fed8e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 0x7fed901a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:57990/0 (socket says 172.21.15.90:57990) 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed8e800640 1 -- 172.21.15.90:0/3298765327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed90075f40 msgr2=0x7fed9019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed8e800640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed90075f40 0x7fed9019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed8e800640 1 -- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed90075540 msgr2=0x7fed9019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed8e800640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed90075540 0x7fed9019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed8e800640 1 -- 172.21.15.90:0/3298765327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed901a11e0 con 0x7fed9010f220 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed8d400640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed90075f40 0x7fed9019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed8de00640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed90075540 0x7fed9019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed8e800640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 0x7fed901a0a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed88005e30 tx=0x7fed88030d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed86a00640 1 -- 172.21.15.90:0/3298765327 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88004040 con 0x7fed9010f220 2024-09-10T22:08:33.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.357+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed901a1470 con 0x7fed9010f220 2024-09-10T22:08:33.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.358+0000 7fed86a00640 1 -- 172.21.15.90:0/3298765327 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fed88040070 con 0x7fed9010f220 2024-09-10T22:08:33.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.358+0000 7fed86a00640 1 -- 172.21.15.90:0/3298765327 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed88045a40 con 0x7fed9010f220 2024-09-10T22:08:33.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.358+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed901a7b90 con 0x7fed9010f220 2024-09-10T22:08:33.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.359+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed90075920 con 0x7fed9010f220 2024-09-10T22:08:33.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.361+0000 7fed86a00640 1 -- 172.21.15.90:0/3298765327 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fed8804d030 con 0x7fed9010f220 2024-09-10T22:08:33.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.363+0000 7fed86a00640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fed740b5500 0x7fed740b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:33.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.363+0000 7fed86a00640 1 -- 172.21.15.90:0/3298765327 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fed88106f60 con 0x7fed9010f220 2024-09-10T22:08:33.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.363+0000 7fed8de00640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fed740b5500 0x7fed740b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:33.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.364+0000 7fed8de00640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fed740b5500 0x7fed740b79c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed80011500 tx=0x7fed80005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:33.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.365+0000 7fed86a00640 1 -- 172.21.15.90:0/3298765327 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fed880cd710 con 0x7fed9010f220 2024-09-10T22:08:33.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 48580 1.19 MB/sec execute 85 sec latency 19.959 ms 2024-09-10T22:08:33.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.612+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed9019c260 con 0x7fed9010f220 2024-09-10T22:08:33.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.612+0000 7fed86a00640 1 -- 172.21.15.90:0/3298765327 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11986 (secure 0 0 0) 0x7fed880cd010 con 0x7fed9010f220 2024-09-10T22:08:33.615 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:08:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"btime":"2024-09-10T22:08:03:817761+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:08:03.499417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:08:33.615 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-09-10T22:08:33.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.615+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fed740b5500 msgr2=0x7fed740b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:33.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.615+0000 7fed8f200640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fed740b5500 0x7fed740b79c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed80011500 tx=0x7fed80005f90 comp rx=0 tx=0).stop 2024-09-10T22:08:33.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.615+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 msgr2=0x7fed901a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:33.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.615+0000 7fed8f200640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 0x7fed901a0a10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fed88005e30 tx=0x7fed88030d50 comp rx=0 tx=0).stop 2024-09-10T22:08:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.615+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 shutdown_connections 2024-09-10T22:08:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.615+0000 7fed8f200640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fed740b5500 0x7fed740b79c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.615+0000 7fed8f200640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fed9010f220 0x7fed901a0a10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.615+0000 7fed8f200640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fed90075f40 0x7fed9019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.615+0000 7fed8f200640 1 --2- 172.21.15.90:0/3298765327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed90075540 0x7fed9019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.616+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 >> 172.21.15.90:0/3298765327 conn(0x7fed900fe6d0 msgr2=0x7fed900ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.616+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 shutdown_connections 2024-09-10T22:08:33.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:33.616+0000 7fed8f200640 1 -- 172.21.15.90:0/3298765327 wait complete. 2024-09-10T22:08:33.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:33 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3298765327' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:08:34.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:33 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3298765327' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:08:34.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:33 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3298765327' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:08:34.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49061 1.24 MB/sec execute 86 sec latency 20.498 ms 2024-09-10T22:08:34.586 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T22:08:34.586 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T22:08:35.005 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:08:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:34 smithi090 ceph-mon[30350]: pgmap v558: 129 pgs: 129 active+clean; 55 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-10T22:08:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:34 smithi154 ceph-mon[34495]: pgmap v558: 129 pgs: 129 active+clean; 55 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-10T22:08:35.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:34 smithi171 ceph-mon[34138]: pgmap v558: 129 pgs: 129 active+clean; 55 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 19 op/s 2024-09-10T22:08:35.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49460 1.25 MB/sec execute 87 sec latency 17.441 ms 2024-09-10T22:08:35.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.631+0000 7fcf7a400640 1 -- 172.21.15.90:0/292094268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf74102c10 msgr2=0x7fcf74102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:35.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.631+0000 7fcf7a400640 1 --2- 172.21.15.90:0/292094268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf74102c10 0x7fcf74102ff0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009900 tx=0x7fcf6802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.632+0000 7fcf7a400640 1 -- 172.21.15.90:0/292094268 shutdown_connections 2024-09-10T22:08:35.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.632+0000 7fcf7a400640 1 --2- 172.21.15.90:0/292094268 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf74108fa0 0x7fcf7410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.632+0000 7fcf7a400640 1 --2- 172.21.15.90:0/292094268 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf74108620 0x7fcf74108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.632+0000 7fcf7a400640 1 --2- 172.21.15.90:0/292094268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf74102c10 0x7fcf74102ff0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.632+0000 7fcf7a400640 1 -- 172.21.15.90:0/292094268 >> 172.21.15.90:0/292094268 conn(0x7fcf740fe660 msgr2=0x7fcf74100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:35.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.632+0000 7fcf7a400640 1 -- 172.21.15.90:0/292094268 shutdown_connections 2024-09-10T22:08:35.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.632+0000 7fcf7a400640 1 -- 172.21.15.90:0/292094268 wait complete. 2024-09-10T22:08:35.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.633+0000 7fcf7a400640 1 Processor -- start 2024-09-10T22:08:35.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.633+0000 7fcf7a400640 1 -- start start 2024-09-10T22:08:35.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.633+0000 7fcf7a400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf74102c10 0x7fcf7410e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.633+0000 7fcf7a400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf74108620 0x7fcf7410ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.633+0000 7fcf7a400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf74108fa0 0x7fcf741a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.633+0000 7fcf7a400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74112b20 con 0x7fcf74102c10 2024-09-10T22:08:35.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.633+0000 7fcf7a400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741129a0 con 0x7fcf74108620 2024-09-10T22:08:35.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.633+0000 7fcf7a400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74112ca0 con 0x7fcf74108fa0 2024-09-10T22:08:35.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf79000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf74102c10 0x7fcf7410e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf73e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf74108620 0x7fcf7410ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf79000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf74102c10 0x7fcf7410e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34646/0 (socket says 172.21.15.90:34646) 2024-09-10T22:08:35.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf79000640 1 -- 172.21.15.90:0/4090930313 learned_addr learned my addr 172.21.15.90:0/4090930313 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:35.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf79a00640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf74108fa0 0x7fcf741a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf73e00640 1 -- 172.21.15.90:0/4090930313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf74108fa0 msgr2=0x7fcf741a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:35.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf73e00640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf74108fa0 0x7fcf741a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf73e00640 1 -- 172.21.15.90:0/4090930313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf74102c10 msgr2=0x7fcf7410e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:35.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf73e00640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf74102c10 0x7fcf7410e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf73e00640 1 -- 172.21.15.90:0/4090930313 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf741a3390 con 0x7fcf74108620 2024-09-10T22:08:35.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.634+0000 7fcf79a00640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf74108fa0 0x7fcf741a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:08:35.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.635+0000 7fcf73e00640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf74108620 0x7fcf7410ed80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000c970 tx=0x7fcf6000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:35.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.635+0000 7fcf71600640 1 -- 172.21.15.90:0/4090930313 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60007bb0 con 0x7fcf74108620 2024-09-10T22:08:35.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.635+0000 7fcf7a400640 1 -- 172.21.15.90:0/4090930313 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf741a3680 con 0x7fcf74108620 2024-09-10T22:08:35.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.635+0000 7fcf71600640 1 -- 172.21.15.90:0/4090930313 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf60007d50 con 0x7fcf74108620 2024-09-10T22:08:35.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.635+0000 7fcf71600640 1 -- 172.21.15.90:0/4090930313 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600186f0 con 0x7fcf74108620 2024-09-10T22:08:35.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.635+0000 7fcf7a400640 1 -- 172.21.15.90:0/4090930313 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf741abf30 con 0x7fcf74108620 2024-09-10T22:08:35.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.639+0000 7fcf7e435740 1 Processor -- start 2024-09-10T22:08:35.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.639+0000 7fcf7e435740 1 -- start start 2024-09-10T22:08:35.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6c2f0 0x560d9da6c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da78660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da78ba0 0x560d9da7af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf7e435740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560d9d9e1ca0 con 0x560d9da6cc10 2024-09-10T22:08:35.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf7e435740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560d9d9d10d0 con 0x560d9da78ba0 2024-09-10T22:08:35.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf7e435740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560d9d9d4ee0 con 0x560d9da6c2f0 2024-09-10T22:08:35.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf79000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da78660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf79000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da78660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34650/0 (socket says 172.21.15.90:34650) 2024-09-10T22:08:35.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf79000640 1 -- 172.21.15.90:0/3374734589 learned_addr learned my addr 172.21.15.90:0/3374734589 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:35.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf79a00640 1 --2- 172.21.15.90:0/3374734589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6c2f0 0x560d9da6c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.640+0000 7fcf71600640 1 -- 172.21.15.90:0/4090930313 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fcf6000f450 con 0x7fcf74108620 2024-09-10T22:08:35.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.641+0000 7fcf79000640 1 -- 172.21.15.90:0/3374734589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6c2f0 msgr2=0x560d9da6c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:35.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.641+0000 7fcf79000640 1 --2- 172.21.15.90:0/3374734589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6c2f0 0x560d9da6c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.641+0000 7fcf79000640 1 -- 172.21.15.90:0/3374734589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da78ba0 msgr2=0x560d9da7af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:35.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.641+0000 7fcf79000640 1 --2- 172.21.15.90:0/3374734589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da78ba0 0x560d9da7af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.641+0000 7fcf79000640 1 -- 172.21.15.90:0/3374734589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9d983400 con 0x560d9da6cc10 2024-09-10T22:08:35.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.641+0000 7fcf79000640 1 --2- 172.21.15.90:0/3374734589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da78660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcf68004340 tx=0x7fcf68004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff79268057bcbad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:35.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.641+0000 7fcf66000640 1 -- 172.21.15.90:0/3374734589 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68040070 con 0x560d9da6cc10 2024-09-10T22:08:35.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf66000640 1 -- 172.21.15.90:0/3374734589 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf68004700 con 0x560d9da6cc10 2024-09-10T22:08:35.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf66000640 1 -- 172.21.15.90:0/3374734589 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68035040 con 0x560d9da6cc10 2024-09-10T22:08:35.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf71600640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf5c0b5500 0x7fcf5c0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf7e435740 1 -- 172.21.15.90:0/3374734589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 msgr2=0x560d9da78660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:35.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf7e435740 1 --2- 172.21.15.90:0/3374734589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da78660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fcf68004340 tx=0x7fcf68004370 comp rx=0 tx=0).stop 2024-09-10T22:08:35.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf71600640 1 -- 172.21.15.90:0/4090930313 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fcf600e6470 con 0x7fcf74108620 2024-09-10T22:08:35.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf7e435740 1 -- 172.21.15.90:0/3374734589 shutdown_connections 2024-09-10T22:08:35.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf7e435740 1 --2- 172.21.15.90:0/3374734589 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da78ba0 0x560d9da7af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf7e435740 1 --2- 172.21.15.90:0/3374734589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da78660 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf7e435740 1 --2- 172.21.15.90:0/3374734589 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6c2f0 0x560d9da6c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.642+0000 7fcf7e435740 1 -- 172.21.15.90:0/3374734589 >> 172.21.15.90:0/3374734589 conn(0x560d9da6a120 msgr2=0x560d9da6b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:35.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.643+0000 7fcf79a00640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf5c0b5500 0x7fcf5c0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.643+0000 7fcf7e435740 1 -- 172.21.15.90:0/3374734589 shutdown_connections 2024-09-10T22:08:35.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.643+0000 7fcf7e435740 1 -- 172.21.15.90:0/3374734589 wait complete. 2024-09-10T22:08:35.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.643+0000 7fcf7e435740 1 -- start start 2024-09-10T22:08:35.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.643+0000 7fcf79a00640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf5c0b5500 0x7fcf5c0b79c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c005fd0 tx=0x7fcf6c0098a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.644+0000 7fcf7e435740 1 Processor -- start 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.644+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da78490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.644+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6cc10 0x560d9da758e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.644+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da78ba0 0x560d9da75f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.644+0000 7fcf7e435740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560d9d8327a0 con 0x560d9da6c2f0 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.644+0000 7fcf7e435740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560d9d976a10 con 0x560d9da78ba0 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.644+0000 7fcf7e435740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560d9da7b7c0 con 0x560d9da6cc10 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.644+0000 7fcf7e435740 10 client.0 _ll_register_callbacks cb 0x560d9d986be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf79000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da78490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf79000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da78490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34660/0 (socket says 172.21.15.90:34660) 2024-09-10T22:08:35.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf73e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6cc10 0x560d9da758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf79000640 1 -- 172.21.15.90:0/761031443 learned_addr learned my addr 172.21.15.90:0/761031443 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:35.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf79a00640 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da78ba0 0x560d9da75f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf73e00640 1 -- 172.21.15.90:0/761031443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da78ba0 msgr2=0x560d9da75f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:35.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf73e00640 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da78ba0 0x560d9da75f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf73e00640 1 -- 172.21.15.90:0/761031443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 msgr2=0x560d9da78490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:35.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf73e00640 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da78490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.645+0000 7fcf73e00640 1 -- 172.21.15.90:0/761031443 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9d73eb90 con 0x560d9da6cc10 2024-09-10T22:08:35.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.646+0000 7fcf73e00640 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6cc10 0x560d9da758e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf600e37f0 tx=0x7fcf600abc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8e00624140ff7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:35.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.646+0000 7fcf5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:08:35.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.646+0000 7fcf5aa00640 1 -- 172.21.15.90:0/761031443 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600e3c80 con 0x560d9da6cc10 2024-09-10T22:08:35.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.646+0000 7fcf5aa00640 1 -- 172.21.15.90:0/761031443 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fcf600b5620 con 0x560d9da6cc10 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.646+0000 7fcf7e435740 1 -- 172.21.15.90:0/761031443 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf68004700 con 0x560d9da6cc10 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.646+0000 7fcf5aa00640 1 -- 172.21.15.90:0/761031443 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcf600acc20 con 0x560d9da6cc10 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.646+0000 7fcf7e435740 10 client.25567 fetch_fsmap learned FSMap version 35 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.646+0000 7fcf7e435740 1 -- 172.21.15.90:0/761031443 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d9da87120 con 0x560d9da6cc10 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.647+0000 7fcf5aa00640 1 -- 172.21.15.90:0/761031443 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf68004700 con 0x560d9da6cc10 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.647+0000 7fcf5aa00640 1 -- 172.21.15.90:0/761031443 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fcf60093040 con 0x560d9da6cc10 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.647+0000 7fcf7e435740 10 client.25567 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.647+0000 7fcf7e435740 10 client.25567 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.647+0000 7fcf7e435740 20 client.25567 populate_metadata read hostname 'smithi090' 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.647+0000 7fcf7e435740 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560d9da885e0 0x560d9da8a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.647+0000 7fcf7e435740 4 client.25567 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.647+0000 7fcf7e435740 1 -- 172.21.15.90:0/761031443 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d9d9e3bf0 con 0x560d9da885e0 2024-09-10T22:08:35.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.648+0000 7fcf79000640 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560d9da885e0 0x560d9da8a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.648+0000 7fcf79000640 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560d9da885e0 0x560d9da8a9d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48886372c251a798 server_cookie=6cb2dd0c72d302db in_seq=0 out_seq=0 2024-09-10T22:08:35.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.648+0000 7fcf5aa00640 10 client.25567 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:35.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.649+0000 7fcf5aa00640 1 -- 172.21.15.90:0/761031443 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d9d9e3bf0 con 0x560d9da885e0 2024-09-10T22:08:35.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.649+0000 7fcf5aa00640 10 client.25567 handle_command_reply: tid=0 2024-09-10T22:08:35.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.649+0000 7fcf7e435740 1 client.25567 shutdown 2024-09-10T22:08:35.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 -- 172.21.15.90:0/761031443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6cc10 msgr2=0x560d9da758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:35.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6cc10 0x560d9da758e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf600e37f0 tx=0x7fcf600abc90 comp rx=0 tx=0).stop 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 -- 172.21.15.90:0/761031443 shutdown_connections 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560d9da885e0 0x560d9da8a9d0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da78ba0 0x560d9da75f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da6cc10 0x560d9da758e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 --2- 172.21.15.90:0/761031443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da78490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 -- 172.21.15.90:0/761031443 >> 172.21.15.90:0/761031443 conn(0x560d9da6a120 msgr2=0x560d9da6fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf5aa00640 0 client.25567 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 -- 172.21.15.90:0/761031443 shutdown_connections 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 1 -- 172.21.15.90:0/761031443 wait complete. 2024-09-10T22:08:35.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.650+0000 7fcf7e435740 20 client.25567 trim_cache size 0 max 16384 2024-09-10T22:08:35.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf7e435740 1 Processor -- start 2024-09-10T22:08:35.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf7e435740 1 -- start start 2024-09-10T22:08:35.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6c2f0 0x560d9da69af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 0x560d9da72af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf7e435740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b5fb0 con 0x560d9da6cc10 2024-09-10T22:08:35.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf7e435740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b7af0 con 0x560d9da6c2f0 2024-09-10T22:08:35.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf7e435740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b6680 con 0x560d9da78ba0 2024-09-10T22:08:35.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf79000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6c2f0 0x560d9da69af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf79000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6c2f0 0x560d9da69af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:58038/0 (socket says 172.21.15.90:58038) 2024-09-10T22:08:35.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf73e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.674+0000 7fcf79000640 1 -- 172.21.15.90:0/2528351517 learned_addr learned my addr 172.21.15.90:0/2528351517 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:35.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.675+0000 7fcf79a00640 1 --2- 172.21.15.90:0/2528351517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 0x560d9da72af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.675+0000 7fcf73e00640 1 -- 172.21.15.90:0/2528351517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 msgr2=0x560d9da72af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:35.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.675+0000 7fcf73e00640 1 --2- 172.21.15.90:0/2528351517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 0x560d9da72af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.675+0000 7fcf73e00640 1 -- 172.21.15.90:0/2528351517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6c2f0 msgr2=0x560d9da69af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:35.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.675+0000 7fcf73e00640 1 --2- 172.21.15.90:0/2528351517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6c2f0 0x560d9da69af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.675+0000 7fcf73e00640 1 -- 172.21.15.90:0/2528351517 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d9da6aba0 con 0x560d9da6cc10 2024-09-10T22:08:35.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.675+0000 7fcf79000640 1 --2- 172.21.15.90:0/2528351517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6c2f0 0x560d9da69af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:08:35.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.675+0000 7fcf73e00640 1 --2- 172.21.15.90:0/2528351517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da779e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fcf600ac220 tx=0x7fcf600b59b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89c5ce400bee42c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:35.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.676+0000 7fcf65600640 1 -- 172.21.15.90:0/2528351517 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600ac2f0 con 0x560d9da6cc10 2024-09-10T22:08:35.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.676+0000 7fcf65600640 1 -- 172.21.15.90:0/2528351517 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcf600b53f0 con 0x560d9da6cc10 2024-09-10T22:08:35.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.676+0000 7fcf65600640 1 -- 172.21.15.90:0/2528351517 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf600ab7b0 con 0x560d9da6cc10 2024-09-10T22:08:35.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.676+0000 7fcf7e435740 1 -- 172.21.15.90:0/2528351517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 msgr2=0x560d9da779e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:35.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.676+0000 7fcf7e435740 1 --2- 172.21.15.90:0/2528351517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da779e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fcf600ac220 tx=0x7fcf600b59b0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.677+0000 7fcf7e435740 1 -- 172.21.15.90:0/2528351517 shutdown_connections 2024-09-10T22:08:35.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.677+0000 7fcf7e435740 1 --2- 172.21.15.90:0/2528351517 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 0x560d9da72af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.677+0000 7fcf7e435740 1 --2- 172.21.15.90:0/2528351517 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6cc10 0x560d9da779e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.677+0000 7fcf7e435740 1 --2- 172.21.15.90:0/2528351517 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6c2f0 0x560d9da69af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.677+0000 7fcf7e435740 1 -- 172.21.15.90:0/2528351517 >> 172.21.15.90:0/2528351517 conn(0x560d9da6a120 msgr2=0x560d9da84230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.677+0000 7fcf7e435740 1 -- 172.21.15.90:0/2528351517 shutdown_connections 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.677+0000 7fcf7e435740 1 -- 172.21.15.90:0/2528351517 wait complete. 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.677+0000 7fcf7e435740 1 -- start start 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf7e435740 1 Processor -- start 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da7e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6cc10 0x560d9da7ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf7e435740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 0x560d9daee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf7e435740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b7420 con 0x560d9da6c2f0 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf7e435740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b6d50 con 0x560d9da6cc10 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf7e435740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741b3f90 con 0x560d9da78ba0 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf7e435740 10 client.0 _ll_register_callbacks cb 0x7fcf60007f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf79000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da7e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf79000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da7e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34678/0 (socket says 172.21.15.90:34678) 2024-09-10T22:08:35.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf79000640 1 -- 172.21.15.90:0/196396179 learned_addr learned my addr 172.21.15.90:0/196396179 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:08:35.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.678+0000 7fcf73e00640 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6cc10 0x560d9da7ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.679+0000 7fcf79a00640 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 0x560d9daee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.679+0000 7fcf79000640 1 -- 172.21.15.90:0/196396179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 msgr2=0x560d9daee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:35.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.679+0000 7fcf79000640 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 0x560d9daee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.679+0000 7fcf79000640 1 -- 172.21.15.90:0/196396179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6cc10 msgr2=0x560d9da7ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:08:35.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.679+0000 7fcf79000640 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6cc10 0x560d9da7ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.679+0000 7fcf79000640 1 -- 172.21.15.90:0/196396179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d9daeebb0 con 0x560d9da6c2f0 2024-09-10T22:08:35.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.679+0000 7fcf79000640 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da7e4e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcf6802a910 tx=0x7fcf6802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405c3d25cdf525f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.679+0000 7fcf5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf5aa00640 1 -- 172.21.15.90:0/196396179 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6805b060 con 0x560d9da6c2f0 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf5aa00640 1 -- 172.21.15.90:0/196396179 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fcf68050630 con 0x560d9da6c2f0 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf7e435740 1 -- 172.21.15.90:0/196396179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560d9d9df6a0 con 0x560d9da6c2f0 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf5aa00640 1 -- 172.21.15.90:0/196396179 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fcf68062050 con 0x560d9da6c2f0 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf7e435740 10 client.15936 fetch_fsmap learned FSMap version 35 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf7e435740 1 -- 172.21.15.90:0/196396179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560d9daf7140 con 0x560d9da6c2f0 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf5aa00640 1 -- 172.21.15.90:0/196396179 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560d9d9df6a0 con 0x560d9da6c2f0 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf5aa00640 1 -- 172.21.15.90:0/196396179 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fcf680567f0 con 0x560d9da6c2f0 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf7e435740 10 client.15936 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf7e435740 10 client.15936 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.680+0000 7fcf7e435740 20 client.15936 populate_metadata read hostname 'smithi090' 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.681+0000 7fcf7e435740 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560d9daf7570 0x560d9daf9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:08:35.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.681+0000 7fcf7e435740 4 client.15936 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T22:08:35.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.681+0000 7fcf7e435740 1 -- 172.21.15.90:0/196396179 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x560d9d9dcad0 con 0x560d9daf7570 2024-09-10T22:08:35.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.681+0000 7fcf73e00640 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560d9daf7570 0x560d9daf9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:08:35.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.681+0000 7fcf73e00640 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560d9daf7570 0x560d9daf9960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=750565ff96c2faa2 server_cookie=87fe95ff165b2fc0 in_seq=0 out_seq=0 2024-09-10T22:08:35.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.681+0000 7fcf5aa00640 10 client.15936 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:35.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf5aa00640 1 -- 172.21.15.90:0/196396179 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560d9d9dcad0 con 0x560d9daf7570 2024-09-10T22:08:35.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf5aa00640 10 client.15936 handle_command_reply: tid=0 2024-09-10T22:08:35.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 client.15936 shutdown 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 -- 172.21.15.90:0/196396179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 msgr2=0x560d9da7e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da7e4e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcf6802a910 tx=0x7fcf6802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 -- 172.21.15.90:0/196396179 shutdown_connections 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560d9daf7570 0x560d9daf9960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 35, 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 18, 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:08:35.759 INFO:teuthology.orchestra.run.smithi090.stdout: "df61d4a4": { 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 18, 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:08:35.760 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:08:35.761 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:08:35.761 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:35.761 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:35.761 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:35.761 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:35.761 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:08:35.761 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560d9da78ba0 0x560d9daee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560d9da6cc10 0x560d9da7ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 --2- 172.21.15.90:0/196396179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560d9da6c2f0 0x560d9da7e4e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf5aa00640 0 client.15936 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 -- 172.21.15.90:0/196396179 >> 172.21.15.90:0/196396179 conn(0x560d9da6a120 msgr2=0x560d9da68ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 -- 172.21.15.90:0/196396179 shutdown_connections 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.756+0000 7fcf7e435740 1 -- 172.21.15.90:0/196396179 wait complete. 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.757+0000 7fcf7e435740 20 client.15936 trim_cache size 0 max 16384 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.757+0000 7fcf7a400640 1 -- 172.21.15.90:0/4090930313 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf5c0b5500 msgr2=0x7fcf5c0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.757+0000 7fcf7a400640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf5c0b5500 0x7fcf5c0b79c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c005fd0 tx=0x7fcf6c0098a0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.757+0000 7fcf7a400640 1 -- 172.21.15.90:0/4090930313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf74108620 msgr2=0x7fcf7410ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.757+0000 7fcf7a400640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf74108620 0x7fcf7410ed80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000c970 tx=0x7fcf6000ce30 comp rx=0 tx=0).stop 2024-09-10T22:08:35.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.757+0000 7fcf7a400640 1 -- 172.21.15.90:0/4090930313 shutdown_connections 2024-09-10T22:08:35.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.757+0000 7fcf7a400640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcf5c0b5500 0x7fcf5c0b79c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.757+0000 7fcf7a400640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcf74108fa0 0x7fcf741a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.757+0000 7fcf7a400640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcf74108620 0x7fcf7410ed80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.758+0000 7fcf7a400640 1 --2- 172.21.15.90:0/4090930313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcf74102c10 0x7fcf7410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:08:35.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.758+0000 7fcf7a400640 1 -- 172.21.15.90:0/4090930313 >> 172.21.15.90:0/4090930313 conn(0x7fcf740fe660 msgr2=0x7fcf74075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:08:35.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.758+0000 7fcf7a400640 1 -- 172.21.15.90:0/4090930313 shutdown_connections 2024-09-10T22:08:35.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:08:35.758+0000 7fcf7a400640 1 -- 172.21.15.90:0/4090930313 wait complete. 2024-09-10T22:08:36.464 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.23 MB/sec execute 88 sec latency 731.346 ms 2024-09-10T22:08:36.744 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'df61d4a4', quiesce took 4.4 seconds. Will release after: 25.6 2024-09-10T22:08:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:36 smithi090 ceph-mon[30350]: pgmap v559: 129 pgs: 129 active+clean; 56 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 224 KiB/s wr, 28 op/s 2024-09-10T22:08:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:36 smithi154 ceph-mon[34495]: pgmap v559: 129 pgs: 129 active+clean; 56 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 224 KiB/s wr, 28 op/s 2024-09-10T22:08:37.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:36 smithi171 ceph-mon[34138]: pgmap v559: 129 pgs: 129 active+clean; 56 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 224 KiB/s wr, 28 op/s 2024-09-10T22:08:37.465 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.22 MB/sec execute 89 sec latency 1731.459 ms 2024-09-10T22:08:38.465 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.21 MB/sec execute 90 sec latency 2731.613 ms 2024-09-10T22:08:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:38 smithi090 ceph-mon[30350]: pgmap v560: 129 pgs: 129 active+clean; 57 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 275 KiB/s wr, 26 op/s 2024-09-10T22:08:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:38 smithi154 ceph-mon[34495]: pgmap v560: 129 pgs: 129 active+clean; 57 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 275 KiB/s wr, 26 op/s 2024-09-10T22:08:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:38 smithi171 ceph-mon[34138]: pgmap v560: 129 pgs: 129 active+clean; 57 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 275 KiB/s wr, 26 op/s 2024-09-10T22:08:39.465 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.19 MB/sec execute 91 sec latency 3731.740 ms 2024-09-10T22:08:40.465 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.18 MB/sec execute 92 sec latency 4731.865 ms 2024-09-10T22:08:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:40 smithi090 ceph-mon[30350]: pgmap v561: 129 pgs: 129 active+clean; 58 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 379 KiB/s wr, 42 op/s 2024-09-10T22:08:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:40 smithi154 ceph-mon[34495]: pgmap v561: 129 pgs: 129 active+clean; 58 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 379 KiB/s wr, 42 op/s 2024-09-10T22:08:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:40 smithi171 ceph-mon[34138]: pgmap v561: 129 pgs: 129 active+clean; 58 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 379 KiB/s wr, 42 op/s 2024-09-10T22:08:41.465 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.17 MB/sec execute 93 sec latency 5732.004 ms 2024-09-10T22:08:42.465 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.16 MB/sec execute 94 sec latency 6732.176 ms 2024-09-10T22:08:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:42 smithi090 ceph-mon[30350]: pgmap v562: 129 pgs: 129 active+clean; 58 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 393 KiB/s wr, 43 op/s 2024-09-10T22:08:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:42 smithi154 ceph-mon[34495]: pgmap v562: 129 pgs: 129 active+clean; 58 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 393 KiB/s wr, 43 op/s 2024-09-10T22:08:43.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:42 smithi171 ceph-mon[34138]: pgmap v562: 129 pgs: 129 active+clean; 58 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 393 KiB/s wr, 43 op/s 2024-09-10T22:08:43.465 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.14 MB/sec execute 95 sec latency 7732.303 ms 2024-09-10T22:08:44.465 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.13 MB/sec execute 96 sec latency 8732.391 ms 2024-09-10T22:08:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:44 smithi090 ceph-mon[30350]: pgmap v563: 129 pgs: 129 active+clean; 58 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 425 KiB/s wr, 44 op/s 2024-09-10T22:08:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:44 smithi154 ceph-mon[34495]: pgmap v563: 129 pgs: 129 active+clean; 58 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 425 KiB/s wr, 44 op/s 2024-09-10T22:08:45.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:44 smithi171 ceph-mon[34138]: pgmap v563: 129 pgs: 129 active+clean; 58 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 425 KiB/s wr, 44 op/s 2024-09-10T22:08:45.466 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.12 MB/sec execute 97 sec latency 9732.514 ms 2024-09-10T22:08:46.466 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.11 MB/sec execute 98 sec latency 10732.654 ms 2024-09-10T22:08:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:46 smithi090 ceph-mon[30350]: pgmap v564: 129 pgs: 129 active+clean; 58 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 419 KiB/s wr, 37 op/s 2024-09-10T22:08:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:46 smithi154 ceph-mon[34495]: pgmap v564: 129 pgs: 129 active+clean; 58 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 419 KiB/s wr, 37 op/s 2024-09-10T22:08:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:46 smithi171 ceph-mon[34138]: pgmap v564: 129 pgs: 129 active+clean; 58 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 419 KiB/s wr, 37 op/s 2024-09-10T22:08:47.466 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.10 MB/sec execute 99 sec latency 11732.793 ms 2024-09-10T22:08:48.466 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.09 MB/sec execute 100 sec latency 12732.912 ms 2024-09-10T22:08:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:48 smithi090 ceph-mon[30350]: pgmap v565: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 249 KiB/s wr, 26 op/s 2024-09-10T22:08:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:48 smithi154 ceph-mon[34495]: pgmap v565: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 249 KiB/s wr, 26 op/s 2024-09-10T22:08:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:48 smithi171 ceph-mon[34138]: pgmap v565: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 249 KiB/s wr, 26 op/s 2024-09-10T22:08:49.466 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.08 MB/sec execute 101 sec latency 13733.048 ms 2024-09-10T22:08:50.466 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.07 MB/sec execute 102 sec latency 14733.170 ms 2024-09-10T22:08:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:50 smithi090 ceph-mon[30350]: pgmap v566: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 198 KiB/s wr, 29 op/s 2024-09-10T22:08:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:50 smithi154 ceph-mon[34495]: pgmap v566: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 198 KiB/s wr, 29 op/s 2024-09-10T22:08:51.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:50 smithi171 ceph-mon[34138]: pgmap v566: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 198 KiB/s wr, 29 op/s 2024-09-10T22:08:51.466 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.05 MB/sec execute 103 sec latency 15733.240 ms 2024-09-10T22:08:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:08:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:08:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:08:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:51 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:08:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:08:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:08:52.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:08:52.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:51 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:08:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:08:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:08:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:08:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:08:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:51 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:08:52.466 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.04 MB/sec execute 104 sec latency 16733.313 ms 2024-09-10T22:08:53.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:52 smithi090 ceph-mon[30350]: pgmap v567: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 96 KiB/s wr, 15 op/s 2024-09-10T22:08:53.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:52 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-09-10T22:08:53.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:52 smithi090 ceph-mon[30350]: Unable to set osd_memory_target on smithi090 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-10T22:08:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:52 smithi154 ceph-mon[34495]: pgmap v567: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 96 KiB/s wr, 15 op/s 2024-09-10T22:08:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:52 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-09-10T22:08:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:52 smithi154 ceph-mon[34495]: Unable to set osd_memory_target on smithi090 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-10T22:08:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:52 smithi171 ceph-mon[34138]: pgmap v567: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 96 KiB/s wr, 15 op/s 2024-09-10T22:08:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:52 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-09-10T22:08:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:52 smithi171 ceph-mon[34138]: Unable to set osd_memory_target on smithi090 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-10T22:08:53.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.03 MB/sec execute 105 sec latency 17733.436 ms 2024-09-10T22:08:54.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.02 MB/sec execute 106 sec latency 18733.586 ms 2024-09-10T22:08:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:54 smithi090 ceph-mon[30350]: pgmap v568: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 82 KiB/s wr, 12 op/s 2024-09-10T22:08:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:54 smithi154 ceph-mon[34495]: pgmap v568: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 82 KiB/s wr, 12 op/s 2024-09-10T22:08:55.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:54 smithi171 ceph-mon[34138]: pgmap v568: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 82 KiB/s wr, 12 op/s 2024-09-10T22:08:55.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.02 MB/sec execute 107 sec latency 19733.639 ms 2024-09-10T22:08:56.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.01 MB/sec execute 108 sec latency 20733.780 ms 2024-09-10T22:08:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:56 smithi090 ceph-mon[30350]: pgmap v569: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 50 KiB/s wr, 13 op/s 2024-09-10T22:08:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:56 smithi154 ceph-mon[34495]: pgmap v569: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 50 KiB/s wr, 13 op/s 2024-09-10T22:08:57.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:56 smithi171 ceph-mon[34138]: pgmap v569: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 50 KiB/s wr, 13 op/s 2024-09-10T22:08:57.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 1.00 MB/sec execute 109 sec latency 21733.954 ms 2024-09-10T22:08:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:08:57 smithi154 ceph-mon[34495]: pgmap v570: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-10T22:08:58.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:08:57 smithi171 ceph-mon[34138]: pgmap v570: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-10T22:08:58.331 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:08:57 smithi090 ceph-mon[30350]: pgmap v570: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-09-10T22:08:58.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 0.99 MB/sec execute 110 sec latency 22734.048 ms 2024-09-10T22:08:59.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 0.98 MB/sec execute 111 sec latency 23734.093 ms 2024-09-10T22:09:00.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 0.97 MB/sec execute 112 sec latency 24734.215 ms 2024-09-10T22:09:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:00 smithi090 ceph-mon[30350]: pgmap v571: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T22:09:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:00 smithi154 ceph-mon[34495]: pgmap v571: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T22:09:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:00 smithi171 ceph-mon[34138]: pgmap v571: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-10T22:09:01.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 0.96 MB/sec execute 113 sec latency 25734.379 ms 2024-09-10T22:09:02.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:09:02.467 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 0.95 MB/sec execute 114 sec latency 26734.426 ms 2024-09-10T22:09:02.727 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:09:03.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:02 smithi090 ceph-mon[30350]: pgmap v572: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-10T22:09:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:02 smithi154 ceph-mon[34495]: pgmap v572: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-10T22:09:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:02 smithi171 ceph-mon[34138]: pgmap v572: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-10T22:09:03.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.266+0000 7fa447000640 1 -- 172.21.15.90:0/2311953364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 msgr2=0x7fa44010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.266+0000 7fa447000640 1 --2- 172.21.15.90:0/2311953364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 0x7fa44010ac60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009900 tx=0x7fa42c02a400 comp rx=0 tx=0).stop 2024-09-10T22:09:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.266+0000 7fa447000640 1 -- 172.21.15.90:0/2311953364 shutdown_connections 2024-09-10T22:09:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.266+0000 7fa447000640 1 --2- 172.21.15.90:0/2311953364 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44010b1a0 0x7fa4401119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.266+0000 7fa447000640 1 --2- 172.21.15.90:0/2311953364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 0x7fa44010ac60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.266+0000 7fa447000640 1 --2- 172.21.15.90:0/2311953364 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa440104e10 0x7fa4401051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.266+0000 7fa447000640 1 -- 172.21.15.90:0/2311953364 >> 172.21.15.90:0/2311953364 conn(0x7fa440100840 msgr2=0x7fa440102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.266+0000 7fa447000640 1 -- 172.21.15.90:0/2311953364 shutdown_connections 2024-09-10T22:09:03.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.266+0000 7fa447000640 1 -- 172.21.15.90:0/2311953364 wait complete. 2024-09-10T22:09:03.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.267+0000 7fa447000640 1 Processor -- start 2024-09-10T22:09:03.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.268+0000 7fa447000640 1 -- start start 2024-09-10T22:09:03.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.268+0000 7fa447000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa440104e10 0x7fa44019b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:03.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.268+0000 7fa447000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 0x7fa44019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:03.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.268+0000 7fa447000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44010b1a0 0x7fa4401a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.268+0000 7fa447000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa440114d10 con 0x7fa44010a820 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.268+0000 7fa447000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa440114b90 con 0x7fa440104e10 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.268+0000 7fa447000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa440114e90 con 0x7fa44010b1a0 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.268+0000 7fa445200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 0x7fa44019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa445200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 0x7fa44019b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41236/0 (socket says 172.21.15.90:41236) 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa445c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa440104e10 0x7fa44019b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa445200640 1 -- 172.21.15.90:0/367286497 learned_addr learned my addr 172.21.15.90:0/367286497 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa446600640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44010b1a0 0x7fa4401a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa445200640 1 -- 172.21.15.90:0/367286497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44010b1a0 msgr2=0x7fa4401a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa445200640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44010b1a0 0x7fa4401a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa445200640 1 -- 172.21.15.90:0/367286497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa440104e10 msgr2=0x7fa44019b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:03.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa445200640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa440104e10 0x7fa44019b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa445200640 1 -- 172.21.15.90:0/367286497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4401a1190 con 0x7fa44010a820 2024-09-10T22:09:03.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa446600640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44010b1a0 0x7fa4401a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:09:03.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.269+0000 7fa445c00640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa440104e10 0x7fa44019b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:09:03.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.270+0000 7fa445200640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 0x7fa44019b620 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009230 tx=0x7fa42c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:03.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.270+0000 7fa43e000640 1 -- 172.21.15.90:0/367286497 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c03f030 con 0x7fa44010a820 2024-09-10T22:09:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.270+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4401a1420 con 0x7fa44010a820 2024-09-10T22:09:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.270+0000 7fa43e000640 1 -- 172.21.15.90:0/367286497 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa42c003a80 con 0x7fa44010a820 2024-09-10T22:09:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.270+0000 7fa43e000640 1 -- 172.21.15.90:0/367286497 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c035070 con 0x7fa44010a820 2024-09-10T22:09:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.270+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4401a7b40 con 0x7fa44010a820 2024-09-10T22:09:03.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.271+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4401051f0 con 0x7fa44010a820 2024-09-10T22:09:03.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.273+0000 7fa43e000640 1 -- 172.21.15.90:0/367286497 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa42c03ba00 con 0x7fa44010a820 2024-09-10T22:09:03.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.275+0000 7fa43e000640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa4280b5500 0x7fa4280b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:03.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.276+0000 7fa43e000640 1 -- 172.21.15.90:0/367286497 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa42c1071f0 con 0x7fa44010a820 2024-09-10T22:09:03.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.276+0000 7fa445c00640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa4280b5500 0x7fa4280b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:03.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.277+0000 7fa445c00640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa4280b5500 0x7fa4280b79c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa43400bfd0 tx=0x7fa43400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:03.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.277+0000 7fa43e000640 1 -- 172.21.15.90:0/367286497 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa42c0cd9a0 con 0x7fa44010a820 2024-09-10T22:09:03.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 0.94 MB/sec execute 115 sec latency 27734.486 ms 2024-09-10T22:09:03.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.502+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4401a7dd0 con 0x7fa44010a820 2024-09-10T22:09:03.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.503+0000 7fa43e000640 1 -- 172.21.15.90:0/367286497 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11986 (secure 0 0 0) 0x7fa42c0e2e20 con 0x7fa44010a820 2024-09-10T22:09:03.505 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:09:03.505 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"btime":"2024-09-10T22:08:03:817761+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:08:03.499417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:09:03.505 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-09-10T22:09:03.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa4280b5500 msgr2=0x7fa4280b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:03.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa4280b5500 0x7fa4280b79c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa43400bfd0 tx=0x7fa43400b390 comp rx=0 tx=0).stop 2024-09-10T22:09:03.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 msgr2=0x7fa44019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:03.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 0x7fa44019b620 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009230 tx=0x7fa42c004270 comp rx=0 tx=0).stop 2024-09-10T22:09:03.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 shutdown_connections 2024-09-10T22:09:03.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa4280b5500 0x7fa4280b79c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa44010b1a0 0x7fa4401a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa44010a820 0x7fa44019b620 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 --2- 172.21.15.90:0/367286497 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa440104e10 0x7fa44019b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:03.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 >> 172.21.15.90:0/367286497 conn(0x7fa440100840 msgr2=0x7fa440102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:03.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 shutdown_connections 2024-09-10T22:09:03.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:03.505+0000 7fa447000640 1 -- 172.21.15.90:0/367286497 wait complete. 2024-09-10T22:09:04.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:03 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/367286497' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:09:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:03 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/367286497' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:09:04.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:03 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/367286497' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:09:04.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 0.94 MB/sec execute 116 sec latency 28734.530 ms 2024-09-10T22:09:04.996 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id df61d4a4 --release --await' 2024-09-10T22:09:04.996 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id df61d4a4 --release --await 2024-09-10T22:09:05.047 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:04 smithi090 ceph-mon[30350]: pgmap v573: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-10T22:09:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:04 smithi154 ceph-mon[34495]: pgmap v573: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-10T22:09:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:04 smithi171 ceph-mon[34138]: pgmap v573: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-09-10T22:09:05.371 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:09:05.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49480 0.93 MB/sec execute 117 sec latency 29734.588 ms 2024-09-10T22:09:05.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.989+0000 7fa8f9200640 1 -- 172.21.15.90:0/270575806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075f40 msgr2=0x7fa8f410ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:05.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.989+0000 7fa8f9200640 1 --2- 172.21.15.90:0/270575806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075f40 0x7fa8f410ee10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009900 tx=0x7fa8e002a410 comp rx=0 tx=0).stop 2024-09-10T22:09:05.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.989+0000 7fa8f9200640 1 -- 172.21.15.90:0/270575806 shutdown_connections 2024-09-10T22:09:05.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.989+0000 7fa8f9200640 1 --2- 172.21.15.90:0/270575806 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f410f3e0 0x7fa8f41117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.989+0000 7fa8f9200640 1 --2- 172.21.15.90:0/270575806 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075f40 0x7fa8f410ee10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.989+0000 7fa8f9200640 1 --2- 172.21.15.90:0/270575806 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8f4075540 0x7fa8f4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.989+0000 7fa8f9200640 1 -- 172.21.15.90:0/270575806 >> 172.21.15.90:0/270575806 conn(0x7fa8f40fe830 msgr2=0x7fa8f4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:05.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.989+0000 7fa8f9200640 1 -- 172.21.15.90:0/270575806 shutdown_connections 2024-09-10T22:09:05.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.989+0000 7fa8f9200640 1 -- 172.21.15.90:0/270575806 wait complete. 2024-09-10T22:09:05.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.990+0000 7fa8f9200640 1 Processor -- start 2024-09-10T22:09:05.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.990+0000 7fa8f9200640 1 -- start start 2024-09-10T22:09:05.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f9200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075540 0x7fa8f419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f9200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8f4075f40 0x7fa8f419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f9200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f410f3e0 0x7fa8f41a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f9200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4114e90 con 0x7fa8f4075540 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f9200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4114d10 con 0x7fa8f410f3e0 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f9200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4115010 con 0x7fa8f4075f40 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075540 0x7fa8f419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075540 0x7fa8f419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41246/0 (socket says 172.21.15.90:41246) 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1e00640 1 -- 172.21.15.90:0/3170947771 learned_addr learned my addr 172.21.15.90:0/3170947771 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1400640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8f4075f40 0x7fa8f419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f2800640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f410f3e0 0x7fa8f41a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1e00640 1 -- 172.21.15.90:0/3170947771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8f4075f40 msgr2=0x7fa8f419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8f4075f40 0x7fa8f419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1e00640 1 -- 172.21.15.90:0/3170947771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f410f3e0 msgr2=0x7fa8f41a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f410f3e0 0x7fa8f41a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1e00640 1 -- 172.21.15.90:0/3170947771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f41a1490 con 0x7fa8f4075540 2024-09-10T22:09:05.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f1400640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8f4075f40 0x7fa8f419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:09:05.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.991+0000 7fa8f2800640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f410f3e0 0x7fa8f41a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:09:05.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.992+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075540 0x7fa8f419b3e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400ea80 tx=0x7fa8e400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:05.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.992+0000 7fa8eaa00640 1 -- 172.21.15.90:0/3170947771 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e400ce40 con 0x7fa8f4075540 2024-09-10T22:09:05.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.992+0000 7fa8eaa00640 1 -- 172.21.15.90:0/3170947771 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8e4005020 con 0x7fa8f4075540 2024-09-10T22:09:05.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.992+0000 7fa8eaa00640 1 -- 172.21.15.90:0/3170947771 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40116d0 con 0x7fa8f4075540 2024-09-10T22:09:05.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.992+0000 7fa8f9200640 1 -- 172.21.15.90:0/3170947771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f41a1780 con 0x7fa8f4075540 2024-09-10T22:09:05.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.992+0000 7fa8f9200640 1 -- 172.21.15.90:0/3170947771 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f41a7e40 con 0x7fa8f4075540 2024-09-10T22:09:05.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.995+0000 7fa8fbdf4740 1 Processor -- start 2024-09-10T22:09:05.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.996+0000 7fa8fbdf4740 1 -- start start 2024-09-10T22:09:05.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.996+0000 7fa8eaa00640 1 -- 172.21.15.90:0/3170947771 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa8e400f040 con 0x7fa8f4075540 2024-09-10T22:09:05.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.996+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287e93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.996+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287f5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.996+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287f58a0 0x5650287f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.996+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x565028760290 con 0x5650287f58a0 2024-09-10T22:09:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.996+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56502874f6c0 con 0x5650287e9910 2024-09-10T22:09:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.996+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5650287534d0 con 0x5650287e8ff0 2024-09-10T22:09:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.996+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287f58a0 0x5650287f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287f58a0 0x5650287f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41252/0 (socket says 172.21.15.90:41252) 2024-09-10T22:09:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f1400640 1 -- 172.21.15.90:0/2755363095 learned_addr learned my addr 172.21.15.90:0/2755363095 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:09:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/2755363095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f2800640 1 --2- 172.21.15.90:0/2755363095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287e93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:05.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f1400640 1 -- 172.21.15.90:0/2755363095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 msgr2=0x5650287e93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:09:05.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f1400640 1 --2- 172.21.15.90:0/2755363095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287e93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f1400640 1 -- 172.21.15.90:0/2755363095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 msgr2=0x5650287f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:09:05.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f1400640 1 --2- 172.21.15.90:0/2755363095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:05.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f1400640 1 -- 172.21.15.90:0/2755363095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650286dc460 con 0x5650287f58a0 2024-09-10T22:09:06.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.997+0000 7fa8f1400640 1 --2- 172.21.15.90:0/2755363095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287f58a0 0x5650287f7c90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009900 tx=0x7fa8e00042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649ee701e6263a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:06.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.998+0000 7fa8eaa00640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8d80b5450 0x7fa8d80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.998+0000 7fa8df400640 1 -- 172.21.15.90:0/2755363095 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e003f070 con 0x5650287f58a0 2024-09-10T22:09:06.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.998+0000 7fa8eaa00640 1 -- 172.21.15.90:0/3170947771 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa8e40e66f0 con 0x7fa8f4075540 2024-09-10T22:09:06.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.998+0000 7fa8df400640 1 -- 172.21.15.90:0/2755363095 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8e00044d0 con 0x5650287f58a0 2024-09-10T22:09:06.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.998+0000 7fa8df400640 1 -- 172.21.15.90:0/2755363095 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0035070 con 0x5650287f58a0 2024-09-10T22:09:06.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.998+0000 7fa8f2800640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8d80b5450 0x7fa8d80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/2755363095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287f58a0 msgr2=0x5650287f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:06.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/2755363095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287f58a0 0x5650287f7c90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009900 tx=0x7fa8e00042e0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8f2800640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8d80b5450 0x7fa8d80b7910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa8f406c0b0 tx=0x7fa8ec005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:06.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/2755363095 shutdown_connections 2024-09-10T22:09:06.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/2755363095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287f58a0 0x5650287f7c90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/2755363095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/2755363095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287e93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/2755363095 >> 172.21.15.90:0/2755363095 conn(0x5650287e6e20 msgr2=0x5650287e7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/2755363095 shutdown_connections 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:05.999+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/2755363095 wait complete. 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.000+0000 7fa8fbdf4740 1 -- start start 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.000+0000 7fa8fbdf4740 1 Processor -- start 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.000+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.000+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.000+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287f58a0 0x5650287f3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.000+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5650285b0bb0 con 0x5650287e8ff0 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.000+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5650286ea6c0 con 0x5650287e9910 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.000+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5650287f84f0 con 0x5650287f58a0 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.000+0000 7fa8fbdf4740 10 client.0 _ll_register_callbacks cb 0x565028705490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41260/0 (socket says 172.21.15.90:41260) 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1e00640 1 -- 172.21.15.90:0/1430908258 learned_addr learned my addr 172.21.15.90:0/1430908258 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:09:06.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f2800640 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287f58a0 0x5650287f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1400640 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1e00640 1 -- 172.21.15.90:0/1430908258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287f58a0 msgr2=0x5650287f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:09:06.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287f58a0 0x5650287f3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1e00640 1 -- 172.21.15.90:0/1430908258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 msgr2=0x5650287f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:09:06.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1e00640 1 -- 172.21.15.90:0/1430908258 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650284adb00 con 0x5650287e8ff0 2024-09-10T22:09:06.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1400640 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:09:06.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f27d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009270 tx=0x7fa8e40a8700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538cf591c86a0e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:06.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.001+0000 7fa8d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:09:06.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8d3e00640 1 -- 172.21.15.90:0/1430908258 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e401f070 con 0x5650287e8ff0 2024-09-10T22:09:06.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1430908258 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e00044d0 con 0x5650287e8ff0 2024-09-10T22:09:06.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8d3e00640 1 -- 172.21.15.90:0/1430908258 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa8e40e6470 con 0x5650287e8ff0 2024-09-10T22:09:06.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8d3e00640 1 -- 172.21.15.90:0/1430908258 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa8e40e3f00 con 0x5650287e8ff0 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8fbdf4740 10 client.15954 fetch_fsmap learned FSMap version 35 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1430908258 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x565028524960 con 0x5650287e8ff0 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8d3e00640 1 -- 172.21.15.90:0/1430908258 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8e00044d0 con 0x5650287e8ff0 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8d3e00640 1 -- 172.21.15.90:0/1430908258 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fa8e40b58a0 con 0x5650287e8ff0 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8fbdf4740 10 client.15954 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8fbdf4740 10 client.15954 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8fbdf4740 20 client.15954 populate_metadata read hostname 'smithi090' 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5650288053f0 0x565028807880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8fbdf4740 4 client.15954 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.002+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1430908258 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650287621e0 con 0x5650288053f0 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.003+0000 7fa8f1400640 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5650288053f0 0x565028807880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.004+0000 7fa8f1400640 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5650288053f0 0x565028807880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdcc12b04a908865 server_cookie=66226c3d235831a4 in_seq=0 out_seq=0 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.004+0000 7fa8d3e00640 10 client.15954 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8d3e00640 1 -- 172.21.15.90:0/1430908258 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5650287621e0 con 0x5650288053f0 2024-09-10T22:09:06.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8d3e00640 10 client.15954 handle_command_reply: tid=0 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 client.15954 shutdown 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1430908258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 msgr2=0x5650287f27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f27d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009270 tx=0x7fa8e40a8700 comp rx=0 tx=0).stop 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1430908258 shutdown_connections 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5650288053f0 0x565028807880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287f58a0 0x5650287f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1430908258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f27d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1430908258 >> 172.21.15.90:0/1430908258 conn(0x5650287e6e20 msgr2=0x5650287ec9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8d3e00640 0 client.15954 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1430908258 shutdown_connections 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1430908258 wait complete. 2024-09-10T22:09:06.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.005+0000 7fa8fbdf4740 20 client.15954 trim_cache size 0 max 16384 2024-09-10T22:09:06.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.023+0000 7fa8fbdf4740 1 Processor -- start 2024-09-10T22:09:06.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.023+0000 7fa8fbdf4740 1 -- start start 2024-09-10T22:09:06.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.023+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e6e20 0x565028801220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.023+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.023+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.023+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41b1e30 con 0x5650287e8ff0 2024-09-10T22:09:06.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.023+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41b3970 con 0x5650287e9910 2024-09-10T22:09:06.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.023+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41b2500 con 0x5650287e6e20 2024-09-10T22:09:06.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41266/0 (socket says 172.21.15.90:41266) 2024-09-10T22:09:06.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1400640 1 -- 172.21.15.90:0/1062477295 learned_addr learned my addr 172.21.15.90:0/1062477295 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:09:06.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f2800640 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e6e20 0x565028801220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1400640 1 -- 172.21.15.90:0/1062477295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e6e20 msgr2=0x565028801220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1400640 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e6e20 0x565028801220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1400640 1 -- 172.21.15.90:0/1062477295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 msgr2=0x5650287ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1400640 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1400640 1 -- 172.21.15.90:0/1062477295 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565028803f80 con 0x5650287e8ff0 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f2800640 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.024+0000 7fa8f1400640 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f1a00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00057c0 tx=0x7fa8e0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e726200dd36feba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8dea00640 1 -- 172.21.15.90:0/1062477295 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e003f040 con 0x5650287e8ff0 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8dea00640 1 -- 172.21.15.90:0/1062477295 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8e00049c0 con 0x5650287e8ff0 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8dea00640 1 -- 172.21.15.90:0/1062477295 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0035040 con 0x5650287e8ff0 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1062477295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 msgr2=0x5650287f1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f1a00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa8e00057c0 tx=0x7fa8e0005b50 comp rx=0 tx=0).stop 2024-09-10T22:09:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1062477295 shutdown_connections 2024-09-10T22:09:06.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e9910 0x5650287ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e8ff0 0x5650287f1a00 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/1062477295 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e6e20 0x565028801220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1062477295 >> 172.21.15.90:0/1062477295 conn(0x5650287e8780 msgr2=0x5650287f3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:06.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1062477295 shutdown_connections 2024-09-10T22:09:06.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.025+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/1062477295 wait complete. 2024-09-10T22:09:06.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.026+0000 7fa8fbdf4740 1 -- start start 2024-09-10T22:09:06.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.026+0000 7fa8fbdf4740 1 Processor -- start 2024-09-10T22:09:06.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.026+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e6e20 0x5650287fdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.026+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287fa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.026+0000 7fa8fbdf4740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e9910 0x5650287facc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.026+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41b32a0 con 0x5650287e9910 2024-09-10T22:09:06.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.026+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41b2bd0 con 0x5650287e6e20 2024-09-10T22:09:06.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.026+0000 7fa8fbdf4740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f41afe10 con 0x5650287e8ff0 2024-09-10T22:09:06.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8fbdf4740 10 client.0 _ll_register_callbacks cb 0x7fa8e40a8df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:09:06.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287fa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e9910 0x5650287facc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287fa780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:39134/0 (socket says 172.21.15.90:39134) 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e9910 0x5650287facc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41282/0 (socket says 172.21.15.90:41282) 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f1400640 1 -- 172.21.15.90:0/75925910 learned_addr learned my addr 172.21.15.90:0/75925910 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e6e20 0x5650287fdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f2800640 1 -- 172.21.15.90:0/75925910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 msgr2=0x5650287fa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f2800640 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287fa780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f2800640 1 -- 172.21.15.90:0/75925910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e6e20 msgr2=0x5650287fdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f2800640 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e6e20 0x5650287fdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f2800640 1 -- 172.21.15.90:0/75925910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650287e78a0 con 0x5650287e9910 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.027+0000 7fa8f1400640 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287fa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.028+0000 7fa8f2800640 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e9910 0x5650287facc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec013cc0 tx=0x7fa8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f183fb3459c9c1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.028+0000 7fa8d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.028+0000 7fa8d3e00640 1 -- 172.21.15.90:0/75925910 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec02d060 con 0x5650287e9910 2024-09-10T22:09:06.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.028+0000 7fa8d3e00640 1 -- 172.21.15.90:0/75925910 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9402+0+0 (secure 0 0 0) 0x7fa8ec01c860 con 0x5650287e9910 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.028+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/75925910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e00049c0 con 0x5650287e9910 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.028+0000 7fa8d3e00640 1 -- 172.21.15.90:0/75925910 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa8ec034050 con 0x5650287e9910 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.028+0000 7fa8fbdf4740 10 client.15960 fetch_fsmap learned FSMap version 35 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.028+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/75925910 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56502887baa0 con 0x5650287e9910 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.028+0000 7fa8d3e00640 1 -- 172.21.15.90:0/75925910 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa8e00049c0 con 0x5650287e9910 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.029+0000 7fa8d3e00640 1 -- 172.21.15.90:0/75925910 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fa8ec027070 con 0x5650287e9910 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.029+0000 7fa8fbdf4740 10 client.15960 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.029+0000 7fa8fbdf4740 10 client.15960 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.029+0000 7fa8fbdf4740 20 client.15960 populate_metadata read hostname 'smithi090' 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.029+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5650287f58a0 0x56502887e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.029+0000 7fa8fbdf4740 4 client.15960 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "df61d4a4", "release": true, "await": true}] 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.029+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/75925910 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "df61d4a4", "release": true, "await": true}) -- 0x56502875dc90 con 0x5650287f58a0 2024-09-10T22:09:06.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.029+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5650287f58a0 0x56502887e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:09:06.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.030+0000 7fa8f1e00640 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5650287f58a0 0x56502887e290 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d30332bc21e64747 server_cookie=e4221af2e5a99dbd in_seq=0 out_seq=0 2024-09-10T22:09:06.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.030+0000 7fa8d3e00640 10 client.15960 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:09:06.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.038+0000 7fa8d3e00640 1 -- 172.21.15.90:0/75925910 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56502875dc90 con 0x5650287f58a0 2024-09-10T22:09:06.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.038+0000 7fa8d3e00640 10 client.15960 handle_command_reply: tid=0 2024-09-10T22:09:06.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.038+0000 7fa8fbdf4740 1 client.15960 shutdown 2024-09-10T22:09:06.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/75925910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e9910 msgr2=0x5650287facc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:06.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e9910 0x5650287facc0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec013cc0 tx=0x7fa8ec0046c0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/75925910 shutdown_connections 2024-09-10T22:09:06.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5650287f58a0 0x56502887e290 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5650287e9910 0x5650287facc0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5650287e8ff0 0x5650287fa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8d3e00640 0 client.15960 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:09:06.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 --2- 172.21.15.90:0/75925910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5650287e6e20 0x5650287fdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/75925910 >> 172.21.15.90:0/75925910 conn(0x5650287e8780 msgr2=0x5650287f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:06.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/75925910 shutdown_connections 2024-09-10T22:09:06.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 1 -- 172.21.15.90:0/75925910 wait complete. 2024-09-10T22:09:06.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.039+0000 7fa8fbdf4740 20 client.15960 trim_cache size 0 max 16384 2024-09-10T22:09:06.041 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:09:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 35, 2024-09-10T22:09:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 20, 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "df61d4a4": { 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 20, 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:09:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:09:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:09:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:09:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:09:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:09:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:09:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:09:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:09:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:09:06.043 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 -- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8d80b5450 msgr2=0x7fa8d80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8d80b5450 0x7fa8d80b7910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa8f406c0b0 tx=0x7fa8ec005f90 comp rx=0 tx=0).stop 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 -- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075540 msgr2=0x7fa8f419b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075540 0x7fa8f419b3e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400ea80 tx=0x7fa8e400ef40 comp rx=0 tx=0).stop 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 -- 172.21.15.90:0/3170947771 shutdown_connections 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa8d80b5450 0x7fa8d80b7910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa8f410f3e0 0x7fa8f41a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa8f4075f40 0x7fa8f419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 --2- 172.21.15.90:0/3170947771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8f4075540 0x7fa8f419b3e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 -- 172.21.15.90:0/3170947771 >> 172.21.15.90:0/3170947771 conn(0x7fa8f40fe830 msgr2=0x7fa8f40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.040+0000 7fa8f9200640 1 -- 172.21.15.90:0/3170947771 shutdown_connections 2024-09-10T22:09:06.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:09:06.041+0000 7fa8f9200640 1 -- 172.21.15.90:0/3170947771 wait complete. 2024-09-10T22:09:06.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49681 0.95 MB/sec execute 118 sec latency 30300.109 ms 2024-09-10T22:09:06.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:06 smithi090 ceph-mon[30350]: pgmap v574: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-10T22:09:07.159 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'df61d4a4', total seconds elapsed: 34.8 2024-09-10T22:09:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:06 smithi154 ceph-mon[34495]: pgmap v574: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-10T22:09:07.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:06 smithi171 ceph-mon[34138]: pgmap v574: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-09-10T22:09:07.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 49966 0.95 MB/sec execute 119 sec latency 19.123 ms 2024-09-10T22:09:08.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 50146 0.94 MB/sec execute 120 sec latency 20.611 ms 2024-09-10T22:09:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:08 smithi090 ceph-mon[30350]: pgmap v575: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-10T22:09:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:08 smithi154 ceph-mon[34495]: pgmap v575: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-10T22:09:09.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:08 smithi171 ceph-mon[34138]: pgmap v575: 129 pgs: 129 active+clean; 58 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-09-10T22:09:09.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 50382 0.94 MB/sec execute 121 sec latency 17.747 ms 2024-09-10T22:09:10.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 50747 0.94 MB/sec execute 122 sec latency 23.370 ms 2024-09-10T22:09:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:10 smithi090 ceph-mon[30350]: pgmap v576: 129 pgs: 129 active+clean; 60 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 186 KiB/s wr, 40 op/s 2024-09-10T22:09:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:10 smithi154 ceph-mon[34495]: pgmap v576: 129 pgs: 129 active+clean; 60 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 186 KiB/s wr, 40 op/s 2024-09-10T22:09:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:10 smithi171 ceph-mon[34138]: pgmap v576: 129 pgs: 129 active+clean; 60 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 186 KiB/s wr, 40 op/s 2024-09-10T22:09:11.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 51142 0.96 MB/sec execute 123 sec latency 20.448 ms 2024-09-10T22:09:12.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 51536 0.96 MB/sec execute 124 sec latency 20.725 ms 2024-09-10T22:09:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:12 smithi090 ceph-mon[30350]: pgmap v577: 129 pgs: 129 active+clean; 61 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 287 KiB/s wr, 54 op/s 2024-09-10T22:09:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:12 smithi154 ceph-mon[34495]: pgmap v577: 129 pgs: 129 active+clean; 61 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 287 KiB/s wr, 54 op/s 2024-09-10T22:09:13.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:12 smithi171 ceph-mon[34138]: pgmap v577: 129 pgs: 129 active+clean; 61 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 287 KiB/s wr, 54 op/s 2024-09-10T22:09:13.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 51985 0.96 MB/sec execute 125 sec latency 20.247 ms 2024-09-10T22:09:14.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 52271 0.96 MB/sec execute 126 sec latency 21.443 ms 2024-09-10T22:09:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:14 smithi090 ceph-mon[30350]: pgmap v578: 129 pgs: 129 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 398 KiB/s wr, 71 op/s 2024-09-10T22:09:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:14 smithi154 ceph-mon[34495]: pgmap v578: 129 pgs: 129 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 398 KiB/s wr, 71 op/s 2024-09-10T22:09:15.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:14 smithi171 ceph-mon[34138]: pgmap v578: 129 pgs: 129 active+clean; 61 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 398 KiB/s wr, 71 op/s 2024-09-10T22:09:15.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 52863 1.00 MB/sec execute 127 sec latency 21.290 ms 2024-09-10T22:09:16.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 53065 1.00 MB/sec execute 128 sec latency 22.019 ms 2024-09-10T22:09:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:16 smithi090 ceph-mon[30350]: pgmap v579: 129 pgs: 129 active+clean; 62 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 570 KiB/s wr, 81 op/s 2024-09-10T22:09:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:16 smithi154 ceph-mon[34495]: pgmap v579: 129 pgs: 129 active+clean; 62 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 570 KiB/s wr, 81 op/s 2024-09-10T22:09:17.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:16 smithi171 ceph-mon[34138]: pgmap v579: 129 pgs: 129 active+clean; 62 MiB data, 617 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 570 KiB/s wr, 81 op/s 2024-09-10T22:09:17.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 53469 1.02 MB/sec execute 129 sec latency 20.371 ms 2024-09-10T22:09:18.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 53646 1.02 MB/sec execute 130 sec latency 16.914 ms 2024-09-10T22:09:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:18 smithi090 ceph-mon[30350]: pgmap v580: 129 pgs: 129 active+clean; 63 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 766 KiB/s wr, 80 op/s 2024-09-10T22:09:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:18 smithi154 ceph-mon[34495]: pgmap v580: 129 pgs: 129 active+clean; 63 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 766 KiB/s wr, 80 op/s 2024-09-10T22:09:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:18 smithi171 ceph-mon[34138]: pgmap v580: 129 pgs: 129 active+clean; 63 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 766 KiB/s wr, 80 op/s 2024-09-10T22:09:19.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 53844 1.01 MB/sec execute 131 sec latency 26.335 ms 2024-09-10T22:09:20.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 54178 1.01 MB/sec execute 132 sec latency 19.848 ms 2024-09-10T22:09:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:20 smithi090 ceph-mon[30350]: pgmap v581: 129 pgs: 129 active+clean; 65 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 997 KiB/s wr, 105 op/s 2024-09-10T22:09:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:20 smithi154 ceph-mon[34495]: pgmap v581: 129 pgs: 129 active+clean; 65 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 997 KiB/s wr, 105 op/s 2024-09-10T22:09:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:20 smithi171 ceph-mon[34138]: pgmap v581: 129 pgs: 129 active+clean; 65 MiB data, 637 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 997 KiB/s wr, 105 op/s 2024-09-10T22:09:21.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 54521 1.01 MB/sec execute 133 sec latency 20.315 ms 2024-09-10T22:09:22.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 54883 1.03 MB/sec execute 134 sec latency 29.009 ms 2024-09-10T22:09:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:22 smithi090 ceph-mon[30350]: pgmap v582: 129 pgs: 129 active+clean; 66 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 986 KiB/s wr, 110 op/s 2024-09-10T22:09:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:22 smithi154 ceph-mon[34495]: pgmap v582: 129 pgs: 129 active+clean; 66 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 986 KiB/s wr, 110 op/s 2024-09-10T22:09:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:22 smithi171 ceph-mon[34138]: pgmap v582: 129 pgs: 129 active+clean; 66 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 986 KiB/s wr, 110 op/s 2024-09-10T22:09:23.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 55274 1.02 MB/sec execute 135 sec latency 25.009 ms 2024-09-10T22:09:24.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 55633 1.03 MB/sec execute 136 sec latency 27.671 ms 2024-09-10T22:09:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:24 smithi090 ceph-mon[30350]: pgmap v583: 129 pgs: 129 active+clean; 64 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.0 MiB/s wr, 111 op/s 2024-09-10T22:09:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:24 smithi154 ceph-mon[34495]: pgmap v583: 129 pgs: 129 active+clean; 64 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.0 MiB/s wr, 111 op/s 2024-09-10T22:09:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:24 smithi171 ceph-mon[34138]: pgmap v583: 129 pgs: 129 active+clean; 64 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.0 MiB/s wr, 111 op/s 2024-09-10T22:09:25.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 56028 1.05 MB/sec execute 137 sec latency 20.388 ms 2024-09-10T22:09:26.468 INFO:tasks.workunit.client.0.smithi090.stdout: 1 56533 1.06 MB/sec execute 138 sec latency 20.983 ms 2024-09-10T22:09:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:26 smithi090 ceph-mon[30350]: pgmap v584: 129 pgs: 129 active+clean; 65 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T22:09:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:26 smithi154 ceph-mon[34495]: pgmap v584: 129 pgs: 129 active+clean; 65 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T22:09:27.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:26 smithi171 ceph-mon[34138]: pgmap v584: 129 pgs: 129 active+clean; 65 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T22:09:27.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 56851 1.08 MB/sec execute 139 sec latency 16.907 ms 2024-09-10T22:09:28.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 57085 1.08 MB/sec execute 140 sec latency 19.094 ms 2024-09-10T22:09:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:28 smithi090 ceph-mon[30350]: pgmap v585: 129 pgs: 129 active+clean; 65 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-10T22:09:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:28 smithi154 ceph-mon[34495]: pgmap v585: 129 pgs: 129 active+clean; 65 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-10T22:09:29.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:28 smithi171 ceph-mon[34138]: pgmap v585: 129 pgs: 129 active+clean; 65 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-10T22:09:29.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 57255 1.07 MB/sec execute 141 sec latency 18.041 ms 2024-09-10T22:09:30.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 57510 1.07 MB/sec execute 142 sec latency 18.347 ms 2024-09-10T22:09:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:30 smithi090 ceph-mon[30350]: pgmap v586: 129 pgs: 129 active+clean; 67 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T22:09:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:30 smithi154 ceph-mon[34495]: pgmap v586: 129 pgs: 129 active+clean; 67 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T22:09:31.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:30 smithi171 ceph-mon[34138]: pgmap v586: 129 pgs: 129 active+clean; 67 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T22:09:31.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 57871 1.07 MB/sec execute 143 sec latency 18.544 ms 2024-09-10T22:09:32.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 58233 1.09 MB/sec execute 144 sec latency 22.732 ms 2024-09-10T22:09:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:32 smithi090 ceph-mon[30350]: pgmap v587: 129 pgs: 129 active+clean; 69 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 991 KiB/s wr, 110 op/s 2024-09-10T22:09:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:32 smithi154 ceph-mon[34495]: pgmap v587: 129 pgs: 129 active+clean; 69 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 991 KiB/s wr, 110 op/s 2024-09-10T22:09:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:32 smithi171 ceph-mon[34138]: pgmap v587: 129 pgs: 129 active+clean; 69 MiB data, 684 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 991 KiB/s wr, 110 op/s 2024-09-10T22:09:33.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 58581 1.08 MB/sec execute 145 sec latency 26.026 ms 2024-09-10T22:09:34.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 58965 1.08 MB/sec execute 146 sec latency 24.022 ms 2024-09-10T22:09:35.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:34 smithi090 ceph-mon[30350]: pgmap v588: 129 pgs: 129 active+clean; 67 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1019 KiB/s wr, 103 op/s 2024-09-10T22:09:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:34 smithi154 ceph-mon[34495]: pgmap v588: 129 pgs: 129 active+clean; 67 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1019 KiB/s wr, 103 op/s 2024-09-10T22:09:35.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:34 smithi171 ceph-mon[34138]: pgmap v588: 129 pgs: 129 active+clean; 67 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 288 KiB/s rd, 1019 KiB/s wr, 103 op/s 2024-09-10T22:09:35.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 59279 1.08 MB/sec execute 147 sec latency 18.699 ms 2024-09-10T22:09:36.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 59783 1.11 MB/sec execute 148 sec latency 18.318 ms 2024-09-10T22:09:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:36 smithi090 ceph-mon[30350]: pgmap v589: 129 pgs: 129 active+clean; 68 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-09-10T22:09:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:36 smithi154 ceph-mon[34495]: pgmap v589: 129 pgs: 129 active+clean; 68 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-09-10T22:09:37.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:36 smithi171 ceph-mon[34138]: pgmap v589: 129 pgs: 129 active+clean; 68 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-09-10T22:09:37.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 60112 1.11 MB/sec execute 149 sec latency 22.121 ms 2024-09-10T22:09:38.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 60529 1.14 MB/sec execute 150 sec latency 15.708 ms 2024-09-10T22:09:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:38 smithi090 ceph-mon[30350]: pgmap v590: 129 pgs: 129 active+clean; 68 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 986 KiB/s wr, 89 op/s 2024-09-10T22:09:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:38 smithi154 ceph-mon[34495]: pgmap v590: 129 pgs: 129 active+clean; 68 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 986 KiB/s wr, 89 op/s 2024-09-10T22:09:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:38 smithi171 ceph-mon[34138]: pgmap v590: 129 pgs: 129 active+clean; 68 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 986 KiB/s wr, 89 op/s 2024-09-10T22:09:39.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 60704 1.13 MB/sec execute 151 sec latency 17.307 ms 2024-09-10T22:09:40.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 60885 1.12 MB/sec execute 152 sec latency 19.728 ms 2024-09-10T22:09:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:40 smithi090 ceph-mon[30350]: pgmap v591: 129 pgs: 129 active+clean; 70 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-09-10T22:09:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:40 smithi154 ceph-mon[34495]: pgmap v591: 129 pgs: 129 active+clean; 70 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-09-10T22:09:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:40 smithi171 ceph-mon[34138]: pgmap v591: 129 pgs: 129 active+clean; 70 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-09-10T22:09:41.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 61200 1.12 MB/sec execute 153 sec latency 25.015 ms 2024-09-10T22:09:42.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 61539 1.12 MB/sec execute 154 sec latency 19.184 ms 2024-09-10T22:09:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:42 smithi090 ceph-mon[30350]: pgmap v592: 129 pgs: 129 active+clean; 72 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-10T22:09:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:42 smithi154 ceph-mon[34495]: pgmap v592: 129 pgs: 129 active+clean; 72 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-10T22:09:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:42 smithi171 ceph-mon[34138]: pgmap v592: 129 pgs: 129 active+clean; 72 MiB data, 744 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.0 MiB/s wr, 105 op/s 2024-09-10T22:09:43.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 61917 1.13 MB/sec execute 155 sec latency 27.033 ms 2024-09-10T22:09:44.469 INFO:tasks.workunit.client.0.smithi090.stdout: 1 62297 1.13 MB/sec execute 156 sec latency 22.870 ms 2024-09-10T22:09:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:44 smithi090 ceph-mon[30350]: pgmap v593: 129 pgs: 129 active+clean; 72 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-09-10T22:09:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:44 smithi154 ceph-mon[34495]: pgmap v593: 129 pgs: 129 active+clean; 72 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-09-10T22:09:45.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:44 smithi171 ceph-mon[34138]: pgmap v593: 129 pgs: 129 active+clean; 72 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-09-10T22:09:45.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 62693 1.13 MB/sec execute 157 sec latency 24.921 ms 2024-09-10T22:09:46.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 62950 1.13 MB/sec execute 158 sec latency 19.793 ms 2024-09-10T22:09:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:46 smithi154 ceph-mon[34495]: pgmap v594: 129 pgs: 129 active+clean; 72 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T22:09:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:46 smithi171 ceph-mon[34138]: pgmap v594: 129 pgs: 129 active+clean; 72 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T22:09:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:46 smithi090 ceph-mon[30350]: pgmap v594: 129 pgs: 129 active+clean; 72 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-09-10T22:09:47.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 63605 1.16 MB/sec execute 159 sec latency 19.971 ms 2024-09-10T22:09:48.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:47 smithi154 ceph-mon[34495]: pgmap v595: 129 pgs: 129 active+clean; 73 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-10T22:09:48.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:47 smithi171 ceph-mon[34138]: pgmap v595: 129 pgs: 129 active+clean; 73 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-10T22:09:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:47 smithi090 ceph-mon[30350]: pgmap v595: 129 pgs: 129 active+clean; 73 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.0 MiB/s wr, 92 op/s 2024-09-10T22:09:48.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 63837 1.18 MB/sec execute 160 sec latency 19.821 ms 2024-09-10T22:09:49.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 64126 1.18 MB/sec execute 161 sec latency 18.325 ms 2024-09-10T22:09:50.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 64293 1.17 MB/sec execute 162 sec latency 73.210 ms 2024-09-10T22:09:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:50 smithi090 ceph-mon[30350]: pgmap v596: 129 pgs: 129 active+clean; 74 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-10T22:09:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:50 smithi154 ceph-mon[34495]: pgmap v596: 129 pgs: 129 active+clean; 74 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-10T22:09:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:50 smithi171 ceph-mon[34138]: pgmap v596: 129 pgs: 129 active+clean; 74 MiB data, 781 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-10T22:09:51.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 64490 1.16 MB/sec execute 163 sec latency 23.610 ms 2024-09-10T22:09:52.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 64834 1.16 MB/sec execute 164 sec latency 42.785 ms 2024-09-10T22:09:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:52 smithi090 ceph-mon[30350]: pgmap v597: 129 pgs: 129 active+clean; 75 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 991 KiB/s wr, 108 op/s 2024-09-10T22:09:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:09:53.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:52 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:52 smithi154 ceph-mon[34495]: pgmap v597: 129 pgs: 129 active+clean; 75 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 991 KiB/s wr, 108 op/s 2024-09-10T22:09:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:09:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:52 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:52 smithi171 ceph-mon[34138]: pgmap v597: 129 pgs: 129 active+clean; 75 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 991 KiB/s wr, 108 op/s 2024-09-10T22:09:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:09:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:52 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:53.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 65252 1.18 MB/sec execute 165 sec latency 20.819 ms 2024-09-10T22:09:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi171 to 176.4M 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: Unable to set osd_memory_target on smithi171 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi154 to 176.4M 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: Unable to set osd_memory_target on smithi154 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:09:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:09:54.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi171 to 176.4M 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: Unable to set osd_memory_target on smithi171 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi154 to 176.4M 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: Unable to set osd_memory_target on smithi154 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:09:54.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:09:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi171 to 176.4M 2024-09-10T22:09:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: Unable to set osd_memory_target on smithi171 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:09:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:09:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi154 to 176.4M 2024-09-10T22:09:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: Unable to set osd_memory_target on smithi154 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:09:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:09:54.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:09:54.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:09:54.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 65544 1.18 MB/sec execute 166 sec latency 26.042 ms 2024-09-10T22:09:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:54 smithi154 ceph-mon[34495]: pgmap v598: 129 pgs: 129 active+clean; 75 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 985 KiB/s wr, 104 op/s 2024-09-10T22:09:55.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:54 smithi171 ceph-mon[34138]: pgmap v598: 129 pgs: 129 active+clean; 75 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 985 KiB/s wr, 104 op/s 2024-09-10T22:09:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:54 smithi090 ceph-mon[30350]: pgmap v598: 129 pgs: 129 active+clean; 75 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 985 KiB/s wr, 104 op/s 2024-09-10T22:09:55.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 65952 1.17 MB/sec execute 167 sec latency 23.337 ms 2024-09-10T22:09:56.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:55 smithi154 ceph-mon[34495]: pgmap v599: 129 pgs: 129 active+clean; 76 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-09-10T22:09:56.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:55 smithi171 ceph-mon[34138]: pgmap v599: 129 pgs: 129 active+clean; 76 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-09-10T22:09:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:55 smithi090 ceph-mon[30350]: pgmap v599: 129 pgs: 129 active+clean; 76 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-09-10T22:09:56.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 66303 1.17 MB/sec execute 168 sec latency 23.905 ms 2024-09-10T22:09:57.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 66785 1.20 MB/sec execute 169 sec latency 19.096 ms 2024-09-10T22:09:58.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 67182 1.20 MB/sec execute 170 sec latency 19.677 ms 2024-09-10T22:09:59.058 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:09:58 smithi090 ceph-mon[30350]: pgmap v600: 129 pgs: 129 active+clean; 76 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1012 KiB/s wr, 94 op/s 2024-09-10T22:09:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:09:58 smithi154 ceph-mon[34495]: pgmap v600: 129 pgs: 129 active+clean; 76 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1012 KiB/s wr, 94 op/s 2024-09-10T22:09:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:09:58 smithi171 ceph-mon[34138]: pgmap v600: 129 pgs: 129 active+clean; 76 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1012 KiB/s wr, 94 op/s 2024-09-10T22:09:59.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 67525 1.21 MB/sec execute 171 sec latency 17.595 ms 2024-09-10T22:10:00.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 67746 1.21 MB/sec execute 172 sec latency 18.395 ms 2024-09-10T22:10:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:00 smithi090 ceph-mon[30350]: pgmap v601: 129 pgs: 129 active+clean; 77 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-09-10T22:10:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:00 smithi090 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T22:10:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:00 smithi154 ceph-mon[34495]: pgmap v601: 129 pgs: 129 active+clean; 77 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-09-10T22:10:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:00 smithi154 ceph-mon[34495]: overall HEALTH_OK 2024-09-10T22:10:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:00 smithi171 ceph-mon[34138]: pgmap v601: 129 pgs: 129 active+clean; 77 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-09-10T22:10:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:00 smithi171 ceph-mon[34138]: overall HEALTH_OK 2024-09-10T22:10:01.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 67928 1.21 MB/sec execute 173 sec latency 21.754 ms 2024-09-10T22:10:02.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 68233 1.20 MB/sec execute 174 sec latency 20.589 ms 2024-09-10T22:10:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:02 smithi090 ceph-mon[30350]: pgmap v602: 129 pgs: 129 active+clean; 78 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 953 KiB/s wr, 107 op/s 2024-09-10T22:10:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:02 smithi154 ceph-mon[34495]: pgmap v602: 129 pgs: 129 active+clean; 78 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 953 KiB/s wr, 107 op/s 2024-09-10T22:10:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:02 smithi171 ceph-mon[34138]: pgmap v602: 129 pgs: 129 active+clean; 78 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 953 KiB/s wr, 107 op/s 2024-09-10T22:10:03.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 68588 1.21 MB/sec execute 175 sec latency 23.418 ms 2024-09-10T22:10:04.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 68939 1.22 MB/sec execute 176 sec latency 26.898 ms 2024-09-10T22:10:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:04 smithi090 ceph-mon[30350]: pgmap v603: 129 pgs: 129 active+clean; 78 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-09-10T22:10:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:04 smithi090 ceph-mon[30350]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T22:10:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:04 smithi154 ceph-mon[34495]: pgmap v603: 129 pgs: 129 active+clean; 78 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-09-10T22:10:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:04 smithi154 ceph-mon[34495]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T22:10:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:04 smithi171 ceph-mon[34138]: pgmap v603: 129 pgs: 129 active+clean; 78 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-09-10T22:10:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:04 smithi171 ceph-mon[34138]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T22:10:05.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 69293 1.21 MB/sec execute 177 sec latency 22.622 ms 2024-09-10T22:10:06.470 INFO:tasks.workunit.client.0.smithi090.stdout: 1 69728 1.21 MB/sec execute 178 sec latency 27.185 ms 2024-09-10T22:10:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:06 smithi090 ceph-mon[30350]: pgmap v605: 129 pgs: 129 active+clean; 80 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T22:10:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:06 smithi154 ceph-mon[34495]: pgmap v605: 129 pgs: 129 active+clean; 80 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T22:10:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:06 smithi171 ceph-mon[34138]: pgmap v605: 129 pgs: 129 active+clean; 80 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T22:10:07.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 70010 1.21 MB/sec execute 179 sec latency 21.649 ms 2024-09-10T22:10:08.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 70606 1.23 MB/sec execute 180 sec latency 20.867 ms 2024-09-10T22:10:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:08 smithi090 ceph-mon[30350]: pgmap v606: 129 pgs: 129 active+clean; 81 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T22:10:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:08 smithi154 ceph-mon[34495]: pgmap v606: 129 pgs: 129 active+clean; 81 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T22:10:09.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:08 smithi171 ceph-mon[34138]: pgmap v606: 129 pgs: 129 active+clean; 81 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-09-10T22:10:09.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 70809 1.23 MB/sec execute 181 sec latency 18.069 ms 2024-09-10T22:10:10.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 71206 1.25 MB/sec execute 182 sec latency 16.786 ms 2024-09-10T22:10:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:10 smithi090 ceph-mon[30350]: pgmap v607: 129 pgs: 129 active+clean; 83 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-09-10T22:10:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:10 smithi090 ceph-mon[30350]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T22:10:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:10 smithi154 ceph-mon[34495]: pgmap v607: 129 pgs: 129 active+clean; 83 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-09-10T22:10:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:10 smithi154 ceph-mon[34495]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T22:10:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:10 smithi171 ceph-mon[34138]: pgmap v607: 129 pgs: 129 active+clean; 83 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-09-10T22:10:11.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:10 smithi171 ceph-mon[34138]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T22:10:11.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 71379 1.25 MB/sec execute 183 sec latency 19.659 ms 2024-09-10T22:10:12.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 71576 1.24 MB/sec execute 184 sec latency 24.786 ms 2024-09-10T22:10:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:12 smithi090 ceph-mon[30350]: pgmap v609: 129 pgs: 129 active+clean; 84 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-09-10T22:10:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:12 smithi154 ceph-mon[34495]: pgmap v609: 129 pgs: 129 active+clean; 84 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-09-10T22:10:13.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:12 smithi171 ceph-mon[34138]: pgmap v609: 129 pgs: 129 active+clean; 84 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.1 MiB/s wr, 119 op/s 2024-09-10T22:10:13.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 71936 1.24 MB/sec execute 185 sec latency 22.136 ms 2024-09-10T22:10:14.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 72348 1.25 MB/sec execute 186 sec latency 19.645 ms 2024-09-10T22:10:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:14 smithi090 ceph-mon[30350]: pgmap v610: 129 pgs: 129 active+clean; 85 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-10T22:10:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:14 smithi154 ceph-mon[34495]: pgmap v610: 129 pgs: 129 active+clean; 85 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-10T22:10:15.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:14 smithi171 ceph-mon[34138]: pgmap v610: 129 pgs: 129 active+clean; 85 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-10T22:10:15.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 72655 1.25 MB/sec execute 187 sec latency 24.816 ms 2024-09-10T22:10:16.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 73070 1.24 MB/sec execute 188 sec latency 25.383 ms 2024-09-10T22:10:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:16 smithi090 ceph-mon[30350]: pgmap v611: 129 pgs: 129 active+clean; 87 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-09-10T22:10:16.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:16 smithi154 ceph-mon[34495]: pgmap v611: 129 pgs: 129 active+clean; 87 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-09-10T22:10:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:16 smithi171 ceph-mon[34138]: pgmap v611: 129 pgs: 129 active+clean; 87 MiB data, 847 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-09-10T22:10:17.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 73421 1.24 MB/sec execute 189 sec latency 24.844 ms 2024-09-10T22:10:18.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 73893 1.26 MB/sec execute 190 sec latency 18.854 ms 2024-09-10T22:10:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:18 smithi090 ceph-mon[30350]: pgmap v612: 129 pgs: 129 active+clean; 87 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-09-10T22:10:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:18 smithi154 ceph-mon[34495]: pgmap v612: 129 pgs: 129 active+clean; 87 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-09-10T22:10:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:18 smithi171 ceph-mon[34138]: pgmap v612: 129 pgs: 129 active+clean; 87 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.1 MiB/s wr, 108 op/s 2024-09-10T22:10:19.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 74303 1.27 MB/sec execute 191 sec latency 20.205 ms 2024-09-10T22:10:20.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 74627 1.28 MB/sec execute 192 sec latency 19.570 ms 2024-09-10T22:10:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:20 smithi090 ceph-mon[30350]: pgmap v613: 129 pgs: 129 active+clean; 88 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-09-10T22:10:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:20 smithi154 ceph-mon[34495]: pgmap v613: 129 pgs: 129 active+clean; 88 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-09-10T22:10:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:20 smithi171 ceph-mon[34138]: pgmap v613: 129 pgs: 129 active+clean; 88 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.2 MiB/s wr, 114 op/s 2024-09-10T22:10:21.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 74860 1.28 MB/sec execute 193 sec latency 17.174 ms 2024-09-10T22:10:22.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 75031 1.27 MB/sec execute 194 sec latency 34.117 ms 2024-09-10T22:10:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:22 smithi090 ceph-mon[30350]: pgmap v614: 129 pgs: 129 active+clean; 88 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-09-10T22:10:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:22 smithi154 ceph-mon[34495]: pgmap v614: 129 pgs: 129 active+clean; 88 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-09-10T22:10:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:22 smithi171 ceph-mon[34138]: pgmap v614: 129 pgs: 129 active+clean; 88 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.0 MiB/s wr, 102 op/s 2024-09-10T22:10:23.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 75329 1.27 MB/sec execute 195 sec latency 21.915 ms 2024-09-10T22:10:24.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 75674 1.27 MB/sec execute 196 sec latency 21.158 ms 2024-09-10T22:10:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:24 smithi090 ceph-mon[30350]: pgmap v615: 129 pgs: 129 active+clean; 89 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-10T22:10:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:24 smithi154 ceph-mon[34495]: pgmap v615: 129 pgs: 129 active+clean; 89 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-10T22:10:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:24 smithi171 ceph-mon[34138]: pgmap v615: 129 pgs: 129 active+clean; 89 MiB data, 861 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-10T22:10:25.471 INFO:tasks.workunit.client.0.smithi090.stdout: 1 76049 1.28 MB/sec execute 197 sec latency 21.075 ms 2024-09-10T22:10:26.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 76404 1.28 MB/sec execute 198 sec latency 24.166 ms 2024-09-10T22:10:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:26 smithi090 ceph-mon[30350]: pgmap v616: 129 pgs: 129 active+clean; 91 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-10T22:10:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:26 smithi154 ceph-mon[34495]: pgmap v616: 129 pgs: 129 active+clean; 91 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-10T22:10:27.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:26 smithi171 ceph-mon[34138]: pgmap v616: 129 pgs: 129 active+clean; 91 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.2 MiB/s wr, 125 op/s 2024-09-10T22:10:27.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 76686 1.27 MB/sec execute 199 sec latency 304.126 ms 2024-09-10T22:10:28.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 76985 1.27 MB/sec execute 200 sec latency 408.831 ms 2024-09-10T22:10:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:28 smithi090 ceph-mon[30350]: pgmap v617: 129 pgs: 129 active+clean; 90 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 985 KiB/s wr, 91 op/s 2024-09-10T22:10:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:28 smithi154 ceph-mon[34495]: pgmap v617: 129 pgs: 129 active+clean; 90 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 985 KiB/s wr, 91 op/s 2024-09-10T22:10:29.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:28 smithi171 ceph-mon[34138]: pgmap v617: 129 pgs: 129 active+clean; 90 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 985 KiB/s wr, 91 op/s 2024-09-10T22:10:29.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 77476 1.29 MB/sec execute 201 sec latency 18.603 ms 2024-09-10T22:10:30.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 77859 1.30 MB/sec execute 202 sec latency 20.179 ms 2024-09-10T22:10:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:30 smithi090 ceph-mon[30350]: pgmap v618: 129 pgs: 129 active+clean; 90 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-09-10T22:10:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:30 smithi154 ceph-mon[34495]: pgmap v618: 129 pgs: 129 active+clean; 90 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-09-10T22:10:31.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:30 smithi171 ceph-mon[34138]: pgmap v618: 129 pgs: 129 active+clean; 90 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-09-10T22:10:31.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 78205 1.31 MB/sec execute 203 sec latency 16.499 ms 2024-09-10T22:10:32.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 78432 1.31 MB/sec execute 204 sec latency 15.452 ms 2024-09-10T22:10:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:32 smithi090 ceph-mon[30350]: pgmap v619: 129 pgs: 129 active+clean; 90 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 856 KiB/s wr, 92 op/s 2024-09-10T22:10:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:32 smithi154 ceph-mon[34495]: pgmap v619: 129 pgs: 129 active+clean; 90 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 856 KiB/s wr, 92 op/s 2024-09-10T22:10:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:32 smithi171 ceph-mon[34138]: pgmap v619: 129 pgs: 129 active+clean; 90 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 856 KiB/s wr, 92 op/s 2024-09-10T22:10:33.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 78625 1.30 MB/sec execute 205 sec latency 17.445 ms 2024-09-10T22:10:34.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 78945 1.30 MB/sec execute 206 sec latency 20.046 ms 2024-09-10T22:10:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:34 smithi090 ceph-mon[30350]: pgmap v620: 129 pgs: 129 active+clean; 92 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-09-10T22:10:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:34 smithi154 ceph-mon[34495]: pgmap v620: 129 pgs: 129 active+clean; 92 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-09-10T22:10:35.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:34 smithi171 ceph-mon[34138]: pgmap v620: 129 pgs: 129 active+clean; 92 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.1 MiB/s wr, 96 op/s 2024-09-10T22:10:35.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 79270 1.30 MB/sec execute 207 sec latency 19.775 ms 2024-09-10T22:10:36.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 79646 1.31 MB/sec execute 208 sec latency 25.027 ms 2024-09-10T22:10:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:36 smithi090 ceph-mon[30350]: pgmap v621: 129 pgs: 129 active+clean; 94 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-10T22:10:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:36 smithi154 ceph-mon[34495]: pgmap v621: 129 pgs: 129 active+clean; 94 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-10T22:10:37.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:36 smithi171 ceph-mon[34138]: pgmap v621: 129 pgs: 129 active+clean; 94 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-10T22:10:37.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 80005 1.30 MB/sec execute 209 sec latency 27.876 ms 2024-09-10T22:10:38.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 80425 1.30 MB/sec execute 210 sec latency 25.385 ms 2024-09-10T22:10:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:38 smithi090 ceph-mon[30350]: pgmap v622: 129 pgs: 129 active+clean; 93 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 930 KiB/s wr, 92 op/s 2024-09-10T22:10:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:38 smithi154 ceph-mon[34495]: pgmap v622: 129 pgs: 129 active+clean; 93 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 930 KiB/s wr, 92 op/s 2024-09-10T22:10:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:38 smithi171 ceph-mon[34138]: pgmap v622: 129 pgs: 129 active+clean; 93 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 930 KiB/s wr, 92 op/s 2024-09-10T22:10:39.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 80703 1.30 MB/sec execute 211 sec latency 20.282 ms 2024-09-10T22:10:40.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 81351 1.32 MB/sec execute 212 sec latency 20.702 ms 2024-09-10T22:10:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:40 smithi090 ceph-mon[30350]: pgmap v623: 129 pgs: 129 active+clean; 93 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-10T22:10:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:40 smithi154 ceph-mon[34495]: pgmap v623: 129 pgs: 129 active+clean; 93 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-10T22:10:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:40 smithi171 ceph-mon[34138]: pgmap v623: 129 pgs: 129 active+clean; 93 MiB data, 929 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-10T22:10:41.472 INFO:tasks.workunit.client.0.smithi090.stdout: 1 81595 1.33 MB/sec execute 213 sec latency 18.693 ms 2024-09-10T22:10:42.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 81889 1.33 MB/sec execute 214 sec latency 19.484 ms 2024-09-10T22:10:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:42 smithi090 ceph-mon[30350]: pgmap v624: 129 pgs: 129 active+clean; 94 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 944 KiB/s wr, 96 op/s 2024-09-10T22:10:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:42 smithi154 ceph-mon[34495]: pgmap v624: 129 pgs: 129 active+clean; 94 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 944 KiB/s wr, 96 op/s 2024-09-10T22:10:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:42 smithi171 ceph-mon[34138]: pgmap v624: 129 pgs: 129 active+clean; 94 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 944 KiB/s wr, 96 op/s 2024-09-10T22:10:43.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 82069 1.33 MB/sec execute 215 sec latency 17.616 ms 2024-09-10T22:10:44.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 82299 1.32 MB/sec execute 216 sec latency 25.722 ms 2024-09-10T22:10:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:44 smithi090 ceph-mon[30350]: pgmap v625: 129 pgs: 129 active+clean; 95 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-10T22:10:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:44 smithi154 ceph-mon[34495]: pgmap v625: 129 pgs: 129 active+clean; 95 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-10T22:10:45.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:44 smithi171 ceph-mon[34138]: pgmap v625: 129 pgs: 129 active+clean; 95 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-09-10T22:10:45.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 82639 1.32 MB/sec execute 217 sec latency 18.436 ms 2024-09-10T22:10:46.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 83040 1.33 MB/sec execute 218 sec latency 20.973 ms 2024-09-10T22:10:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:46 smithi090 ceph-mon[30350]: pgmap v626: 129 pgs: 129 active+clean; 98 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T22:10:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:46 smithi154 ceph-mon[34495]: pgmap v626: 129 pgs: 129 active+clean; 98 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T22:10:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:46 smithi171 ceph-mon[34138]: pgmap v626: 129 pgs: 129 active+clean; 98 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T22:10:47.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 83348 1.33 MB/sec execute 219 sec latency 25.124 ms 2024-09-10T22:10:48.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 83779 1.32 MB/sec execute 220 sec latency 23.362 ms 2024-09-10T22:10:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:48 smithi090 ceph-mon[30350]: pgmap v627: 129 pgs: 129 active+clean; 98 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1018 KiB/s wr, 96 op/s 2024-09-10T22:10:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:48 smithi154 ceph-mon[34495]: pgmap v627: 129 pgs: 129 active+clean; 98 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1018 KiB/s wr, 96 op/s 2024-09-10T22:10:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:48 smithi171 ceph-mon[34138]: pgmap v627: 129 pgs: 129 active+clean; 98 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 1018 KiB/s wr, 96 op/s 2024-09-10T22:10:49.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 84107 1.33 MB/sec execute 221 sec latency 21.387 ms 2024-09-10T22:10:50.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 84610 1.34 MB/sec execute 222 sec latency 16.146 ms 2024-09-10T22:10:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:50 smithi090 ceph-mon[30350]: pgmap v628: 129 pgs: 129 active+clean; 97 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T22:10:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:50 smithi154 ceph-mon[34495]: pgmap v628: 129 pgs: 129 active+clean; 97 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T22:10:51.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:50 smithi171 ceph-mon[34138]: pgmap v628: 129 pgs: 129 active+clean; 97 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T22:10:51.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 84963 1.34 MB/sec execute 223 sec latency 18.337 ms 2024-09-10T22:10:52.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 85319 1.35 MB/sec execute 224 sec latency 41.169 ms 2024-09-10T22:10:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:52 smithi090 ceph-mon[30350]: pgmap v629: 129 pgs: 129 active+clean; 97 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 996 KiB/s wr, 94 op/s 2024-09-10T22:10:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:52 smithi154 ceph-mon[34495]: pgmap v629: 129 pgs: 129 active+clean; 97 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 996 KiB/s wr, 94 op/s 2024-09-10T22:10:53.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:52 smithi171 ceph-mon[34138]: pgmap v629: 129 pgs: 129 active+clean; 97 MiB data, 919 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 996 KiB/s wr, 94 op/s 2024-09-10T22:10:53.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 85543 1.35 MB/sec execute 225 sec latency 17.486 ms 2024-09-10T22:10:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:53 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:10:54.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:53 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:10:54.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:53 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:10:54.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 85623 1.35 MB/sec execute 226 sec latency 511.874 ms 2024-09-10T22:10:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:54 smithi090 ceph-mon[30350]: pgmap v630: 129 pgs: 129 active+clean; 101 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-10T22:10:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:10:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:10:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:10:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:54 smithi154 ceph-mon[34495]: pgmap v630: 129 pgs: 129 active+clean; 101 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-10T22:10:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:10:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:10:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:10:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:54 smithi171 ceph-mon[34138]: pgmap v630: 129 pgs: 129 active+clean; 101 MiB data, 921 MiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-10T22:10:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:10:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:10:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:10:55.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 85842 1.34 MB/sec execute 227 sec latency 23.944 ms 2024-09-10T22:10:56.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 86219 1.34 MB/sec execute 228 sec latency 20.034 ms 2024-09-10T22:10:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:56 smithi154 ceph-mon[34495]: pgmap v631: 129 pgs: 129 active+clean; 101 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-10T22:10:57.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:56 smithi171 ceph-mon[34138]: pgmap v631: 129 pgs: 129 active+clean; 101 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-10T22:10:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:56 smithi090 ceph-mon[30350]: pgmap v631: 129 pgs: 129 active+clean; 101 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-10T22:10:57.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 86588 1.35 MB/sec execute 229 sec latency 20.474 ms 2024-09-10T22:10:58.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 86889 1.35 MB/sec execute 230 sec latency 23.031 ms 2024-09-10T22:10:58.589 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.8 2024-09-10T22:10:58.590 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:10:58.980 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:10:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:10:58 smithi090 ceph-mon[30350]: pgmap v632: 129 pgs: 129 active+clean; 101 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 959 KiB/s wr, 87 op/s 2024-09-10T22:10:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:10:58 smithi154 ceph-mon[34495]: pgmap v632: 129 pgs: 129 active+clean; 101 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 959 KiB/s wr, 87 op/s 2024-09-10T22:10:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:10:58 smithi171 ceph-mon[34138]: pgmap v632: 129 pgs: 129 active+clean; 101 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 959 KiB/s wr, 87 op/s 2024-09-10T22:10:59.473 INFO:tasks.workunit.client.0.smithi090.stdout: 1 87059 1.34 MB/sec execute 231 sec latency 473.407 ms 2024-09-10T22:11:00.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 87272 1.34 MB/sec execute 232 sec latency 494.555 ms 2024-09-10T22:11:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:00 smithi154 ceph-mon[34495]: pgmap v633: 129 pgs: 129 active+clean; 101 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-10T22:11:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:00 smithi171 ceph-mon[34138]: pgmap v633: 129 pgs: 129 active+clean; 101 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-10T22:11:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:00 smithi090 ceph-mon[30350]: pgmap v633: 129 pgs: 129 active+clean; 101 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-10T22:11:01.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 87481 1.34 MB/sec execute 233 sec latency 547.125 ms 2024-09-10T22:11:02.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 87762 1.34 MB/sec execute 234 sec latency 521.414 ms 2024-09-10T22:11:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:02 smithi154 ceph-mon[34495]: pgmap v634: 129 pgs: 129 active+clean; 100 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 886 KiB/s wr, 93 op/s 2024-09-10T22:11:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:02 smithi171 ceph-mon[34138]: pgmap v634: 129 pgs: 129 active+clean; 100 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 886 KiB/s wr, 93 op/s 2024-09-10T22:11:03.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:02 smithi090 ceph-mon[30350]: pgmap v634: 129 pgs: 129 active+clean; 100 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 886 KiB/s wr, 93 op/s 2024-09-10T22:11:03.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 88185 1.35 MB/sec execute 235 sec latency 197.072 ms 2024-09-10T22:11:04.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:03 smithi171 ceph-mon[34138]: pgmap v635: 129 pgs: 129 active+clean; 102 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-09-10T22:11:04.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:03 smithi171 ceph-mon[34138]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T22:11:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:03 smithi090 ceph-mon[30350]: pgmap v635: 129 pgs: 129 active+clean; 102 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-09-10T22:11:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:03 smithi090 ceph-mon[30350]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T22:11:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:03 smithi154 ceph-mon[34495]: pgmap v635: 129 pgs: 129 active+clean; 102 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-09-10T22:11:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:03 smithi154 ceph-mon[34495]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T22:11:04.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 88486 1.35 MB/sec execute 236 sec latency 520.400 ms 2024-09-10T22:11:05.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 88797 1.36 MB/sec execute 237 sec latency 22.277 ms 2024-09-10T22:11:05.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.624+0000 7f5ccfe00640 1 -- 172.21.15.90:0/638671852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cd0104e10 msgr2=0x7f5cd01051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:05.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.624+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/638671852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cd0104e10 0x7f5cd01051f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0009860 tx=0x7f5cc002a340 comp rx=0 tx=0).stop 2024-09-10T22:11:05.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.624+0000 7f5ccfe00640 1 -- 172.21.15.90:0/638671852 shutdown_connections 2024-09-10T22:11:05.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.624+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/638671852 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cd010b1a0 0x7f5cd01119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.624+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/638671852 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd010a820 0x7f5cd010ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.624+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/638671852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cd0104e10 0x7f5cd01051f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.624+0000 7f5ccfe00640 1 -- 172.21.15.90:0/638671852 >> 172.21.15.90:0/638671852 conn(0x7f5cd0100840 msgr2=0x7f5cd0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:05.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.625+0000 7f5ccfe00640 1 -- 172.21.15.90:0/638671852 shutdown_connections 2024-09-10T22:11:05.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.625+0000 7f5ccfe00640 1 -- 172.21.15.90:0/638671852 wait complete. 2024-09-10T22:11:05.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.626+0000 7f5ccfe00640 1 Processor -- start 2024-09-10T22:11:05.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.626+0000 7f5ccfe00640 1 -- start start 2024-09-10T22:11:05.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.626+0000 7f5ccfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cd0104e10 0x7f5cd019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:05.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.626+0000 7f5ccfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd010a820 0x7f5cd019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:05.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.626+0000 7f5ccfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cd010b1a0 0x7f5cd01a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:05.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.626+0000 7f5ccfe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd0114d10 con 0x7f5cd010b1a0 2024-09-10T22:11:05.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.626+0000 7f5ccfe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd0114b90 con 0x7f5cd010a820 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.626+0000 7f5ccfe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd0114e90 con 0x7f5cd0104e10 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5cce000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd010a820 0x7f5cd019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5cce000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd010a820 0x7f5cd019b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:42226/0 (socket says 172.21.15.90:42226) 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5cce000640 1 -- 172.21.15.90:0/1157746279 learned_addr learned my addr 172.21.15.90:0/1157746279 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5ccea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cd0104e10 0x7f5cd019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5ccf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cd010b1a0 0x7f5cd01a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5cce000640 1 -- 172.21.15.90:0/1157746279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cd0104e10 msgr2=0x7f5cd019b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5cce000640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cd0104e10 0x7f5cd019b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5cce000640 1 -- 172.21.15.90:0/1157746279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cd010b1a0 msgr2=0x7f5cd01a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5cce000640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cd010b1a0 0x7f5cd01a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.627+0000 7f5cce000640 1 -- 172.21.15.90:0/1157746279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd01a11d0 con 0x7f5cd010a820 2024-09-10T22:11:05.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.628+0000 7f5cce000640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd010a820 0x7f5cd019b660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00e9c0 tx=0x7f5cbc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:05.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.628+0000 7f5cd4a00640 1 -- 172.21.15.90:0/1157746279 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00ccc0 con 0x7f5cd010a820 2024-09-10T22:11:05.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.628+0000 7f5cd4a00640 1 -- 172.21.15.90:0/1157746279 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5cbc0054b0 con 0x7f5cd010a820 2024-09-10T22:11:05.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.628+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd01a14c0 con 0x7f5cd010a820 2024-09-10T22:11:05.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.628+0000 7f5cd4a00640 1 -- 172.21.15.90:0/1157746279 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc010430 con 0x7f5cd010a820 2024-09-10T22:11:05.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.628+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd01a7b80 con 0x7f5cd010a820 2024-09-10T22:11:05.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.630+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd01051f0 con 0x7f5cd010a820 2024-09-10T22:11:05.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.632+0000 7f5cd4a00640 1 -- 172.21.15.90:0/1157746279 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5cbc004eb0 con 0x7f5cd010a820 2024-09-10T22:11:05.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.634+0000 7f5cd4a00640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cb40b5500 0x7f5cb40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:05.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.634+0000 7f5cd4a00640 1 -- 172.21.15.90:0/1157746279 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9578+0+0 (secure 0 0 0) 0x7f5cbc0e6850 con 0x7f5cd010a820 2024-09-10T22:11:05.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.634+0000 7f5ccea00640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cb40b5500 0x7f5cb40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:05.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.635+0000 7f5ccea00640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cb40b5500 0x7f5cb40b79c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0005cd0 tx=0x7f5cc0000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:05.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.638+0000 7f5cd4a00640 1 -- 172.21.15.90:0/1157746279 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5cbc0acf50 con 0x7f5cd010a820 2024-09-10T22:11:05.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.910+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cd01a7ed0 con 0x7f5cd010a820 2024-09-10T22:11:05.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.910+0000 7f5cd4a00640 1 -- 172.21.15.90:0/1157746279 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11986 (secure 0 0 0) 0x7f5cbc0ac850 con 0x7f5cd010a820 2024-09-10T22:11:05.913 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:11:05.913 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"btime":"2024-09-10T22:08:03:817761+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:08:03.499417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:11:05.913 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-09-10T22:11:05.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cb40b5500 msgr2=0x7f5cb40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:05.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cb40b5500 0x7f5cb40b79c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0005cd0 tx=0x7f5cc0000bc0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd010a820 msgr2=0x7f5cd019b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:05.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd010a820 0x7f5cd019b660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00e9c0 tx=0x7f5cbc00ee80 comp rx=0 tx=0).stop 2024-09-10T22:11:05.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 shutdown_connections 2024-09-10T22:11:05.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f5cb40b5500 0x7f5cb40b79c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cd010b1a0 0x7f5cd01a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5cd010a820 0x7f5cd019b660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 --2- 172.21.15.90:0/1157746279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5cd0104e10 0x7f5cd019b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:05.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 >> 172.21.15.90:0/1157746279 conn(0x7f5cd0100840 msgr2=0x7f5cd0102010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:05.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.913+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 shutdown_connections 2024-09-10T22:11:05.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:05.914+0000 7f5ccfe00640 1 -- 172.21.15.90:0/1157746279 wait complete. 2024-09-10T22:11:06.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89028 1.36 MB/sec execute 238 sec latency 18.625 ms 2024-09-10T22:11:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:06 smithi090 ceph-mon[30350]: pgmap v637: 129 pgs: 129 active+clean; 103 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 758 KiB/s wr, 109 op/s 2024-09-10T22:11:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:06 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1157746279' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:11:06.882 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0' 2024-09-10T22:11:06.882 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 146.8 --await-for 45.0 2024-09-10T22:11:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:06 smithi154 ceph-mon[34495]: pgmap v637: 129 pgs: 129 active+clean; 103 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 758 KiB/s wr, 109 op/s 2024-09-10T22:11:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:06 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1157746279' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:11:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:06 smithi171 ceph-mon[34138]: pgmap v637: 129 pgs: 129 active+clean; 103 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 758 KiB/s wr, 109 op/s 2024-09-10T22:11:07.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:06 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1157746279' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:11:07.304 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:11:07.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89198 1.35 MB/sec execute 239 sec latency 17.376 ms 2024-09-10T22:11:08.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.897+0000 7ff16de00640 1 -- 172.21.15.90:0/516479015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810b350 msgr2=0x7ff168111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.897+0000 7ff16de00640 1 --2- 172.21.15.90:0/516479015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810b350 0x7ff168111b50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff160009ca0 tx=0x7ff16002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.897+0000 7ff16de00640 1 -- 172.21.15.90:0/516479015 shutdown_connections 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.897+0000 7ff16de00640 1 --2- 172.21.15.90:0/516479015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810b350 0x7ff168111b50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.897+0000 7ff16de00640 1 --2- 172.21.15.90:0/516479015 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff16810a9d0 0x7ff16810ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.897+0000 7ff16de00640 1 --2- 172.21.15.90:0/516479015 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff168104fc0 0x7ff1681053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.897+0000 7ff16de00640 1 -- 172.21.15.90:0/516479015 >> 172.21.15.90:0/516479015 conn(0x7ff1681009f0 msgr2=0x7ff168102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.897+0000 7ff16de00640 1 -- 172.21.15.90:0/516479015 shutdown_connections 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.897+0000 7ff16de00640 1 -- 172.21.15.90:0/516479015 wait complete. 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.899+0000 7ff16de00640 1 Processor -- start 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.899+0000 7ff16de00640 1 -- start start 2024-09-10T22:11:08.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.899+0000 7ff16de00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff168104fc0 0x7ff16819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.899+0000 7ff16de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810a9d0 0x7ff16819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.899+0000 7ff16de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff16810b350 0x7ff1681a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.899+0000 7ff16de00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff168114e80 con 0x7ff16810a9d0 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.899+0000 7ff16de00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff168114d00 con 0x7ff16810b350 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.899+0000 7ff16de00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff168115000 con 0x7ff168104fc0 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810a9d0 0x7ff16819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810a9d0 0x7ff16819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58400/0 (socket says 172.21.15.90:58400) 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166000640 1 -- 172.21.15.90:0/53338678 learned_addr learned my addr 172.21.15.90:0/53338678 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166a00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff168104fc0 0x7ff16819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff167400640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff16810b350 0x7ff1681a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166a00640 1 -- 172.21.15.90:0/53338678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff16810b350 msgr2=0x7ff1681a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166a00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff16810b350 0x7ff1681a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166a00640 1 -- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810a9d0 msgr2=0x7ff16819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166a00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810a9d0 0x7ff16819b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166a00640 1 -- 172.21.15.90:0/53338678 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1681a1380 con 0x7ff168104fc0 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.900+0000 7ff166000640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810a9d0 0x7ff16819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.901+0000 7ff166a00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff168104fc0 0x7ff16819b2d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff158005fd0 tx=0x7ff15800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.901+0000 7ff16ca00640 1 -- 172.21.15.90:0/53338678 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158009c30 con 0x7ff168104fc0 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.901+0000 7ff16de00640 1 -- 172.21.15.90:0/53338678 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1681a1670 con 0x7ff168104fc0 2024-09-10T22:11:08.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.901+0000 7ff16ca00640 1 -- 172.21.15.90:0/53338678 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1580044c0 con 0x7ff168104fc0 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.901+0000 7ff16ca00640 1 -- 172.21.15.90:0/53338678 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158003d70 con 0x7ff168104fc0 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.902+0000 7ff16de00640 1 -- 172.21.15.90:0/53338678 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1681a7f00 con 0x7ff168104fc0 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.905+0000 7ff1709ca740 1 Processor -- start 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff1709ca740 1 -- start start 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3ec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c3fae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff1709ca740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55984c361ca0 con 0x55984c3ecaa0 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff1709ca740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55984c3510d0 con 0x55984c3ec180 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff1709ca740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55984c354ee0 con 0x55984c3f8a30 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff16ca00640 1 -- 172.21.15.90:0/53338678 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff158023020 con 0x7ff168104fc0 2024-09-10T22:11:08.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff167400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff167400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3ec560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:56030/0 (socket says 172.21.15.90:56030) 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff167400640 1 -- 172.21.15.90:0/3117497642 learned_addr learned my addr 172.21.15.90:0/3117497642 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.906+0000 7ff166a00640 1 --2- 172.21.15.90:0/3117497642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.907+0000 7ff167400640 1 -- 172.21.15.90:0/3117497642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 msgr2=0x55984c3fae20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.907+0000 7ff167400640 1 --2- 172.21.15.90:0/3117497642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c3fae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.907+0000 7ff167400640 1 -- 172.21.15.90:0/3117497642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 msgr2=0x55984c3f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.907+0000 7ff167400640 1 --2- 172.21.15.90:0/3117497642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3f84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.907+0000 7ff167400640 1 -- 172.21.15.90:0/3117497642 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984c303400 con 0x55984c3ec180 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.907+0000 7ff167400640 1 --2- 172.21.15.90:0/3117497642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3ec560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff160009430 tx=0x7ff160031590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bb352a48520d196 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.908+0000 7ff153e00640 1 -- 172.21.15.90:0/3117497642 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160031b80 con 0x55984c3ec180 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.908+0000 7ff153e00640 1 -- 172.21.15.90:0/3117497642 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff160036040 con 0x55984c3ec180 2024-09-10T22:11:08.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.908+0000 7ff153e00640 1 -- 172.21.15.90:0/3117497642 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1600044e0 con 0x55984c3ec180 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.908+0000 7ff16ca00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff14c0b5550 0x7ff14c0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.908+0000 7ff16ca00640 1 -- 172.21.15.90:0/53338678 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9578+0+0 (secure 0 0 0) 0x7ff1580e5c80 con 0x7ff168104fc0 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff166a00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff14c0b5550 0x7ff14c0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff1709ca740 1 -- 172.21.15.90:0/3117497642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 msgr2=0x55984c3ec560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff1709ca740 1 --2- 172.21.15.90:0/3117497642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3ec560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff160009430 tx=0x7ff160031590 comp rx=0 tx=0).stop 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff1709ca740 1 -- 172.21.15.90:0/3117497642 shutdown_connections 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff1709ca740 1 --2- 172.21.15.90:0/3117497642 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c3fae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff1709ca740 1 --2- 172.21.15.90:0/3117497642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff1709ca740 1 --2- 172.21.15.90:0/3117497642 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3ec560 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff1709ca740 1 -- 172.21.15.90:0/3117497642 >> 172.21.15.90:0/3117497642 conn(0x55984c3e9fb0 msgr2=0x55984c3eb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff1709ca740 1 -- 172.21.15.90:0/3117497642 shutdown_connections 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.909+0000 7ff1709ca740 1 -- 172.21.15.90:0/3117497642 wait complete. 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.910+0000 7ff166a00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff14c0b5550 0x7ff14c0b7a10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55984c3eae90 tx=0x7ff1580e93b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.910+0000 7ff1709ca740 1 -- start start 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.910+0000 7ff1709ca740 1 Processor -- start 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3ec180 0x55984c3f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3f8a30 0x55984c3f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff1709ca740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55984c1b27a0 con 0x55984c3ecaa0 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff1709ca740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55984c2f6a10 con 0x55984c3f8a30 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff1709ca740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55984c3fb650 con 0x55984c3ec180 2024-09-10T22:11:08.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff1709ca740 10 client.0 _ll_register_callbacks cb 0x55984c306be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff167400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3f8a30 0x55984c3f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff167400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3f8a30 0x55984c3f5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:56036/0 (socket says 172.21.15.90:56036) 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff167400640 1 -- 172.21.15.90:0/4110300465 learned_addr learned my addr 172.21.15.90:0/4110300465 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff167400640 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3ec180 0x55984c3f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.911+0000 7ff166000640 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.912+0000 7ff167400640 1 -- 172.21.15.90:0/4110300465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3ec180 msgr2=0x55984c3f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.912+0000 7ff167400640 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3ec180 0x55984c3f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.912+0000 7ff167400640 1 -- 172.21.15.90:0/4110300465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 msgr2=0x55984c3f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.912+0000 7ff167400640 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.912+0000 7ff167400640 1 -- 172.21.15.90:0/4110300465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984c0beb90 con 0x55984c3f8a30 2024-09-10T22:11:08.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.912+0000 7ff166000640 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.912+0000 7ff167400640 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3f8a30 0x55984c3f5e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff160009620 tx=0x7ff16003e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2985e12f852f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.912+0000 7ff150c00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.913+0000 7ff150c00640 1 -- 172.21.15.90:0/4110300465 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff160059060 con 0x55984c3f8a30 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.913+0000 7ff150c00640 1 -- 172.21.15.90:0/4110300465 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9578+0+0 (secure 0 0 0) 0x7ff160031980 con 0x55984c3f8a30 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.913+0000 7ff1709ca740 1 -- 172.21.15.90:0/4110300465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff160036040 con 0x55984c3f8a30 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.914+0000 7ff150c00640 1 -- 172.21.15.90:0/4110300465 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff160031ba0 con 0x55984c3f8a30 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.914+0000 7ff1709ca740 10 client.35639 fetch_fsmap learned FSMap version 35 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.914+0000 7ff1709ca740 1 -- 172.21.15.90:0/4110300465 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55984c406fc0 con 0x55984c3f8a30 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.914+0000 7ff150c00640 1 -- 172.21.15.90:0/4110300465 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff160036040 con 0x55984c3f8a30 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.914+0000 7ff150c00640 1 -- 172.21.15.90:0/4110300465 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ff16005c050 con 0x55984c3f8a30 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.914+0000 7ff1709ca740 10 client.35639 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.914+0000 7ff1709ca740 10 client.35639 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.915+0000 7ff1709ca740 20 client.35639 populate_metadata read hostname 'smithi090' 2024-09-10T22:11:08.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.915+0000 7ff1709ca740 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55984c4083b0 0x55984c40a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.915+0000 7ff1709ca740 4 client.35639 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.915+0000 7ff1709ca740 1 -- 172.21.15.90:0/4110300465 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55984c363bf0 con 0x55984c4083b0 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.915+0000 7ff166000640 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55984c4083b0 0x55984c40a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.915+0000 7ff166000640 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55984c4083b0 0x55984c40a7a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec9d7d88fcf92af server_cookie=feb06d3ee6eaa898 in_seq=0 out_seq=0 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.915+0000 7ff150c00640 10 client.35639 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.916+0000 7ff150c00640 1 -- 172.21.15.90:0/4110300465 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55984c363bf0 con 0x55984c4083b0 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.916+0000 7ff150c00640 10 client.35639 handle_command_reply: tid=0 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.916+0000 7ff1709ca740 1 client.35639 shutdown 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.916+0000 7ff1709ca740 1 -- 172.21.15.90:0/4110300465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3f8a30 msgr2=0x55984c3f5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3f8a30 0x55984c3f5e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff160009620 tx=0x7ff16003e040 comp rx=0 tx=0).stop 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 1 -- 172.21.15.90:0/4110300465 shutdown_connections 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55984c4083b0 0x55984c40a7a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3f8a30 0x55984c3f5e30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 1 --2- 172.21.15.90:0/4110300465 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3ec180 0x55984c3f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 1 -- 172.21.15.90:0/4110300465 >> 172.21.15.90:0/4110300465 conn(0x55984c3e9fb0 msgr2=0x55984c3efb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff150c00640 0 client.35639 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 1 -- 172.21.15.90:0/4110300465 shutdown_connections 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 1 -- 172.21.15.90:0/4110300465 wait complete. 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.917+0000 7ff1709ca740 20 client.35639 trim_cache size 0 max 16384 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.936+0000 7ff1709ca740 1 Processor -- start 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.936+0000 7ff1709ca740 1 -- start start 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.936+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.936+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3eb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.936+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c3f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.936+0000 7ff1709ca740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b21f0 con 0x55984c3ecaa0 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.936+0000 7ff1709ca740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b3d30 con 0x55984c3ec180 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.936+0000 7ff1709ca740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b28c0 con 0x55984c3f8a30 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.936+0000 7ff166000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff166000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3eb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58426/0 (socket says 172.21.15.90:58426) 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff166000640 1 -- 172.21.15.90:0/1424875279 learned_addr learned my addr 172.21.15.90:0/1424875279 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff167400640 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff167400640 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c3f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff166000640 1 -- 172.21.15.90:0/1424875279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 msgr2=0x55984c3f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff166000640 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c3f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff166000640 1 -- 172.21.15.90:0/1424875279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 msgr2=0x55984c3f0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff166000640 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3f0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff166000640 1 -- 172.21.15.90:0/1424875279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55984c3ea800 con 0x55984c3ecaa0 2024-09-10T22:11:08.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff167400640 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff166000640 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3eb530 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff15400afc0 tx=0x7ff154012d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5fb7e28ac43e8a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.937+0000 7ff153400640 1 -- 172.21.15.90:0/1424875279 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15401f040 con 0x55984c3ecaa0 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.938+0000 7ff153400640 1 -- 172.21.15.90:0/1424875279 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff154010d20 con 0x55984c3ecaa0 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.938+0000 7ff153400640 1 -- 172.21.15.90:0/1424875279 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154012660 con 0x55984c3ecaa0 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.938+0000 7ff1709ca740 1 -- 172.21.15.90:0/1424875279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 msgr2=0x55984c3eb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.938+0000 7ff1709ca740 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3eb530 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff15400afc0 tx=0x7ff154012d40 comp rx=0 tx=0).stop 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.938+0000 7ff1709ca740 1 -- 172.21.15.90:0/1424875279 shutdown_connections 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.938+0000 7ff1709ca740 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c3f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.938+0000 7ff1709ca740 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c3eb530 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.938+0000 7ff1709ca740 1 --2- 172.21.15.90:0/1424875279 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c3f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.938+0000 7ff1709ca740 1 -- 172.21.15.90:0/1424875279 >> 172.21.15.90:0/1424875279 conn(0x55984c3e9fb0 msgr2=0x55984c3f7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.939+0000 7ff1709ca740 1 -- 172.21.15.90:0/1424875279 shutdown_connections 2024-09-10T22:11:08.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.939+0000 7ff1709ca740 1 -- 172.21.15.90:0/1424875279 wait complete. 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.939+0000 7ff1709ca740 1 -- start start 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.939+0000 7ff1709ca740 1 Processor -- start 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c40fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c410320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff1709ca740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c40cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff1709ca740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b3660 con 0x55984c3ecaa0 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff1709ca740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b2f90 con 0x55984c3ec180 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff1709ca740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff1681b01f0 con 0x55984c3f8a30 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff1709ca740 10 client.0 _ll_register_callbacks cb 0x7ff1580e9590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff166000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c410320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff166000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c410320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58432/0 (socket says 172.21.15.90:58432) 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff167400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c40fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff166000640 1 -- 172.21.15.90:0/2023902479 learned_addr learned my addr 172.21.15.90:0/2023902479 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.940+0000 7ff167400640 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c40cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.941+0000 7ff166000640 1 -- 172.21.15.90:0/2023902479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 msgr2=0x55984c40cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.941+0000 7ff166000640 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c40cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.941+0000 7ff166000640 1 -- 172.21.15.90:0/2023902479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 msgr2=0x55984c40fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.941+0000 7ff166000640 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c40fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.941+0000 7ff166000640 1 -- 172.21.15.90:0/2023902479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55984c40d300 con 0x55984c3ecaa0 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.941+0000 7ff167400640 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c40fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.941+0000 7ff166000640 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c410320 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff154012040 tx=0x7ff154010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4f5ed6f67bcba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.941+0000 7ff150c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.942+0000 7ff150c00640 1 -- 172.21.15.90:0/2023902479 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154033060 con 0x55984c3ecaa0 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.942+0000 7ff150c00640 1 -- 172.21.15.90:0/2023902479 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9578+0+0 (secure 0 0 0) 0x7ff1540125e0 con 0x55984c3ecaa0 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.942+0000 7ff1709ca740 1 -- 172.21.15.90:0/2023902479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff154010d20 con 0x55984c3ecaa0 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.942+0000 7ff150c00640 1 -- 172.21.15.90:0/2023902479 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff15403a050 con 0x55984c3ecaa0 2024-09-10T22:11:08.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.942+0000 7ff1709ca740 10 client.15984 fetch_fsmap learned FSMap version 35 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.942+0000 7ff1709ca740 1 -- 172.21.15.90:0/2023902479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55984c475860 con 0x55984c3ecaa0 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.942+0000 7ff150c00640 1 -- 172.21.15.90:0/2023902479 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff154010d20 con 0x55984c3ecaa0 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.943+0000 7ff150c00640 1 -- 172.21.15.90:0/2023902479 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7ff1540046c0 con 0x55984c3ecaa0 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.943+0000 7ff1709ca740 10 client.15984 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.943+0000 7ff1709ca740 10 client.15984 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.943+0000 7ff1709ca740 20 client.15984 populate_metadata read hostname 'smithi090' 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.943+0000 7ff1709ca740 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55984c476030 0x55984c478450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.943+0000 7ff1709ca740 4 client.15984 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}] 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.943+0000 7ff1709ca740 1 -- 172.21.15.90:0/2023902479 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.8, "await_for": 45.0}) -- 0x55984c35f6a0 con 0x55984c476030 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.943+0000 7ff167400640 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55984c476030 0x55984c478450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.944+0000 7ff167400640 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55984c476030 0x55984c478450 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf0a300eafc677e server_cookie=17809b029a7f6ffb in_seq=0 out_seq=0 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.944+0000 7ff150c00640 10 client.15984 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.999+0000 7ff150c00640 1 -- 172.21.15.90:0/2023902479 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55984c35f6a0 con 0x55984c476030 2024-09-10T22:11:08.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.999+0000 7ff150c00640 10 client.15984 handle_command_reply: tid=0 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.999+0000 7ff1709ca740 1 client.15984 shutdown 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.999+0000 7ff1709ca740 1 -- 172.21.15.90:0/2023902479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 msgr2=0x55984c410320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:07.999+0000 7ff1709ca740 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c410320 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff154012040 tx=0x7ff154010660 comp rx=0 tx=0).stop 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff1709ca740 1 -- 172.21.15.90:0/2023902479 shutdown_connections 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff1709ca740 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55984c476030 0x55984c478450 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 35, 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 22, 2024-09-10T22:11:08.070 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "11553012": { 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 22, 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 146.8, 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:11:08.071 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:11:08.072 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:11:08.072 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:08.072 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:08.072 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:08.072 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:08.072 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:08.072 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff1709ca740 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55984c3f8a30 0x55984c40cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff1709ca740 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55984c3ecaa0 0x55984c410320 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff1709ca740 1 --2- 172.21.15.90:0/2023902479 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55984c3ec180 0x55984c40fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff1709ca740 1 -- 172.21.15.90:0/2023902479 >> 172.21.15.90:0/2023902479 conn(0x55984c3e9fb0 msgr2=0x55984c3e8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff150c00640 0 client.15984 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff1709ca740 1 -- 172.21.15.90:0/2023902479 shutdown_connections 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff1709ca740 1 -- 172.21.15.90:0/2023902479 wait complete. 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.000+0000 7ff1709ca740 20 client.15984 trim_cache size 0 max 16384 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 -- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff14c0b5550 msgr2=0x7ff14c0b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:08.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff14c0b5550 0x7ff14c0b7a10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55984c3eae90 tx=0x7ff1580e93b0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 -- 172.21.15.90:0/53338678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff168104fc0 msgr2=0x7ff16819b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff168104fc0 0x7ff16819b2d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff158005fd0 tx=0x7ff15800dde0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 -- 172.21.15.90:0/53338678 shutdown_connections 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff14c0b5550 0x7ff14c0b7a10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff16810b350 0x7ff1681a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff16810a9d0 0x7ff16819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 --2- 172.21.15.90:0/53338678 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff168104fc0 0x7ff16819b2d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 -- 172.21.15.90:0/53338678 >> 172.21.15.90:0/53338678 conn(0x7ff1681009f0 msgr2=0x7ff168102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 -- 172.21.15.90:0/53338678 shutdown_connections 2024-09-10T22:11:08.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:08.001+0000 7ff16de00640 1 -- 172.21.15.90:0/53338678 wait complete. 2024-09-10T22:11:08.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.35 MB/sec execute 240 sec latency 489.557 ms 2024-09-10T22:11:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:08 smithi090 ceph-mon[30350]: pgmap v638: 129 pgs: 129 active+clean; 104 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 899 KiB/s wr, 108 op/s 2024-09-10T22:11:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:08 smithi154 ceph-mon[34495]: pgmap v638: 129 pgs: 129 active+clean; 104 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 899 KiB/s wr, 108 op/s 2024-09-10T22:11:09.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:08 smithi171 ceph-mon[34138]: pgmap v638: 129 pgs: 129 active+clean; 104 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 899 KiB/s wr, 108 op/s 2024-09-10T22:11:09.262 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '11553012', quiesce took 10.7 seconds. Will release after: 16.1 2024-09-10T22:11:09.474 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.34 MB/sec execute 241 sec latency 1489.670 ms 2024-09-10T22:11:10.475 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.34 MB/sec execute 242 sec latency 2489.806 ms 2024-09-10T22:11:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:10 smithi090 ceph-mon[30350]: pgmap v639: 129 pgs: 129 active+clean; 107 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 985 KiB/s wr, 103 op/s 2024-09-10T22:11:11.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:10 smithi090 ceph-mon[30350]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T22:11:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:10 smithi154 ceph-mon[34495]: pgmap v639: 129 pgs: 129 active+clean; 107 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 985 KiB/s wr, 103 op/s 2024-09-10T22:11:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:10 smithi154 ceph-mon[34495]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T22:11:11.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:10 smithi171 ceph-mon[34138]: pgmap v639: 129 pgs: 129 active+clean; 107 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 345 KiB/s rd, 985 KiB/s wr, 103 op/s 2024-09-10T22:11:11.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:10 smithi171 ceph-mon[34138]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T22:11:11.475 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.33 MB/sec execute 243 sec latency 3489.885 ms 2024-09-10T22:11:12.475 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.33 MB/sec execute 244 sec latency 4489.997 ms 2024-09-10T22:11:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:12 smithi090 ceph-mon[30350]: pgmap v641: 129 pgs: 129 active+clean; 107 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 875 KiB/s wr, 75 op/s 2024-09-10T22:11:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:12 smithi154 ceph-mon[34495]: pgmap v641: 129 pgs: 129 active+clean; 107 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 875 KiB/s wr, 75 op/s 2024-09-10T22:11:13.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:12 smithi171 ceph-mon[34138]: pgmap v641: 129 pgs: 129 active+clean; 107 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 875 KiB/s wr, 75 op/s 2024-09-10T22:11:13.475 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.32 MB/sec execute 245 sec latency 5490.123 ms 2024-09-10T22:11:14.475 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.32 MB/sec execute 246 sec latency 6490.290 ms 2024-09-10T22:11:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:14 smithi090 ceph-mon[30350]: pgmap v642: 129 pgs: 129 active+clean; 105 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 711 KiB/s wr, 64 op/s 2024-09-10T22:11:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:14 smithi154 ceph-mon[34495]: pgmap v642: 129 pgs: 129 active+clean; 105 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 711 KiB/s wr, 64 op/s 2024-09-10T22:11:15.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:14 smithi171 ceph-mon[34138]: pgmap v642: 129 pgs: 129 active+clean; 105 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 711 KiB/s wr, 64 op/s 2024-09-10T22:11:15.475 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.31 MB/sec execute 247 sec latency 7490.454 ms 2024-09-10T22:11:16.475 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.31 MB/sec execute 248 sec latency 8490.506 ms 2024-09-10T22:11:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:16 smithi090 ceph-mon[30350]: pgmap v643: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 612 KiB/s wr, 59 op/s 2024-09-10T22:11:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:16 smithi154 ceph-mon[34495]: pgmap v643: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 612 KiB/s wr, 59 op/s 2024-09-10T22:11:17.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:16 smithi171 ceph-mon[34138]: pgmap v643: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 612 KiB/s wr, 59 op/s 2024-09-10T22:11:17.475 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.30 MB/sec execute 249 sec latency 9490.650 ms 2024-09-10T22:11:18.476 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.29 MB/sec execute 250 sec latency 10490.793 ms 2024-09-10T22:11:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:18 smithi090 ceph-mon[30350]: pgmap v644: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 404 KiB/s wr, 58 op/s 2024-09-10T22:11:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:18 smithi154 ceph-mon[34495]: pgmap v644: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 404 KiB/s wr, 58 op/s 2024-09-10T22:11:19.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:18 smithi171 ceph-mon[34138]: pgmap v644: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 404 KiB/s wr, 58 op/s 2024-09-10T22:11:19.476 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.29 MB/sec execute 251 sec latency 11490.930 ms 2024-09-10T22:11:20.476 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.28 MB/sec execute 252 sec latency 12491.065 ms 2024-09-10T22:11:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:20 smithi090 ceph-mon[30350]: pgmap v645: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-09-10T22:11:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:20 smithi154 ceph-mon[34495]: pgmap v645: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-09-10T22:11:21.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:20 smithi171 ceph-mon[34138]: pgmap v645: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 36 KiB/s wr, 20 op/s 2024-09-10T22:11:21.476 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.28 MB/sec execute 253 sec latency 13491.208 ms 2024-09-10T22:11:22.476 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.27 MB/sec execute 254 sec latency 14491.374 ms 2024-09-10T22:11:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:22 smithi090 ceph-mon[30350]: pgmap v646: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-09-10T22:11:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:22 smithi154 ceph-mon[34495]: pgmap v646: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-09-10T22:11:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:22 smithi171 ceph-mon[34138]: pgmap v646: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-09-10T22:11:23.476 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.27 MB/sec execute 255 sec latency 15491.495 ms 2024-09-10T22:11:24.476 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.26 MB/sec execute 256 sec latency 16491.674 ms 2024-09-10T22:11:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:24 smithi090 ceph-mon[30350]: pgmap v647: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-10T22:11:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:24 smithi154 ceph-mon[34495]: pgmap v647: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-10T22:11:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:24 smithi171 ceph-mon[34138]: pgmap v647: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 KiB/s wr, 15 op/s 2024-09-10T22:11:25.363 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:11:25.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.26 MB/sec execute 257 sec latency 17491.726 ms 2024-09-10T22:11:25.746 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:11:26.476 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.25 MB/sec execute 258 sec latency 18491.795 ms 2024-09-10T22:11:26.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.348+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1695341562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 msgr2=0x7fc1d0104070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:26.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.348+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1695341562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 0x7fc1d0104070 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009900 tx=0x7fc1b802a410 comp rx=0 tx=0).stop 2024-09-10T22:11:26.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.348+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1695341562 shutdown_connections 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.348+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1695341562 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1d010d0f0 0x7fc1d010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.348+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1695341562 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1d01045b0 0x7fc1d010caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.348+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1695341562 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 0x7fc1d0104070 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.348+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1695341562 >> 172.21.15.90:0/1695341562 conn(0x7fc1d00fd710 msgr2=0x7fc1d00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.348+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1695341562 shutdown_connections 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.348+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1695341562 wait complete. 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.349+0000 7fc1d4c00640 1 Processor -- start 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1d4c00640 1 -- start start 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1d4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 0x7fc1d0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1d4c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1d01045b0 0x7fc1d01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1d4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1d010d0f0 0x7fc1d019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:26.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1d4c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0112c60 con 0x7fc1d0103c90 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1d4c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0112ae0 con 0x7fc1d010d0f0 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1d4c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0112de0 con 0x7fc1d01045b0 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1cde00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 0x7fc1d0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1cde00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 0x7fc1d0199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58474/0 (socket says 172.21.15.90:58474) 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1cde00640 1 -- 172.21.15.90:0/1400178599 learned_addr learned my addr 172.21.15.90:0/1400178599 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.350+0000 7fc1cd400640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1d01045b0 0x7fc1d01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1ce800640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1d010d0f0 0x7fc1d019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1cde00640 1 -- 172.21.15.90:0/1400178599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1d01045b0 msgr2=0x7fc1d01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1cde00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1d01045b0 0x7fc1d01996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1cde00640 1 -- 172.21.15.90:0/1400178599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1d010d0f0 msgr2=0x7fc1d019ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1cde00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1d010d0f0 0x7fc1d019ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1cde00640 1 -- 172.21.15.90:0/1400178599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d019f220 con 0x7fc1d0103c90 2024-09-10T22:11:26.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1ce800640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1d010d0f0 0x7fc1d019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1cde00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 0x7fc1d0199170 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc1b802a930 tx=0x7fc1b8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1c6a00640 1 -- 172.21.15.90:0/1400178599 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b803f070 con 0x7fc1d0103c90 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1c6a00640 1 -- 172.21.15.90:0/1400178599 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc1b8003e40 con 0x7fc1d0103c90 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d019f4b0 con 0x7fc1d0103c90 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1c6a00640 1 -- 172.21.15.90:0/1400178599 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8035030 con 0x7fc1d0103c90 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.351+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d01a5bd0 con 0x7fc1d0103c90 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.352+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d0104070 con 0x7fc1d0103c90 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.354+0000 7fc1c6a00640 1 -- 172.21.15.90:0/1400178599 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc1b8031400 con 0x7fc1d0103c90 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.356+0000 7fc1c6a00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc1b40b5450 0x7fc1b40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.356+0000 7fc1c6a00640 1 -- 172.21.15.90:0/1400178599 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9490+0+0 (secure 0 0 0) 0x7fc1b8107360 con 0x7fc1d0103c90 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.356+0000 7fc1cd400640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc1b40b5450 0x7fc1b40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.357+0000 7fc1cd400640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc1b40b5450 0x7fc1b40b7910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00058c0 tx=0x7fc1c0005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:26.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.358+0000 7fc1c6a00640 1 -- 172.21.15.90:0/1400178599 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc1b80cdab0 con 0x7fc1d0103c90 2024-09-10T22:11:26.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.579+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1d01a5e70 con 0x7fc1d0103c90 2024-09-10T22:11:26.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.580+0000 7fc1c6a00640 1 -- 172.21.15.90:0/1400178599 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11986 (secure 0 0 0) 0x7fc1b80cd3b0 con 0x7fc1d0103c90 2024-09-10T22:11:26.582 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:11:26.582 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"btime":"2024-09-10T22:08:03:817761+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:08:03.499417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:11:26.583 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-09-10T22:11:26.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.582+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc1b40b5450 msgr2=0x7fc1b40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.582+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc1b40b5450 0x7fc1b40b7910 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc1c00058c0 tx=0x7fc1c0005850 comp rx=0 tx=0).stop 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.582+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 msgr2=0x7fc1d0199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.582+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 0x7fc1d0199170 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc1b802a930 tx=0x7fc1b8003c20 comp rx=0 tx=0).stop 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.583+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 shutdown_connections 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.583+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc1b40b5450 0x7fc1b40b7910 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.583+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc1d010d0f0 0x7fc1d019ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.583+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc1d01045b0 0x7fc1d01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.583+0000 7fc1d4c00640 1 --2- 172.21.15.90:0/1400178599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1d0103c90 0x7fc1d0199170 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.583+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 >> 172.21.15.90:0/1400178599 conn(0x7fc1d00fd710 msgr2=0x7fc1d00fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.583+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 shutdown_connections 2024-09-10T22:11:26.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:26.583+0000 7fc1d4c00640 1 -- 172.21.15.90:0/1400178599 wait complete. 2024-09-10T22:11:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:26 smithi090 ceph-mon[30350]: pgmap v648: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-09-10T22:11:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:26 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1400178599' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:11:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:26 smithi154 ceph-mon[34495]: pgmap v648: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-09-10T22:11:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:26 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1400178599' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:11:27.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:26 smithi171 ceph-mon[34138]: pgmap v648: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 30 KiB/s wr, 16 op/s 2024-09-10T22:11:27.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:26 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1400178599' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:11:27.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.25 MB/sec execute 259 sec latency 19491.842 ms 2024-09-10T22:11:27.585 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 11553012 --release --await' 2024-09-10T22:11:27.586 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 11553012 --release --await 2024-09-10T22:11:27.955 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:11:28.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89304 1.25 MB/sec execute 260 sec latency 20491.898 ms 2024-09-10T22:11:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.686+0000 7fca0c800640 1 -- 172.21.15.90:0/302134773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410b330 msgr2=0x7fca04111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.686+0000 7fca0c800640 1 --2- 172.21.15.90:0/302134773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410b330 0x7fca04111b30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fca00009ca0 tx=0x7fca0002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.686+0000 7fca0c800640 1 -- 172.21.15.90:0/302134773 shutdown_connections 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.686+0000 7fca0c800640 1 --2- 172.21.15.90:0/302134773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410b330 0x7fca04111b30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.686+0000 7fca0c800640 1 --2- 172.21.15.90:0/302134773 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0410a9b0 0x7fca0410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.686+0000 7fca0c800640 1 --2- 172.21.15.90:0/302134773 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca04104fa0 0x7fca04105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.686+0000 7fca0c800640 1 -- 172.21.15.90:0/302134773 >> 172.21.15.90:0/302134773 conn(0x7fca04100a50 msgr2=0x7fca04102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.686+0000 7fca0c800640 1 -- 172.21.15.90:0/302134773 shutdown_connections 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.686+0000 7fca0c800640 1 -- 172.21.15.90:0/302134773 wait complete. 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.687+0000 7fca0c800640 1 Processor -- start 2024-09-10T22:11:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.687+0000 7fca0c800640 1 -- start start 2024-09-10T22:11:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.687+0000 7fca0c800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca04104fa0 0x7fca0419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.687+0000 7fca0c800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410a9b0 0x7fca0419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.687+0000 7fca0c800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0410b330 0x7fca041a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.687+0000 7fca0c800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca04114e60 con 0x7fca0410a9b0 2024-09-10T22:11:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.687+0000 7fca0c800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca04114ce0 con 0x7fca0410b330 2024-09-10T22:11:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.687+0000 7fca0c800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca04114fe0 con 0x7fca04104fa0 2024-09-10T22:11:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.687+0000 7fca09400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410a9b0 0x7fca0419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca09400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410a9b0 0x7fca0419b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58238/0 (socket says 172.21.15.90:58238) 2024-09-10T22:11:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca09400640 1 -- 172.21.15.90:0/3647587585 learned_addr learned my addr 172.21.15.90:0/3647587585 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca0a800640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0410b330 0x7fca041a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca09e00640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca04104fa0 0x7fca0419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca09400640 1 -- 172.21.15.90:0/3647587585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca04104fa0 msgr2=0x7fca0419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca09400640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca04104fa0 0x7fca0419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca09400640 1 -- 172.21.15.90:0/3647587585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0410b330 msgr2=0x7fca041a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca09400640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0410b330 0x7fca041a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca09400640 1 -- 172.21.15.90:0/3647587585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca041a1380 con 0x7fca0410a9b0 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.688+0000 7fca09400640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410a9b0 0x7fca0419b810 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400d950 tx=0x7fc9f400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.689+0000 7fc9fea00640 1 -- 172.21.15.90:0/3647587585 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4017070 con 0x7fca0410a9b0 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.689+0000 7fc9fea00640 1 -- 172.21.15.90:0/3647587585 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9f400de80 con 0x7fca0410a9b0 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.689+0000 7fc9fea00640 1 -- 172.21.15.90:0/3647587585 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f400f620 con 0x7fca0410a9b0 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.689+0000 7fca0c800640 1 -- 172.21.15.90:0/3647587585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca041a1670 con 0x7fca0410a9b0 2024-09-10T22:11:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.689+0000 7fca0c800640 1 -- 172.21.15.90:0/3647587585 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca041a7d30 con 0x7fca0410a9b0 2024-09-10T22:11:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.692+0000 7fca0f3e9740 1 Processor -- start 2024-09-10T22:11:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.692+0000 7fca0f3e9740 1 -- start start 2024-09-10T22:11:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.692+0000 7fc9fea00640 1 -- 172.21.15.90:0/3647587585 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc9f4010460 con 0x7fca0410a9b0 2024-09-10T22:11:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a22360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a24c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56277798d290 con 0x562777a15ff0 2024-09-10T22:11:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x56277797c6c0 con 0x562777a228a0 2024-09-10T22:11:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5627779804d0 con 0x562777a16910 2024-09-10T22:11:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca0a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca0a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a163d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58244/0 (socket says 172.21.15.90:58244) 2024-09-10T22:11:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca0a800640 1 -- 172.21.15.90:0/1458516823 learned_addr learned my addr 172.21.15.90:0/1458516823 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca09e00640 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a22360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.693+0000 7fca09400640 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a24c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fca0a800640 1 -- 172.21.15.90:0/1458516823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 msgr2=0x562777a22360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fca0a800640 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a22360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fca0a800640 1 -- 172.21.15.90:0/1458516823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 msgr2=0x562777a24c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fca0a800640 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a24c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fca0a800640 1 -- 172.21.15.90:0/1458516823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562777909460 con 0x562777a15ff0 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fca09400640 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a24c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fca0a800640 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a163d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fca00035000 tx=0x7fca00031590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5f756b62b1b725 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fc9f3000640 1 -- 172.21.15.90:0/1458516823 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00031b80 con 0x562777a15ff0 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fc9f3000640 1 -- 172.21.15.90:0/1458516823 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca00036070 con 0x562777a15ff0 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fc9f3000640 1 -- 172.21.15.90:0/1458516823 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca000043c0 con 0x562777a15ff0 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.694+0000 7fc9fea00640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc9ec0b54d0 0x7fc9ec0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fc9fea00640 1 -- 172.21.15.90:0/3647587585 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9490+0+0 (secure 0 0 0) 0x7fc9f40e5f20 con 0x7fca0410a9b0 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fca09e00640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc9ec0b54d0 0x7fc9ec0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fca0f3e9740 1 -- 172.21.15.90:0/1458516823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 msgr2=0x562777a163d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a163d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fca00035000 tx=0x7fca00031590 comp rx=0 tx=0).stop 2024-09-10T22:11:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fca0f3e9740 1 -- 172.21.15.90:0/1458516823 shutdown_connections 2024-09-10T22:11:28.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a24c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a22360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/1458516823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a163d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fca0f3e9740 1 -- 172.21.15.90:0/1458516823 >> 172.21.15.90:0/1458516823 conn(0x562777a13e20 msgr2=0x562777a14fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:28.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.695+0000 7fca0f3e9740 1 -- 172.21.15.90:0/1458516823 shutdown_connections 2024-09-10T22:11:28.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.696+0000 7fca0f3e9740 1 -- 172.21.15.90:0/1458516823 wait complete. 2024-09-10T22:11:28.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.696+0000 7fca09e00640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc9ec0b54d0 0x7fc9ec0b7990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x562777a19170 tx=0x7fc9f800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:28.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.696+0000 7fca0f3e9740 1 -- start start 2024-09-10T22:11:28.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.696+0000 7fca0f3e9740 1 Processor -- start 2024-09-10T22:11:28.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a15ff0 0x562777a1f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a1fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a228a0 0x562777a202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5627777ddbb0 con 0x562777a228a0 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5627779176c0 con 0x562777a15ff0 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x562777a25370 con 0x562777a16910 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0f3e9740 10 client.0 _ll_register_callbacks cb 0x562777932490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a228a0 0x562777a202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a228a0 0x562777a202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58248/0 (socket says 172.21.15.90:58248) 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca0a800640 1 -- 172.21.15.90:0/3132278002 learned_addr learned my addr 172.21.15.90:0/3132278002 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca09400640 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a1fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.697+0000 7fca09e00640 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a15ff0 0x562777a1f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fca0a800640 1 -- 172.21.15.90:0/3132278002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 msgr2=0x562777a1fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fca0a800640 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a1fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fca0a800640 1 -- 172.21.15.90:0/3132278002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a15ff0 msgr2=0x562777a1f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fca0a800640 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a15ff0 0x562777a1f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fca0a800640 1 -- 172.21.15.90:0/3132278002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627776dab00 con 0x562777a228a0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fca09400640 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a1fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fca0a800640 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a228a0 0x562777a202f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fca00009dc0 tx=0x7fca00003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5cd2ec28c6480 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fc9e7400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fc9e7400640 1 -- 172.21.15.90:0/3132278002 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00004320 con 0x562777a228a0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fc9e7400640 1 -- 172.21.15.90:0/3132278002 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9490+0+0 (secure 0 0 0) 0x7fca000044c0 con 0x562777a228a0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fca0f3e9740 1 -- 172.21.15.90:0/3132278002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca00036070 con 0x562777a228a0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.698+0000 7fc9e7400640 1 -- 172.21.15.90:0/3132278002 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fca000098e0 con 0x562777a228a0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fca0f3e9740 10 client.16002 fetch_fsmap learned FSMap version 35 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fca0f3e9740 1 -- 172.21.15.90:0/3132278002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562777751960 con 0x562777a228a0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fc9e7400640 1 -- 172.21.15.90:0/3132278002 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca00036070 con 0x562777a228a0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fc9e7400640 1 -- 172.21.15.90:0/3132278002 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fca00043020 con 0x562777a228a0 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fca0f3e9740 10 client.16002 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:11:28.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fca0f3e9740 10 client.16002 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:11:28.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fca0f3e9740 20 client.16002 populate_metadata read hostname 'smithi090' 2024-09-10T22:11:28.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562777a320e0 0x562777a344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fca0f3e9740 4 client.16002 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:11:28.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.699+0000 7fca0f3e9740 1 -- 172.21.15.90:0/3132278002 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56277798f1e0 con 0x562777a320e0 2024-09-10T22:11:28.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.700+0000 7fca09e00640 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562777a320e0 0x562777a344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.700+0000 7fc9e7400640 10 client.16002 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:11:28.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.700+0000 7fca09e00640 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562777a320e0 0x562777a344d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63fe2abdf75943bc server_cookie=151097671445d008 in_seq=0 out_seq=0 2024-09-10T22:11:28.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.701+0000 7fc9e7400640 1 -- 172.21.15.90:0/3132278002 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56277798f1e0 con 0x562777a320e0 2024-09-10T22:11:28.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.701+0000 7fc9e7400640 10 client.16002 handle_command_reply: tid=0 2024-09-10T22:11:28.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.701+0000 7fca0f3e9740 1 client.16002 shutdown 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.701+0000 7fca0f3e9740 1 -- 172.21.15.90:0/3132278002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a228a0 msgr2=0x562777a202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.701+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a228a0 0x562777a202f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fca00009dc0 tx=0x7fca00003c20 comp rx=0 tx=0).stop 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fca0f3e9740 1 -- 172.21.15.90:0/3132278002 shutdown_connections 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562777a320e0 0x562777a344d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a228a0 0x562777a202f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a1fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/3132278002 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a15ff0 0x562777a1f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fca0f3e9740 1 -- 172.21.15.90:0/3132278002 >> 172.21.15.90:0/3132278002 conn(0x562777a13e20 msgr2=0x562777a199c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fc9e7400640 0 client.16002 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fca0f3e9740 1 -- 172.21.15.90:0/3132278002 shutdown_connections 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fca0f3e9740 1 -- 172.21.15.90:0/3132278002 wait complete. 2024-09-10T22:11:28.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.702+0000 7fca0f3e9740 20 client.16002 trim_cache size 0 max 16384 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.720+0000 7fca0f3e9740 1 Processor -- start 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.720+0000 7fca0f3e9740 1 -- start start 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.720+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a1ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.720+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a12c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.720+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a1a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.720+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca041b1f30 con 0x562777a15ff0 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.720+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca041b3a70 con 0x562777a228a0 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.720+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca041b2600 con 0x562777a16910 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a1ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a1ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58252/0 (socket says 172.21.15.90:58252) 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09e00640 1 -- 172.21.15.90:0/2946332533 learned_addr learned my addr 172.21.15.90:0/2946332533 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca0a800640 1 --2- 172.21.15.90:0/2946332533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a1a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09400640 1 --2- 172.21.15.90:0/2946332533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a12c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09e00640 1 -- 172.21.15.90:0/2946332533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 msgr2=0x562777a12c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09e00640 1 --2- 172.21.15.90:0/2946332533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a12c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09e00640 1 -- 172.21.15.90:0/2946332533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 msgr2=0x562777a1a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09e00640 1 --2- 172.21.15.90:0/2946332533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a1a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09e00640 1 -- 172.21.15.90:0/2946332533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562777a1dd70 con 0x562777a15ff0 2024-09-10T22:11:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.721+0000 7fca09e00640 1 --2- 172.21.15.90:0/2946332533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a1ad00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x562777a34b00 tx=0x7fc9f8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48664aa6c5c6187d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fc9f2600640 1 -- 172.21.15.90:0/2946332533 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f801be00 con 0x562777a15ff0 2024-09-10T22:11:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fc9f2600640 1 -- 172.21.15.90:0/2946332533 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9f80288e0 con 0x562777a15ff0 2024-09-10T22:11:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fc9f2600640 1 -- 172.21.15.90:0/2946332533 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f800f950 con 0x562777a15ff0 2024-09-10T22:11:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2946332533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 msgr2=0x562777a1ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2946332533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a1ad00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x562777a34b00 tx=0x7fc9f8017000 comp rx=0 tx=0).stop 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2946332533 shutdown_connections 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2946332533 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a1a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2946332533 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a12c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2946332533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a1ad00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2946332533 >> 172.21.15.90:0/2946332533 conn(0x562777a13e20 msgr2=0x562777a216c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.722+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2946332533 shutdown_connections 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.723+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2946332533 wait complete. 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.723+0000 7fca0f3e9740 1 -- start start 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.723+0000 7fca0f3e9740 1 Processor -- start 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.723+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.723+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a26fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.723+0000 7fca0f3e9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a27520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.723+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca041b33a0 con 0x562777a15ff0 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.723+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca041b2cd0 con 0x562777a228a0 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.723+0000 7fca0f3e9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca041aff30 con 0x562777a16910 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca0f3e9740 10 client.0 _ll_register_callbacks cb 0x7fc9f800a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a2ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58266/0 (socket says 172.21.15.90:58266) 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09e00640 1 -- 172.21.15.90:0/2915298043 learned_addr learned my addr 172.21.15.90:0/2915298043 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09400640 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a26fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09e00640 1 -- 172.21.15.90:0/2915298043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 msgr2=0x562777a26fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09e00640 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a26fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09e00640 1 -- 172.21.15.90:0/2915298043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 msgr2=0x562777a27520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09e00640 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a27520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09e00640 1 -- 172.21.15.90:0/2915298043 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562777a27a90 con 0x562777a15ff0 2024-09-10T22:11:28.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.724+0000 7fca09400640 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a26fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:11:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.725+0000 7fca09e00640 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a2ab00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8011010 tx=0x7fc9f801ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694b1180d9a984e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:11:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.725+0000 7fc9e7400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:11:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.725+0000 7fc9e7400640 1 -- 172.21.15.90:0/2915298043 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f803d060 con 0x562777a15ff0 2024-09-10T22:11:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.725+0000 7fc9e7400640 1 -- 172.21.15.90:0/2915298043 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9490+0+0 (secure 0 0 0) 0x7fc9f8028650 con 0x562777a15ff0 2024-09-10T22:11:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.725+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2915298043 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f80288e0 con 0x562777a15ff0 2024-09-10T22:11:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.725+0000 7fc9e7400640 1 -- 172.21.15.90:0/2915298043 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc9f8044050 con 0x562777a15ff0 2024-09-10T22:11:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.725+0000 7fca0f3e9740 10 client.16008 fetch_fsmap learned FSMap version 35 2024-09-10T22:11:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.725+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2915298043 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562777a9fb60 con 0x562777a15ff0 2024-09-10T22:11:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.726+0000 7fc9e7400640 1 -- 172.21.15.90:0/2915298043 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc9f80288e0 con 0x562777a15ff0 2024-09-10T22:11:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.726+0000 7fc9e7400640 1 -- 172.21.15.90:0/2915298043 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fc9f8037030 con 0x562777a15ff0 2024-09-10T22:11:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.726+0000 7fca0f3e9740 10 client.16008 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:11:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.726+0000 7fca0f3e9740 10 client.16008 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:11:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.726+0000 7fca0f3e9740 20 client.16008 populate_metadata read hostname 'smithi090' 2024-09-10T22:11:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.726+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562777aa0220 0x562777aa2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:11:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.726+0000 7fca0f3e9740 4 client.16008 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "11553012", "release": true, "await": true}] 2024-09-10T22:11:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.726+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2915298043 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "11553012", "release": true, "await": true}) -- 0x56277798ac90 con 0x562777aa0220 2024-09-10T22:11:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.726+0000 7fca09400640 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562777aa0220 0x562777aa2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:11:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.727+0000 7fca09400640 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562777aa0220 0x562777aa2640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831464a75368df45 server_cookie=9bb10c6e9a8cd055 in_seq=0 out_seq=0 2024-09-10T22:11:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.727+0000 7fc9e7400640 10 client.16008 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.734+0000 7fc9e7400640 1 -- 172.21.15.90:0/2915298043 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56277798ac90 con 0x562777aa0220 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.734+0000 7fc9e7400640 10 client.16008 handle_command_reply: tid=0 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.734+0000 7fca0f3e9740 1 client.16008 shutdown 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.734+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2915298043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 msgr2=0x562777a2ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.734+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a2ab00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8011010 tx=0x7fc9f801ce10 comp rx=0 tx=0).stop 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2915298043 shutdown_connections 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x562777aa0220 0x562777aa2640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x562777a228a0 0x562777a27520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x562777a16910 0x562777a26fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fca0f3e9740 1 --2- 172.21.15.90:0/2915298043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562777a15ff0 0x562777a2ab00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2915298043 >> 172.21.15.90:0/2915298043 conn(0x562777a13e20 msgr2=0x562777a1e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:28.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fc9e7400640 0 client.16008 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:11:28.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2915298043 shutdown_connections 2024-09-10T22:11:28.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fca0f3e9740 1 -- 172.21.15.90:0/2915298043 wait complete. 2024-09-10T22:11:28.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.735+0000 7fca0f3e9740 20 client.16008 trim_cache size 0 max 16384 2024-09-10T22:11:28.738 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:11:28.738 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 35, 2024-09-10T22:11:28.738 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:11:28.738 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 24, 2024-09-10T22:11:28.738 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "11553012": { 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 24, 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 146.8, 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:11:28.739 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:11:28.740 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:11:28.740 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:28.740 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:28.740 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:28.740 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:28.740 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:11:28.740 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:11:28.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 -- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc9ec0b54d0 msgr2=0x7fc9ec0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc9ec0b54d0 0x7fc9ec0b7990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x562777a19170 tx=0x7fc9f800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 -- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410a9b0 msgr2=0x7fca0419b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410a9b0 0x7fca0419b810 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc9f400d950 tx=0x7fc9f400dd50 comp rx=0 tx=0).stop 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 -- 172.21.15.90:0/3647587585 shutdown_connections 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc9ec0b54d0 0x7fc9ec0b7990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca0410b330 0x7fca041a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca0410a9b0 0x7fca0419b810 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 --2- 172.21.15.90:0/3647587585 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca04104fa0 0x7fca0419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 -- 172.21.15.90:0/3647587585 >> 172.21.15.90:0/3647587585 conn(0x7fca04100a50 msgr2=0x7fca04102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 -- 172.21.15.90:0/3647587585 shutdown_connections 2024-09-10T22:11:28.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:11:28.736+0000 7fca0c800640 1 -- 172.21.15.90:0/3647587585 wait complete. 2024-09-10T22:11:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:28 smithi090 ceph-mon[30350]: pgmap v649: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-10T22:11:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:28 smithi154 ceph-mon[34495]: pgmap v649: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-10T22:11:29.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:28 smithi171 ceph-mon[34138]: pgmap v649: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-10T22:11:29.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89533 1.24 MB/sec execute 261 sec latency 20754.029 ms 2024-09-10T22:11:29.744 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '11553012', total seconds elapsed: 31.2 2024-09-10T22:11:30.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89882 1.24 MB/sec execute 262 sec latency 20.519 ms 2024-09-10T22:11:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:30 smithi154 ceph-mon[34495]: pgmap v650: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-09-10T22:11:31.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:30 smithi171 ceph-mon[34138]: pgmap v650: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-09-10T22:11:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:30 smithi090 ceph-mon[30350]: pgmap v650: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 56 KiB/s wr, 20 op/s 2024-09-10T22:11:31.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 90233 1.25 MB/sec execute 263 sec latency 23.413 ms 2024-09-10T22:11:32.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 90587 1.25 MB/sec execute 264 sec latency 25.156 ms 2024-09-10T22:11:33.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:32 smithi090 ceph-mon[30350]: pgmap v651: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-09-10T22:11:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:32 smithi154 ceph-mon[34495]: pgmap v651: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-09-10T22:11:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:32 smithi171 ceph-mon[34138]: pgmap v651: 129 pgs: 129 active+clean; 106 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 55 KiB/s wr, 18 op/s 2024-09-10T22:11:33.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 91024 1.25 MB/sec execute 265 sec latency 23.155 ms 2024-09-10T22:11:34.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 91303 1.25 MB/sec execute 266 sec latency 20.047 ms 2024-09-10T22:11:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:34 smithi154 ceph-mon[34495]: pgmap v652: 129 pgs: 129 active+clean; 106 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 139 KiB/s wr, 27 op/s 2024-09-10T22:11:35.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:34 smithi171 ceph-mon[34138]: pgmap v652: 129 pgs: 129 active+clean; 106 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 139 KiB/s wr, 27 op/s 2024-09-10T22:11:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:34 smithi090 ceph-mon[30350]: pgmap v652: 129 pgs: 129 active+clean; 106 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 139 KiB/s wr, 27 op/s 2024-09-10T22:11:35.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 91894 1.26 MB/sec execute 267 sec latency 18.240 ms 2024-09-10T22:11:36.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 92082 1.26 MB/sec execute 268 sec latency 22.635 ms 2024-09-10T22:11:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:36 smithi154 ceph-mon[34495]: pgmap v653: 129 pgs: 129 active+clean; 107 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 377 KiB/s wr, 51 op/s 2024-09-10T22:11:37.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:36 smithi171 ceph-mon[34138]: pgmap v653: 129 pgs: 129 active+clean; 107 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 377 KiB/s wr, 51 op/s 2024-09-10T22:11:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:36 smithi090 ceph-mon[30350]: pgmap v653: 129 pgs: 129 active+clean; 107 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 377 KiB/s wr, 51 op/s 2024-09-10T22:11:37.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 92489 1.27 MB/sec execute 269 sec latency 21.052 ms 2024-09-10T22:11:38.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 92659 1.27 MB/sec execute 270 sec latency 19.109 ms 2024-09-10T22:11:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:38 smithi154 ceph-mon[34495]: pgmap v654: 129 pgs: 129 active+clean; 108 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 539 KiB/s wr, 49 op/s 2024-09-10T22:11:39.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:38 smithi171 ceph-mon[34138]: pgmap v654: 129 pgs: 129 active+clean; 108 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 539 KiB/s wr, 49 op/s 2024-09-10T22:11:39.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:38 smithi090 ceph-mon[30350]: pgmap v654: 129 pgs: 129 active+clean; 108 MiB data, 1000 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 539 KiB/s wr, 49 op/s 2024-09-10T22:11:39.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 92845 1.27 MB/sec execute 271 sec latency 19.290 ms 2024-09-10T22:11:40.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 93177 1.26 MB/sec execute 272 sec latency 26.477 ms 2024-09-10T22:11:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:40 smithi154 ceph-mon[34495]: pgmap v655: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1000 KiB/s wr, 89 op/s 2024-09-10T22:11:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:40 smithi171 ceph-mon[34138]: pgmap v655: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1000 KiB/s wr, 89 op/s 2024-09-10T22:11:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:40 smithi090 ceph-mon[30350]: pgmap v655: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1000 KiB/s wr, 89 op/s 2024-09-10T22:11:41.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 93508 1.26 MB/sec execute 273 sec latency 19.073 ms 2024-09-10T22:11:42.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 93878 1.27 MB/sec execute 274 sec latency 24.959 ms 2024-09-10T22:11:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:42 smithi090 ceph-mon[30350]: pgmap v656: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 972 KiB/s wr, 85 op/s 2024-09-10T22:11:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:42 smithi154 ceph-mon[34495]: pgmap v656: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 972 KiB/s wr, 85 op/s 2024-09-10T22:11:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:42 smithi171 ceph-mon[34138]: pgmap v656: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 972 KiB/s wr, 85 op/s 2024-09-10T22:11:43.477 INFO:tasks.workunit.client.0.smithi090.stdout: 1 94251 1.27 MB/sec execute 275 sec latency 24.058 ms 2024-09-10T22:11:44.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:43 smithi171 ceph-mon[34138]: pgmap v657: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.0 MiB/s wr, 96 op/s 2024-09-10T22:11:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:43 smithi090 ceph-mon[30350]: pgmap v657: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.0 MiB/s wr, 96 op/s 2024-09-10T22:11:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:43 smithi154 ceph-mon[34495]: pgmap v657: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.0 MiB/s wr, 96 op/s 2024-09-10T22:11:44.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 94643 1.27 MB/sec execute 276 sec latency 23.355 ms 2024-09-10T22:11:45.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 95028 1.28 MB/sec execute 277 sec latency 20.668 ms 2024-09-10T22:11:46.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 95554 1.29 MB/sec execute 278 sec latency 19.208 ms 2024-09-10T22:11:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:46 smithi090 ceph-mon[30350]: pgmap v658: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-10T22:11:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:46 smithi154 ceph-mon[34495]: pgmap v658: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-10T22:11:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:46 smithi171 ceph-mon[34138]: pgmap v658: 129 pgs: 129 active+clean; 111 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 1.2 MiB/s wr, 116 op/s 2024-09-10T22:11:47.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 95790 1.29 MB/sec execute 279 sec latency 23.434 ms 2024-09-10T22:11:48.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 96078 1.29 MB/sec execute 280 sec latency 19.933 ms 2024-09-10T22:11:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:48 smithi090 ceph-mon[30350]: pgmap v659: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-10T22:11:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:48 smithi154 ceph-mon[34495]: pgmap v659: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-10T22:11:49.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:48 smithi171 ceph-mon[34138]: pgmap v659: 129 pgs: 129 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 359 KiB/s rd, 1.1 MiB/s wr, 91 op/s 2024-09-10T22:11:49.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 96249 1.29 MB/sec execute 281 sec latency 20.321 ms 2024-09-10T22:11:50.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 96470 1.28 MB/sec execute 282 sec latency 23.259 ms 2024-09-10T22:11:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:50 smithi090 ceph-mon[30350]: pgmap v660: 129 pgs: 129 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-10T22:11:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:50 smithi154 ceph-mon[34495]: pgmap v660: 129 pgs: 129 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-10T22:11:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:50 smithi171 ceph-mon[34138]: pgmap v660: 129 pgs: 129 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-10T22:11:51.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 96820 1.28 MB/sec execute 283 sec latency 16.153 ms 2024-09-10T22:11:52.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 97220 1.29 MB/sec execute 284 sec latency 20.975 ms 2024-09-10T22:11:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:52 smithi090 ceph-mon[30350]: pgmap v661: 129 pgs: 129 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 925 KiB/s wr, 92 op/s 2024-09-10T22:11:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:52 smithi154 ceph-mon[34495]: pgmap v661: 129 pgs: 129 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 925 KiB/s wr, 92 op/s 2024-09-10T22:11:53.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:52 smithi171 ceph-mon[34138]: pgmap v661: 129 pgs: 129 active+clean; 115 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 925 KiB/s wr, 92 op/s 2024-09-10T22:11:53.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 97520 1.29 MB/sec execute 285 sec latency 26.601 ms 2024-09-10T22:11:54.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 97912 1.29 MB/sec execute 286 sec latency 26.677 ms 2024-09-10T22:11:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:54 smithi090 ceph-mon[30350]: pgmap v662: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-09-10T22:11:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:11:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:11:55.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:54 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:11:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:54 smithi154 ceph-mon[34495]: pgmap v662: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-09-10T22:11:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:11:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:11:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:54 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:11:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:54 smithi171 ceph-mon[34138]: pgmap v662: 129 pgs: 129 active+clean; 114 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-09-10T22:11:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:11:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:11:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:54 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:11:55.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 98256 1.29 MB/sec execute 287 sec latency 22.104 ms 2024-09-10T22:11:56.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:11:56.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:11:56.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:11:56.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:11:56.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:11:56.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:11:56.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:11:56.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:11:56.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:11:56.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 98733 1.30 MB/sec execute 288 sec latency 21.190 ms 2024-09-10T22:11:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:56 smithi090 ceph-mon[30350]: pgmap v663: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-10T22:11:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:56 smithi154 ceph-mon[34495]: pgmap v663: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-10T22:11:57.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:56 smithi171 ceph-mon[34138]: pgmap v663: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-09-10T22:11:57.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 99130 1.30 MB/sec execute 289 sec latency 17.710 ms 2024-09-10T22:11:58.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 99475 1.31 MB/sec execute 290 sec latency 20.464 ms 2024-09-10T22:11:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:11:58 smithi090 ceph-mon[30350]: pgmap v664: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 982 KiB/s wr, 94 op/s 2024-09-10T22:11:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:11:58 smithi154 ceph-mon[34495]: pgmap v664: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 982 KiB/s wr, 94 op/s 2024-09-10T22:11:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:11:58 smithi171 ceph-mon[34138]: pgmap v664: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 357 KiB/s rd, 982 KiB/s wr, 94 op/s 2024-09-10T22:11:59.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 99689 1.31 MB/sec execute 291 sec latency 22.559 ms 2024-09-10T22:12:00.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 99872 1.31 MB/sec execute 292 sec latency 17.557 ms 2024-09-10T22:12:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:00 smithi090 ceph-mon[30350]: pgmap v665: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-10T22:12:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:00 smithi154 ceph-mon[34495]: pgmap v665: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-10T22:12:01.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:00 smithi171 ceph-mon[34138]: pgmap v665: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 1.3 MiB/s wr, 125 op/s 2024-09-10T22:12:01.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 100159 1.30 MB/sec execute 293 sec latency 19.554 ms 2024-09-10T22:12:02.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 100504 1.30 MB/sec execute 294 sec latency 22.924 ms 2024-09-10T22:12:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:02 smithi090 ceph-mon[30350]: pgmap v666: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 961 KiB/s wr, 91 op/s 2024-09-10T22:12:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:02 smithi154 ceph-mon[34495]: pgmap v666: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 961 KiB/s wr, 91 op/s 2024-09-10T22:12:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:02 smithi171 ceph-mon[34138]: pgmap v666: 129 pgs: 129 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 961 KiB/s wr, 91 op/s 2024-09-10T22:12:03.478 INFO:tasks.workunit.client.0.smithi090.stdout: 1 100858 1.31 MB/sec execute 295 sec latency 23.168 ms 2024-09-10T22:12:04.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 101193 1.31 MB/sec execute 296 sec latency 24.257 ms 2024-09-10T22:12:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:04 smithi090 ceph-mon[30350]: pgmap v667: 129 pgs: 129 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-09-10T22:12:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:04 smithi090 ceph-mon[30350]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T22:12:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:04 smithi154 ceph-mon[34495]: pgmap v667: 129 pgs: 129 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-09-10T22:12:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:04 smithi154 ceph-mon[34495]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T22:12:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:04 smithi171 ceph-mon[34138]: pgmap v667: 129 pgs: 129 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-09-10T22:12:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:04 smithi171 ceph-mon[34138]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T22:12:05.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 101567 1.30 MB/sec execute 297 sec latency 25.971 ms 2024-09-10T22:12:06.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 101914 1.31 MB/sec execute 298 sec latency 19.478 ms 2024-09-10T22:12:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:06 smithi154 ceph-mon[34495]: pgmap v669: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-10T22:12:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:06 smithi171 ceph-mon[34138]: pgmap v669: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-10T22:12:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:06 smithi090 ceph-mon[30350]: pgmap v669: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-09-10T22:12:07.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 102381 1.32 MB/sec execute 299 sec latency 18.493 ms 2024-09-10T22:12:08.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 102709 1.32 MB/sec execute 300 sec latency 21.696 ms 2024-09-10T22:12:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:08 smithi154 ceph-mon[34495]: pgmap v670: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-10T22:12:09.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:08 smithi171 ceph-mon[34138]: pgmap v670: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-10T22:12:09.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:08 smithi090 ceph-mon[30350]: pgmap v670: 129 pgs: 129 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-09-10T22:12:09.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 103122 1.33 MB/sec execute 301 sec latency 21.014 ms 2024-09-10T22:12:10.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 103296 1.33 MB/sec execute 302 sec latency 17.859 ms 2024-09-10T22:12:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:10 smithi154 ceph-mon[34495]: pgmap v671: 129 pgs: 129 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-09-10T22:12:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:10 smithi154 ceph-mon[34495]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T22:12:11.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:10 smithi171 ceph-mon[34138]: pgmap v671: 129 pgs: 129 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-09-10T22:12:11.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:10 smithi171 ceph-mon[34138]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T22:12:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:10 smithi090 ceph-mon[30350]: pgmap v671: 129 pgs: 129 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-09-10T22:12:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:10 smithi090 ceph-mon[30350]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T22:12:11.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 103489 1.32 MB/sec execute 303 sec latency 20.244 ms 2024-09-10T22:12:12.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 103823 1.32 MB/sec execute 304 sec latency 22.886 ms 2024-09-10T22:12:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:12 smithi154 ceph-mon[34495]: pgmap v673: 129 pgs: 129 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1005 KiB/s wr, 89 op/s 2024-09-10T22:12:13.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:12 smithi171 ceph-mon[34138]: pgmap v673: 129 pgs: 129 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1005 KiB/s wr, 89 op/s 2024-09-10T22:12:13.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:12 smithi090 ceph-mon[30350]: pgmap v673: 129 pgs: 129 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1005 KiB/s wr, 89 op/s 2024-09-10T22:12:13.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 104164 1.32 MB/sec execute 305 sec latency 18.796 ms 2024-09-10T22:12:14.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 104526 1.32 MB/sec execute 306 sec latency 23.675 ms 2024-09-10T22:12:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:14 smithi090 ceph-mon[30350]: pgmap v674: 129 pgs: 129 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-10T22:12:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:14 smithi154 ceph-mon[34495]: pgmap v674: 129 pgs: 129 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-10T22:12:15.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:14 smithi171 ceph-mon[34138]: pgmap v674: 129 pgs: 129 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1.2 MiB/s wr, 107 op/s 2024-09-10T22:12:15.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 104914 1.32 MB/sec execute 307 sec latency 25.611 ms 2024-09-10T22:12:16.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 105300 1.32 MB/sec execute 308 sec latency 24.037 ms 2024-09-10T22:12:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:16 smithi090 ceph-mon[30350]: pgmap v675: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-10T22:12:16.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:16 smithi154 ceph-mon[34495]: pgmap v675: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-10T22:12:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:16 smithi171 ceph-mon[34138]: pgmap v675: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-10T22:12:17.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 105683 1.33 MB/sec execute 309 sec latency 20.132 ms 2024-09-10T22:12:18.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 106200 1.34 MB/sec execute 310 sec latency 20.030 ms 2024-09-10T22:12:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:18 smithi090 ceph-mon[30350]: pgmap v676: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-10T22:12:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:18 smithi154 ceph-mon[34495]: pgmap v676: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-10T22:12:19.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:18 smithi171 ceph-mon[34138]: pgmap v676: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-09-10T22:12:19.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 106471 1.34 MB/sec execute 311 sec latency 18.681 ms 2024-09-10T22:12:19.516 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.4 2024-09-10T22:12:19.517 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:12:19.942 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:12:20.480 INFO:tasks.workunit.client.0.smithi090.stdout: 1 106742 1.34 MB/sec execute 312 sec latency 16.867 ms 2024-09-10T22:12:20.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.709+0000 7efe44e00640 1 -- 172.21.15.90:0/2126300081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe4010b180 msgr2=0x7efe40111980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:20.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.709+0000 7efe44e00640 1 --2- 172.21.15.90:0/2126300081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe4010b180 0x7efe40111980 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efe38009ca0 tx=0x7efe3802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.709+0000 7efe44e00640 1 -- 172.21.15.90:0/2126300081 shutdown_connections 2024-09-10T22:12:20.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.709+0000 7efe44e00640 1 --2- 172.21.15.90:0/2126300081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe4010b180 0x7efe40111980 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.709+0000 7efe44e00640 1 --2- 172.21.15.90:0/2126300081 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4010a800 0x7efe4010ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.709+0000 7efe44e00640 1 --2- 172.21.15.90:0/2126300081 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe40104df0 0x7efe401051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.709+0000 7efe44e00640 1 -- 172.21.15.90:0/2126300081 >> 172.21.15.90:0/2126300081 conn(0x7efe40100840 msgr2=0x7efe40102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.709+0000 7efe44e00640 1 -- 172.21.15.90:0/2126300081 shutdown_connections 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.709+0000 7efe44e00640 1 -- 172.21.15.90:0/2126300081 wait complete. 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe44e00640 1 Processor -- start 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe44e00640 1 -- start start 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe44e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe40104df0 0x7efe4019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe44e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4010a800 0x7efe4019b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe44e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4010b180 0x7efe401a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe44e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe40114d40 con 0x7efe40104df0 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe44e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7efe40114bc0 con 0x7efe4010a800 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe44e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7efe40114ec0 con 0x7efe4010b180 2024-09-10T22:12:20.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe3f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe40104df0 0x7efe4019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe3f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe40104df0 0x7efe4019b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56076/0 (socket says 172.21.15.90:56076) 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.711+0000 7efe3f400640 1 -- 172.21.15.90:0/154904626 learned_addr learned my addr 172.21.15.90:0/154904626 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.712+0000 7efe3ea00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4010a800 0x7efe4019b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.712+0000 7efe3fe00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4010b180 0x7efe401a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.712+0000 7efe3f400640 1 -- 172.21.15.90:0/154904626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4010b180 msgr2=0x7efe401a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.712+0000 7efe3f400640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4010b180 0x7efe401a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.712+0000 7efe3f400640 1 -- 172.21.15.90:0/154904626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4010a800 msgr2=0x7efe4019b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.712+0000 7efe3f400640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4010a800 0x7efe4019b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.712+0000 7efe3f400640 1 -- 172.21.15.90:0/154904626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe401a1160 con 0x7efe40104df0 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.712+0000 7efe3f400640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe40104df0 0x7efe4019b0b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efe3000e9f0 tx=0x7efe3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.712+0000 7efe37e00640 1 -- 172.21.15.90:0/154904626 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3000cd80 con 0x7efe40104df0 2024-09-10T22:12:20.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.713+0000 7efe37e00640 1 -- 172.21.15.90:0/154904626 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe300054b0 con 0x7efe40104df0 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.713+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe401a1450 con 0x7efe40104df0 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.713+0000 7efe37e00640 1 -- 172.21.15.90:0/154904626 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30010430 con 0x7efe40104df0 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.713+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe401a7b10 con 0x7efe40104df0 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.714+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe401051d0 con 0x7efe40104df0 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.717+0000 7efe37e00640 1 -- 172.21.15.90:0/154904626 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7efe30005000 con 0x7efe40104df0 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.719+0000 7efe37e00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe240b5580 0x7efe240b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.720+0000 7efe37e00640 1 -- 172.21.15.90:0/154904626 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9490+0+0 (secure 0 0 0) 0x7efe300e6940 con 0x7efe40104df0 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.720+0000 7efe3ea00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe240b5580 0x7efe240b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.720+0000 7efe3ea00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe240b5580 0x7efe240b7a40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efe40102f00 tx=0x7efe2c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:20.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.721+0000 7efe37e00640 1 -- 172.21.15.90:0/154904626 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efe30056220 con 0x7efe40104df0 2024-09-10T22:12:20.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.962+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe401a7f70 con 0x7efe40104df0 2024-09-10T22:12:20.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:20 smithi090 ceph-mon[30350]: pgmap v677: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-10T22:12:20.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.963+0000 7efe37e00640 1 -- 172.21.15.90:0/154904626 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11986 (secure 0 0 0) 0x7efe300ad090 con 0x7efe40104df0 2024-09-10T22:12:20.964 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:12:20.965 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"btime":"2024-09-10T22:08:03:817761+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:08:03.499417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:12:20.965 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-09-10T22:12:20.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe240b5580 msgr2=0x7efe240b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:20.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe240b5580 0x7efe240b7a40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efe40102f00 tx=0x7efe2c005f90 comp rx=0 tx=0).stop 2024-09-10T22:12:20.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe40104df0 msgr2=0x7efe4019b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:20.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe40104df0 0x7efe4019b0b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efe3000e9f0 tx=0x7efe3000edf0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 shutdown_connections 2024-09-10T22:12:20.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7efe240b5580 0x7efe240b7a40 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7efe4010b180 0x7efe401a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7efe4010a800 0x7efe4019b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 --2- 172.21.15.90:0/154904626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe40104df0 0x7efe4019b0b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:20.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 >> 172.21.15.90:0/154904626 conn(0x7efe40100840 msgr2=0x7efe40103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:20.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 shutdown_connections 2024-09-10T22:12:20.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:20.967+0000 7efe44e00640 1 -- 172.21.15.90:0/154904626 wait complete. 2024-09-10T22:12:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:20 smithi154 ceph-mon[34495]: pgmap v677: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-10T22:12:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:20 smithi171 ceph-mon[34138]: pgmap v677: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-10T22:12:21.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 106917 1.34 MB/sec execute 313 sec latency 42.071 ms 2024-09-10T22:12:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:21 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/154904626' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:12:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:21 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/154904626' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:12:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:21 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/154904626' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:12:22.479 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107159 1.34 MB/sec execute 314 sec latency 24.822 ms 2024-09-10T22:12:22.896 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 148.4 --await-for 45.0' 2024-09-10T22:12:22.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 148.4 --await-for 45.0 2024-09-10T22:12:23.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:22 smithi090 ceph-mon[30350]: pgmap v678: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-09-10T22:12:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:22 smithi154 ceph-mon[34495]: pgmap v678: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-09-10T22:12:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:22 smithi171 ceph-mon[34138]: pgmap v678: 129 pgs: 129 active+clean; 129 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.0 MiB/s wr, 99 op/s 2024-09-10T22:12:23.339 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:12:23.480 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107518 1.34 MB/sec execute 315 sec latency 17.366 ms 2024-09-10T22:12:24.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.053+0000 7fde2b000640 1 -- 172.21.15.90:0/2389658775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 msgr2=0x7fde240758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:24.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.053+0000 7fde2b000640 1 --2- 172.21.15.90:0/2389658775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 0x7fde240758a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fde18009900 tx=0x7fde1802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.054+0000 7fde2b000640 1 -- 172.21.15.90:0/2389658775 shutdown_connections 2024-09-10T22:12:24.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.054+0000 7fde2b000640 1 --2- 172.21.15.90:0/2389658775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde2410f220 0x7fde24111610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.054+0000 7fde2b000640 1 --2- 172.21.15.90:0/2389658775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde24075de0 0x7fde2410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.054+0000 7fde2b000640 1 --2- 172.21.15.90:0/2389658775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 0x7fde240758a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.054+0000 7fde2b000640 1 -- 172.21.15.90:0/2389658775 >> 172.21.15.90:0/2389658775 conn(0x7fde240fe660 msgr2=0x7fde24100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:24.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.054+0000 7fde2b000640 1 -- 172.21.15.90:0/2389658775 shutdown_connections 2024-09-10T22:12:24.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.054+0000 7fde2b000640 1 -- 172.21.15.90:0/2389658775 wait complete. 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.055+0000 7fde2b000640 1 Processor -- start 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.055+0000 7fde2b000640 1 -- start start 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.055+0000 7fde2b000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 0x7fde2419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.055+0000 7fde2b000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde24075de0 0x7fde2419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.055+0000 7fde2b000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde2410f220 0x7fde241a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.055+0000 7fde2b000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde24114d00 con 0x7fde240754c0 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.055+0000 7fde2b000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde24114b80 con 0x7fde24075de0 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.055+0000 7fde2b000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde24114e80 con 0x7fde2410f220 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 0x7fde2419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 0x7fde2419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56086/0 (socket says 172.21.15.90:56086) 2024-09-10T22:12:24.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29c00640 1 -- 172.21.15.90:0/89311489 learned_addr learned my addr 172.21.15.90:0/89311489 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:24.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29200640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde24075de0 0x7fde2419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde2a600640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde2410f220 0x7fde241a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29200640 1 -- 172.21.15.90:0/89311489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde2410f220 msgr2=0x7fde241a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:24.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29200640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde2410f220 0x7fde241a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29200640 1 -- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 msgr2=0x7fde2419b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29200640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 0x7fde2419b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29200640 1 -- 172.21.15.90:0/89311489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde241a1240 con 0x7fde24075de0 2024-09-10T22:12:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.056+0000 7fde29c00640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 0x7fde2419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:12:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.057+0000 7fde29200640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde24075de0 0x7fde2419b6d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde1000e9f0 tx=0x7fde1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.057+0000 7fde22000640 1 -- 172.21.15.90:0/89311489 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1000ccc0 con 0x7fde24075de0 2024-09-10T22:12:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.057+0000 7fde2b000640 1 -- 172.21.15.90:0/89311489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde241a1530 con 0x7fde24075de0 2024-09-10T22:12:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.057+0000 7fde22000640 1 -- 172.21.15.90:0/89311489 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde100108c0 con 0x7fde24075de0 2024-09-10T22:12:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.057+0000 7fde22000640 1 -- 172.21.15.90:0/89311489 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde100187c0 con 0x7fde24075de0 2024-09-10T22:12:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.057+0000 7fde2b000640 1 -- 172.21.15.90:0/89311489 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde241a7dc0 con 0x7fde24075de0 2024-09-10T22:12:24.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.061+0000 7fde2efc3740 1 Processor -- start 2024-09-10T22:12:24.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.061+0000 7fde2efc3740 1 -- start start 2024-09-10T22:12:24.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde22000640 1 -- 172.21.15.90:0/89311489 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fde10010430 con 0x7fde24075de0 2024-09-10T22:12:24.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37ded6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37dfbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2efc3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556a37d62ca0 con 0x556a37df9ba0 2024-09-10T22:12:24.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2efc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556a37d520d0 con 0x556a37ded2f0 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2efc3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556a37d55ee0 con 0x556a37dedc10 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2a600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37ded6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde29200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37dfbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2a600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37ded6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:53924/0 (socket says 172.21.15.90:53924) 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2a600640 1 -- 172.21.15.90:0/237341302 learned_addr learned my addr 172.21.15.90:0/237341302 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde29c00640 1 --2- 172.21.15.90:0/237341302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2a600640 1 -- 172.21.15.90:0/237341302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 msgr2=0x556a37df9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2a600640 1 --2- 172.21.15.90:0/237341302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2a600640 1 -- 172.21.15.90:0/237341302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 msgr2=0x556a37dfbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2a600640 1 --2- 172.21.15.90:0/237341302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37dfbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.062+0000 7fde2a600640 1 -- 172.21.15.90:0/237341302 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a37d04400 con 0x556a37ded2f0 2024-09-10T22:12:24.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.063+0000 7fde2a600640 1 --2- 172.21.15.90:0/237341302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37ded6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009f50 tx=0x7fde1c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80683e60764a308 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:24.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.063+0000 7fde16a00640 1 -- 172.21.15.90:0/237341302 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c004d20 con 0x556a37ded2f0 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.063+0000 7fde16a00640 1 -- 172.21.15.90:0/237341302 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde1c004050 con 0x556a37ded2f0 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.063+0000 7fde16a00640 1 -- 172.21.15.90:0/237341302 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c005260 con 0x556a37ded2f0 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde22000640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde0c0b5580 0x7fde0c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde2efc3740 1 -- 172.21.15.90:0/237341302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 msgr2=0x556a37ded6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde2efc3740 1 --2- 172.21.15.90:0/237341302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37ded6d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009f50 tx=0x7fde1c0118d0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde22000640 1 -- 172.21.15.90:0/89311489 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9490+0+0 (secure 0 0 0) 0x7fde100e6660 con 0x7fde24075de0 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde2efc3740 1 -- 172.21.15.90:0/237341302 shutdown_connections 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde2efc3740 1 --2- 172.21.15.90:0/237341302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37dfbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde2efc3740 1 --2- 172.21.15.90:0/237341302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde29c00640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde0c0b5580 0x7fde0c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde2efc3740 1 --2- 172.21.15.90:0/237341302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37ded6d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde2efc3740 1 -- 172.21.15.90:0/237341302 >> 172.21.15.90:0/237341302 conn(0x556a37deb120 msgr2=0x556a37dec2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.064+0000 7fde2efc3740 1 -- 172.21.15.90:0/237341302 shutdown_connections 2024-09-10T22:12:24.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.065+0000 7fde2efc3740 1 -- 172.21.15.90:0/237341302 wait complete. 2024-09-10T22:12:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.065+0000 7fde2efc3740 1 -- start start 2024-09-10T22:12:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.065+0000 7fde29c00640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde0c0b5580 0x7fde0c0b7a40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fde18005ec0 tx=0x7fde180036d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.065+0000 7fde2efc3740 1 Processor -- start 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37df94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37df6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde2efc3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556a37bb37a0 con 0x556a37df9ba0 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde2efc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x556a37cf7a10 con 0x556a37ded2f0 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde2efc3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x556a37dfc820 con 0x556a37dedc10 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde2efc3740 10 client.0 _ll_register_callbacks cb 0x556a37d07be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde29c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37df94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde29c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37df94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:53926/0 (socket says 172.21.15.90:53926) 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde29c00640 1 -- 172.21.15.90:0/271680672 learned_addr learned my addr 172.21.15.90:0/271680672 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:24.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.066+0000 7fde2a600640 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37df6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.067+0000 7fde29200640 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.067+0000 7fde29200640 1 -- 172.21.15.90:0/271680672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 msgr2=0x556a37df94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.067+0000 7fde29200640 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37df94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.067+0000 7fde29200640 1 -- 172.21.15.90:0/271680672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 msgr2=0x556a37df6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.067+0000 7fde29200640 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37df6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.067+0000 7fde29200640 1 -- 172.21.15.90:0/271680672 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a37abfb90 con 0x556a37dedc10 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.067+0000 7fde2a600640 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37df6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.068+0000 7fde29200640 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df68f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde100e3980 tx=0x7fde100ac530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa5ecef6e387137 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.068+0000 7fde0b400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.068+0000 7fde0b400640 1 -- 172.21.15.90:0/271680672 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde100a6e60 con 0x556a37dedc10 2024-09-10T22:12:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.068+0000 7fde2efc3740 1 -- 172.21.15.90:0/271680672 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde1c004050 con 0x556a37dedc10 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.068+0000 7fde0b400640 1 -- 172.21.15.90:0/271680672 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9490+0+0 (secure 0 0 0) 0x7fde100e63d0 con 0x556a37dedc10 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.068+0000 7fde0b400640 1 -- 172.21.15.90:0/271680672 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fde100e3e10 con 0x556a37dedc10 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.068+0000 7fde2efc3740 10 client.25651 fetch_fsmap learned FSMap version 35 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.068+0000 7fde2efc3740 1 -- 172.21.15.90:0/271680672 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556a37e081f0 con 0x556a37dedc10 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.068+0000 7fde0b400640 1 -- 172.21.15.90:0/271680672 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde1c004050 con 0x556a37dedc10 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.069+0000 7fde0b400640 1 -- 172.21.15.90:0/271680672 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fde100e8b90 con 0x556a37dedc10 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.069+0000 7fde2efc3740 10 client.25651 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.069+0000 7fde2efc3740 10 client.25651 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.069+0000 7fde2efc3740 20 client.25651 populate_metadata read hostname 'smithi090' 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.069+0000 7fde2efc3740 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556a37e09600 0x556a37e0b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.069+0000 7fde2efc3740 4 client.25651 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:12:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.069+0000 7fde2efc3740 1 -- 172.21.15.90:0/271680672 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a37d64bf0 con 0x556a37e09600 2024-09-10T22:12:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.070+0000 7fde29c00640 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556a37e09600 0x556a37e0b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.070+0000 7fde29c00640 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556a37e09600 0x556a37e0b9f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bd2415542371869 server_cookie=78ee2dbf5643fc25 in_seq=0 out_seq=0 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.070+0000 7fde0b400640 10 client.25651 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.071+0000 7fde0b400640 1 -- 172.21.15.90:0/271680672 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556a37d64bf0 con 0x556a37e09600 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.071+0000 7fde0b400640 10 client.25651 handle_command_reply: tid=0 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.071+0000 7fde2efc3740 1 client.25651 shutdown 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 -- 172.21.15.90:0/271680672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 msgr2=0x556a37df68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df68f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fde100e3980 tx=0x7fde100ac530 comp rx=0 tx=0).stop 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 -- 172.21.15.90:0/271680672 shutdown_connections 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556a37e09600 0x556a37e0b9f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37df6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df68f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 --2- 172.21.15.90:0/271680672 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37ded2f0 0x556a37df94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde0b400640 0 client.25651 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:12:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 -- 172.21.15.90:0/271680672 >> 172.21.15.90:0/271680672 conn(0x556a37deb120 msgr2=0x556a37df0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:24.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 -- 172.21.15.90:0/271680672 shutdown_connections 2024-09-10T22:12:24.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 1 -- 172.21.15.90:0/271680672 wait complete. 2024-09-10T22:12:24.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.072+0000 7fde2efc3740 20 client.25651 trim_cache size 0 max 16384 2024-09-10T22:12:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.090+0000 7fde2efc3740 1 Processor -- start 2024-09-10T22:12:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.090+0000 7fde2efc3740 1 -- start start 2024-09-10T22:12:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37ded2f0 0x556a37e05a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37df9ba0 0x556a37dfaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde2efc3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b1de0 con 0x556a37ded2f0 2024-09-10T22:12:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde2efc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b3920 con 0x556a37df9ba0 2024-09-10T22:12:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde2efc3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b24b0 con 0x556a37dedc10 2024-09-10T22:12:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde29c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37ded2f0 0x556a37e05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde29c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37ded2f0 0x556a37e05a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56116/0 (socket says 172.21.15.90:56116) 2024-09-10T22:12:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde29c00640 1 -- 172.21.15.90:0/835509871 learned_addr learned my addr 172.21.15.90:0/835509871 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde29200640 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde2a600640 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37df9ba0 0x556a37dfaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.091+0000 7fde29c00640 1 -- 172.21.15.90:0/835509871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 msgr2=0x556a37df6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.092+0000 7fde29c00640 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.092+0000 7fde29c00640 1 -- 172.21.15.90:0/835509871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37df9ba0 msgr2=0x556a37dfaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.092+0000 7fde29c00640 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37df9ba0 0x556a37dfaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.092+0000 7fde29c00640 1 -- 172.21.15.90:0/835509871 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a37df66c0 con 0x556a37ded2f0 2024-09-10T22:12:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.092+0000 7fde29200640 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.092+0000 7fde29c00640 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37ded2f0 0x556a37e05a80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x556a37e0c260 tx=0x7fde1803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399053cb7e9a19ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.092+0000 7fde16000640 1 -- 172.21.15.90:0/835509871 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18035040 con 0x556a37ded2f0 2024-09-10T22:12:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.092+0000 7fde16000640 1 -- 172.21.15.90:0/835509871 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde180543f0 con 0x556a37ded2f0 2024-09-10T22:12:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.092+0000 7fde16000640 1 -- 172.21.15.90:0/835509871 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1804b690 con 0x556a37ded2f0 2024-09-10T22:12:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.093+0000 7fde2efc3740 1 -- 172.21.15.90:0/835509871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37ded2f0 msgr2=0x556a37e05a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.093+0000 7fde2efc3740 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37ded2f0 0x556a37e05a80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x556a37e0c260 tx=0x7fde1803f000 comp rx=0 tx=0).stop 2024-09-10T22:12:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.093+0000 7fde2efc3740 1 -- 172.21.15.90:0/835509871 shutdown_connections 2024-09-10T22:12:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.093+0000 7fde2efc3740 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37df9ba0 0x556a37dfaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.093+0000 7fde2efc3740 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37dedc10 0x556a37df6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.093+0000 7fde2efc3740 1 --2- 172.21.15.90:0/835509871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37ded2f0 0x556a37e05a80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.093+0000 7fde2efc3740 1 -- 172.21.15.90:0/835509871 >> 172.21.15.90:0/835509871 conn(0x556a37deb120 msgr2=0x556a37df8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.093+0000 7fde2efc3740 1 -- 172.21.15.90:0/835509871 shutdown_connections 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.093+0000 7fde2efc3740 1 -- 172.21.15.90:0/835509871 wait complete. 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.094+0000 7fde2efc3740 1 -- start start 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.094+0000 7fde2efc3740 1 Processor -- start 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.094+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37ded2f0 0x556a37df43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.094+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37dedc10 0x556a37e01ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde2efc3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37e020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde2efc3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b3250 con 0x556a37df9ba0 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde2efc3740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fde241b2b80 con 0x556a37dedc10 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde2efc3740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fde241afde0 con 0x556a37ded2f0 2024-09-10T22:12:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde2efc3740 10 client.0 _ll_register_callbacks cb 0x7fde1000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:12:24.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde29c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37ded2f0 0x556a37df43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde29c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37ded2f0 0x556a37df43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:42728/0 (socket says 172.21.15.90:42728) 2024-09-10T22:12:24.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde29c00640 1 -- 172.21.15.90:0/1017129334 learned_addr learned my addr 172.21.15.90:0/1017129334 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:24.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde29200640 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37dedc10 0x556a37e01ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde2a600640 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37e020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde29c00640 1 -- 172.21.15.90:0/1017129334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37dedc10 msgr2=0x556a37e01ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:24.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde29c00640 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37dedc10 0x556a37e01ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde29c00640 1 -- 172.21.15.90:0/1017129334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 msgr2=0x556a37e020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde29c00640 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37e020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.095+0000 7fde29c00640 1 -- 172.21.15.90:0/1017129334 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a37df2b10 con 0x556a37ded2f0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.096+0000 7fde29c00640 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37ded2f0 0x556a37df43d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fde18003bc0 tx=0x7fde18003bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8183443c1bdebdbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.096+0000 7fde0b400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.096+0000 7fde0b400640 1 -- 172.21.15.90:0/1017129334 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18064070 con 0x556a37ded2f0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.096+0000 7fde0b400640 1 -- 172.21.15.90:0/1017129334 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9490+0+0 (secure 0 0 0) 0x7fde1803dd30 con 0x556a37ded2f0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.096+0000 7fde2efc3740 1 -- 172.21.15.90:0/1017129334 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde180543f0 con 0x556a37ded2f0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.096+0000 7fde0b400640 1 -- 172.21.15.90:0/1017129334 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fde180545a0 con 0x556a37ded2f0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.096+0000 7fde2efc3740 10 client.25657 fetch_fsmap learned FSMap version 35 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.096+0000 7fde2efc3740 1 -- 172.21.15.90:0/1017129334 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556a37e77620 con 0x556a37ded2f0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.097+0000 7fde0b400640 1 -- 172.21.15.90:0/1017129334 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fde180543f0 con 0x556a37ded2f0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.097+0000 7fde0b400640 1 -- 172.21.15.90:0/1017129334 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7fde1804b9a0 con 0x556a37ded2f0 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.097+0000 7fde2efc3740 10 client.25657 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.097+0000 7fde2efc3740 10 client.25657 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.097+0000 7fde2efc3740 20 client.25657 populate_metadata read hostname 'smithi090' 2024-09-10T22:12:24.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.097+0000 7fde2efc3740 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556a37e77de0 0x556a37e7a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:24.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.097+0000 7fde2efc3740 4 client.25657 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.4, "await_for": 45.0}] 2024-09-10T22:12:24.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.097+0000 7fde2efc3740 1 -- 172.21.15.90:0/1017129334 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.4, "await_for": 45.0}) -- 0x556a37d606a0 con 0x556a37e77de0 2024-09-10T22:12:24.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.097+0000 7fde29200640 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556a37e77de0 0x556a37e7a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:24.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.098+0000 7fde29200640 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556a37e77de0 0x556a37e7a200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e880a9790bf4952 server_cookie=36386b5f0b416160 in_seq=0 out_seq=0 2024-09-10T22:12:24.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.098+0000 7fde0b400640 10 client.25657 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:12:24.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.162+0000 7fde0b400640 1 -- 172.21.15.90:0/1017129334 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556a37d606a0 con 0x556a37e77de0 2024-09-10T22:12:24.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.162+0000 7fde0b400640 10 client.25657 handle_command_reply: tid=0 2024-09-10T22:12:24.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.162+0000 7fde2efc3740 1 client.25657 shutdown 2024-09-10T22:12:24.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 -- 172.21.15.90:0/1017129334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37ded2f0 msgr2=0x556a37df43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:24.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37ded2f0 0x556a37df43d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fde18003bc0 tx=0x7fde18003bf0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 -- 172.21.15.90:0/1017129334 shutdown_connections 2024-09-10T22:12:24.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x556a37e77de0 0x556a37e7a200 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a37df9ba0 0x556a37e020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x556a37dedc10 0x556a37e01ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 --2- 172.21.15.90:0/1017129334 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x556a37ded2f0 0x556a37df43d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde0b400640 0 client.25657 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 -- 172.21.15.90:0/1017129334 >> 172.21.15.90:0/1017129334 conn(0x556a37deb120 msgr2=0x556a37df5170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 -- 172.21.15.90:0/1017129334 shutdown_connections 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 1 -- 172.21.15.90:0/1017129334 wait complete. 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.163+0000 7fde2efc3740 20 client.25657 trim_cache size 0 max 16384 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 35, 2024-09-10T22:12:24.165 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 26, 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "6a665910": { 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 26, 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 148.4, 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:12:24.166 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.164+0000 7fde2b000640 1 -- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde0c0b5580 msgr2=0x7fde0c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:24.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.164+0000 7fde2b000640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde0c0b5580 0x7fde0c0b7a40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fde18005ec0 tx=0x7fde180036d0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.164+0000 7fde2b000640 1 -- 172.21.15.90:0/89311489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde24075de0 msgr2=0x7fde2419b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:24.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.164+0000 7fde2b000640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde24075de0 0x7fde2419b6d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde1000e9f0 tx=0x7fde1000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.165+0000 7fde2b000640 1 -- 172.21.15.90:0/89311489 shutdown_connections 2024-09-10T22:12:24.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.165+0000 7fde2b000640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fde0c0b5580 0x7fde0c0b7a40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.165+0000 7fde2b000640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fde2410f220 0x7fde241a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.165+0000 7fde2b000640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fde24075de0 0x7fde2419b6d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.165+0000 7fde2b000640 1 --2- 172.21.15.90:0/89311489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde240754c0 0x7fde2419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:24.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.165+0000 7fde2b000640 1 -- 172.21.15.90:0/89311489 >> 172.21.15.90:0/89311489 conn(0x7fde240fe660 msgr2=0x7fde24111440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:24.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.167+0000 7fde2b000640 1 -- 172.21.15.90:0/89311489 shutdown_connections 2024-09-10T22:12:24.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:24.167+0000 7fde2b000640 1 -- 172.21.15.90:0/89311489 wait complete. 2024-09-10T22:12:24.480 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.33 MB/sec execute 316 sec latency 384.078 ms 2024-09-10T22:12:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:24 smithi090 ceph-mon[30350]: pgmap v679: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-09-10T22:12:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:24 smithi154 ceph-mon[34495]: pgmap v679: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-09-10T22:12:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:24 smithi171 ceph-mon[34138]: pgmap v679: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-09-10T22:12:25.480 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.33 MB/sec execute 317 sec latency 1384.141 ms 2024-09-10T22:12:25.513 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6a665910', quiesce took 6.0 seconds. Will release after: 22.4 2024-09-10T22:12:26.480 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.33 MB/sec execute 318 sec latency 2384.245 ms 2024-09-10T22:12:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:26 smithi090 ceph-mon[30350]: pgmap v680: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-10T22:12:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:26 smithi154 ceph-mon[34495]: pgmap v680: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-10T22:12:27.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:26 smithi171 ceph-mon[34138]: pgmap v680: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-09-10T22:12:27.480 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.32 MB/sec execute 319 sec latency 3384.386 ms 2024-09-10T22:12:28.480 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.32 MB/sec execute 320 sec latency 4384.537 ms 2024-09-10T22:12:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:28 smithi090 ceph-mon[30350]: pgmap v681: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-10T22:12:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:28 smithi154 ceph-mon[34495]: pgmap v681: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-10T22:12:29.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:28 smithi171 ceph-mon[34138]: pgmap v681: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-10T22:12:29.480 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.31 MB/sec execute 321 sec latency 5384.701 ms 2024-09-10T22:12:30.481 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.31 MB/sec execute 322 sec latency 6384.843 ms 2024-09-10T22:12:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:30 smithi154 ceph-mon[34495]: pgmap v682: 129 pgs: 129 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-09-10T22:12:31.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:30 smithi171 ceph-mon[34138]: pgmap v682: 129 pgs: 129 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-09-10T22:12:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:30 smithi090 ceph-mon[30350]: pgmap v682: 129 pgs: 129 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-09-10T22:12:31.481 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.30 MB/sec execute 323 sec latency 7384.985 ms 2024-09-10T22:12:32.481 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.30 MB/sec execute 324 sec latency 8385.120 ms 2024-09-10T22:12:33.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:32 smithi090 ceph-mon[30350]: pgmap v683: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 671 KiB/s wr, 71 op/s 2024-09-10T22:12:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:32 smithi154 ceph-mon[34495]: pgmap v683: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 671 KiB/s wr, 71 op/s 2024-09-10T22:12:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:32 smithi171 ceph-mon[34138]: pgmap v683: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 671 KiB/s wr, 71 op/s 2024-09-10T22:12:33.481 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.30 MB/sec execute 325 sec latency 9385.279 ms 2024-09-10T22:12:34.481 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.29 MB/sec execute 326 sec latency 10385.348 ms 2024-09-10T22:12:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:34 smithi154 ceph-mon[34495]: pgmap v684: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 608 KiB/s wr, 69 op/s 2024-09-10T22:12:35.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:34 smithi171 ceph-mon[34138]: pgmap v684: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 608 KiB/s wr, 69 op/s 2024-09-10T22:12:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:34 smithi090 ceph-mon[30350]: pgmap v684: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 608 KiB/s wr, 69 op/s 2024-09-10T22:12:35.481 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.29 MB/sec execute 327 sec latency 11385.459 ms 2024-09-10T22:12:36.481 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.28 MB/sec execute 328 sec latency 12385.612 ms 2024-09-10T22:12:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:36 smithi154 ceph-mon[34495]: pgmap v685: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 278 KiB/s wr, 43 op/s 2024-09-10T22:12:37.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:36 smithi171 ceph-mon[34138]: pgmap v685: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 278 KiB/s wr, 43 op/s 2024-09-10T22:12:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:36 smithi090 ceph-mon[30350]: pgmap v685: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 278 KiB/s wr, 43 op/s 2024-09-10T22:12:37.481 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.28 MB/sec execute 329 sec latency 13385.704 ms 2024-09-10T22:12:38.481 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.28 MB/sec execute 330 sec latency 14385.824 ms 2024-09-10T22:12:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:38 smithi154 ceph-mon[34495]: pgmap v686: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 103 KiB/s wr, 21 op/s 2024-09-10T22:12:39.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:38 smithi171 ceph-mon[34138]: pgmap v686: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 103 KiB/s wr, 21 op/s 2024-09-10T22:12:39.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:38 smithi090 ceph-mon[30350]: pgmap v686: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 103 KiB/s wr, 21 op/s 2024-09-10T22:12:39.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.27 MB/sec execute 331 sec latency 15385.949 ms 2024-09-10T22:12:40.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.27 MB/sec execute 332 sec latency 16386.096 ms 2024-09-10T22:12:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:40 smithi154 ceph-mon[34495]: pgmap v687: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-09-10T22:12:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:40 smithi171 ceph-mon[34138]: pgmap v687: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-09-10T22:12:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:40 smithi090 ceph-mon[30350]: pgmap v687: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-09-10T22:12:41.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.27 MB/sec execute 333 sec latency 17386.187 ms 2024-09-10T22:12:42.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.26 MB/sec execute 334 sec latency 18386.304 ms 2024-09-10T22:12:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:42 smithi154 ceph-mon[34495]: pgmap v688: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-10T22:12:43.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:42 smithi171 ceph-mon[34138]: pgmap v688: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-10T22:12:43.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:42 smithi090 ceph-mon[30350]: pgmap v688: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-10T22:12:43.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.26 MB/sec execute 335 sec latency 19386.402 ms 2024-09-10T22:12:44.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:43 smithi171 ceph-mon[34138]: pgmap v689: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-10T22:12:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:43 smithi090 ceph-mon[30350]: pgmap v689: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-10T22:12:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:43 smithi154 ceph-mon[34495]: pgmap v689: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-10T22:12:44.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.25 MB/sec execute 336 sec latency 20386.522 ms 2024-09-10T22:12:45.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.25 MB/sec execute 337 sec latency 21386.636 ms 2024-09-10T22:12:46.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.25 MB/sec execute 338 sec latency 22386.800 ms 2024-09-10T22:12:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:46 smithi090 ceph-mon[30350]: pgmap v690: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T22:12:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:46 smithi154 ceph-mon[34495]: pgmap v690: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T22:12:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:46 smithi171 ceph-mon[34138]: pgmap v690: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-09-10T22:12:47.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.24 MB/sec execute 339 sec latency 23386.882 ms 2024-09-10T22:12:47.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:12:48.299 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:12:48.482 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.24 MB/sec execute 340 sec latency 24386.956 ms 2024-09-10T22:12:48.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.942+0000 7f585fc00640 1 -- 172.21.15.90:0/4195744313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810b330 msgr2=0x7f5858111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.942+0000 7f585fc00640 1 --2- 172.21.15.90:0/4195744313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810b330 0x7f5858111b30 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5850009ca0 tx=0x7f585002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.942+0000 7f585fc00640 1 -- 172.21.15.90:0/4195744313 shutdown_connections 2024-09-10T22:12:48.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.942+0000 7f585fc00640 1 --2- 172.21.15.90:0/4195744313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810b330 0x7f5858111b30 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.942+0000 7f585fc00640 1 --2- 172.21.15.90:0/4195744313 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f585810a9b0 0x7f585810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.942+0000 7f585fc00640 1 --2- 172.21.15.90:0/4195744313 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f5858104fa0 0x7f5858105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.942+0000 7f585fc00640 1 -- 172.21.15.90:0/4195744313 >> 172.21.15.90:0/4195744313 conn(0x7f5858100a50 msgr2=0x7f5858102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:48.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.942+0000 7f585fc00640 1 -- 172.21.15.90:0/4195744313 shutdown_connections 2024-09-10T22:12:48.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.942+0000 7f585fc00640 1 -- 172.21.15.90:0/4195744313 wait complete. 2024-09-10T22:12:48.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.943+0000 7f585fc00640 1 Processor -- start 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.943+0000 7f585fc00640 1 -- start start 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f585fc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5858104fa0 0x7f585819b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f585fc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810a9b0 0x7f585819b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f585fc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f585810b330 0x7f58581a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f585fc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5858114e60 con 0x7f585810a9b0 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f585fc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f5858114ce0 con 0x7f5858104fa0 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f585fc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f5858114fe0 con 0x7f585810b330 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f5857e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810a9b0 0x7f585819b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f5857e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810a9b0 0x7f585819b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34674/0 (socket says 172.21.15.90:34674) 2024-09-10T22:12:48.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f5857e00640 1 -- 172.21.15.90:0/821752208 learned_addr learned my addr 172.21.15.90:0/821752208 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f585d200640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5858104fa0 0x7f585819b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f585dc00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f585810b330 0x7f58581a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.944+0000 7f5857e00640 1 -- 172.21.15.90:0/821752208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f585810b330 msgr2=0x7f58581a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f5857e00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f585810b330 0x7f58581a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f5857e00640 1 -- 172.21.15.90:0/821752208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5858104fa0 msgr2=0x7f585819b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f5857e00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5858104fa0 0x7f585819b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f5857e00640 1 -- 172.21.15.90:0/821752208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58581a1420 con 0x7f585810a9b0 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f585dc00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f585810b330 0x7f58581a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f5857e00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810a9b0 0x7f585819b8b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f584400d950 tx=0x7f584400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f5855600640 1 -- 172.21.15.90:0/821752208 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844017070 con 0x7f585810a9b0 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f5855600640 1 -- 172.21.15.90:0/821752208 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f584400de80 con 0x7f585810a9b0 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f5855600640 1 -- 172.21.15.90:0/821752208 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584400f620 con 0x7f585810a9b0 2024-09-10T22:12:48.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58581a1710 con 0x7f585810a9b0 2024-09-10T22:12:48.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.945+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58581a7dd0 con 0x7f585810a9b0 2024-09-10T22:12:48.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.946+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5858105380 con 0x7f585810a9b0 2024-09-10T22:12:48.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.949+0000 7f5855600640 1 -- 172.21.15.90:0/821752208 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f5844005030 con 0x7f585810a9b0 2024-09-10T22:12:48.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.950+0000 7f5855600640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58400b54d0 0x7f58400b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:48.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.950+0000 7f5855600640 1 -- 172.21.15.90:0/821752208 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9490+0+0 (secure 0 0 0) 0x7f58440e5f10 con 0x7f585810a9b0 2024-09-10T22:12:48.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.950+0000 7f585d200640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58400b54d0 0x7f58400b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:48.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.951+0000 7f585d200640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58400b54d0 0x7f58400b7990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f584c00abe0 tx=0x7f584c00a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:48.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:48.952+0000 7f5855600640 1 -- 172.21.15.90:0/821752208 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f58440ac6e0 con 0x7f585810a9b0 2024-09-10T22:12:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:48 smithi090 ceph-mon[30350]: pgmap v691: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-10T22:12:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:48 smithi154 ceph-mon[34495]: pgmap v691: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-10T22:12:49.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.173+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f585810d170 con 0x7f585810a9b0 2024-09-10T22:12:49.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.174+0000 7f5855600640 1 -- 172.21.15.90:0/821752208 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11986 (secure 0 0 0) 0x7f58440abfe0 con 0x7f585810a9b0 2024-09-10T22:12:49.176 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:12:49.176 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"btime":"2024-09-10T22:08:03:817761+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:08:03.499417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:12:49.177 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-09-10T22:12:49.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58400b54d0 msgr2=0x7f58400b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58400b54d0 0x7f58400b7990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f584c00abe0 tx=0x7f584c00a2d0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810a9b0 msgr2=0x7f585819b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810a9b0 0x7f585819b8b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f584400d950 tx=0x7f584400dd50 comp rx=0 tx=0).stop 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 shutdown_connections 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f58400b54d0 0x7f58400b7990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f585810b330 0x7f58581a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585810a9b0 0x7f585819b8b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 --2- 172.21.15.90:0/821752208 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f5858104fa0 0x7f585819b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.176+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 >> 172.21.15.90:0/821752208 conn(0x7f5858100a50 msgr2=0x7f5858102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.177+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 shutdown_connections 2024-09-10T22:12:49.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:49.177+0000 7f585fc00640 1 -- 172.21.15.90:0/821752208 wait complete. 2024-09-10T22:12:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:48 smithi171 ceph-mon[34138]: pgmap v691: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-09-10T22:12:49.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.24 MB/sec execute 341 sec latency 25387.033 ms 2024-09-10T22:12:49.938 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 6a665910 --release --await' 2024-09-10T22:12:49.938 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 6a665910 --release --await 2024-09-10T22:12:50.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:49 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/821752208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:12:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:49 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/821752208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:12:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:49 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/821752208' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:12:50.308 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:12:50.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107714 1.23 MB/sec execute 342 sec latency 26387.101 ms 2024-09-10T22:12:50.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.972+0000 7f3b7e400640 1 -- 172.21.15.90:0/472191168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b7810a800 msgr2=0x7f3b7810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.972+0000 7f3b7e400640 1 --2- 172.21.15.90:0/472191168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b7810a800 0x7f3b7810ac40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009900 tx=0x7f3b6402a410 comp rx=0 tx=0).stop 2024-09-10T22:12:50.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.972+0000 7f3b7e400640 1 -- 172.21.15.90:0/472191168 shutdown_connections 2024-09-10T22:12:50.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.972+0000 7f3b7e400640 1 --2- 172.21.15.90:0/472191168 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b7810b180 0x7f3b78111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.972+0000 7f3b7e400640 1 --2- 172.21.15.90:0/472191168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b7810a800 0x7f3b7810ac40 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.972+0000 7f3b7e400640 1 --2- 172.21.15.90:0/472191168 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b78104df0 0x7f3b781051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.972+0000 7f3b7e400640 1 -- 172.21.15.90:0/472191168 >> 172.21.15.90:0/472191168 conn(0x7f3b78100840 msgr2=0x7f3b78102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:50.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.972+0000 7f3b7e400640 1 -- 172.21.15.90:0/472191168 shutdown_connections 2024-09-10T22:12:50.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.972+0000 7f3b7e400640 1 -- 172.21.15.90:0/472191168 wait complete. 2024-09-10T22:12:50.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.973+0000 7f3b7e400640 1 Processor -- start 2024-09-10T22:12:50.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.973+0000 7f3b7e400640 1 -- start start 2024-09-10T22:12:50.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7e400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b78104df0 0x7f3b7819b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7e400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b7810a800 0x7f3b7819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7e400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b7810b180 0x7f3b781a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7e400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78114d00 con 0x7f3b78104df0 2024-09-10T22:12:50.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7e400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78114b80 con 0x7f3b7810a800 2024-09-10T22:12:50.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7e400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78114e80 con 0x7f3b7810b180 2024-09-10T22:12:50.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b78104df0 0x7f3b7819b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b77e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b7810a800 0x7f3b7819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b78104df0 0x7f3b7819b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34682/0 (socket says 172.21.15.90:34682) 2024-09-10T22:12:50.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7d000640 1 -- 172.21.15.90:0/216159555 learned_addr learned my addr 172.21.15.90:0/216159555 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:50.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7da00640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b7810b180 0x7f3b781a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7d000640 1 -- 172.21.15.90:0/216159555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b7810b180 msgr2=0x7f3b781a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7d000640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b7810b180 0x7f3b781a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7d000640 1 -- 172.21.15.90:0/216159555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b7810a800 msgr2=0x7f3b7819b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7d000640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b7810a800 0x7f3b7819b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.974+0000 7f3b7d000640 1 -- 172.21.15.90:0/216159555 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b781a1180 con 0x7f3b78104df0 2024-09-10T22:12:50.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.975+0000 7f3b7d000640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b78104df0 0x7f3b7819b0d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00d950 tx=0x7f3b6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:50.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.975+0000 7f3b75600640 1 -- 172.21.15.90:0/216159555 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c017070 con 0x7f3b78104df0 2024-09-10T22:12:50.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.975+0000 7f3b75600640 1 -- 172.21.15.90:0/216159555 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b6c009c30 con 0x7f3b78104df0 2024-09-10T22:12:50.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.975+0000 7f3b75600640 1 -- 172.21.15.90:0/216159555 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c00f620 con 0x7f3b78104df0 2024-09-10T22:12:50.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.975+0000 7f3b7e400640 1 -- 172.21.15.90:0/216159555 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b781a1470 con 0x7f3b78104df0 2024-09-10T22:12:50.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.975+0000 7f3b7e400640 1 -- 172.21.15.90:0/216159555 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781a7d00 con 0x7f3b78104df0 2024-09-10T22:12:50.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.978+0000 7f3b823be740 1 Processor -- start 2024-09-10T22:12:50.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.978+0000 7f3b823be740 1 -- start start 2024-09-10T22:12:50.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b75600640 1 -- 172.21.15.90:0/216159555 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3b6c010460 con 0x7f3b78104df0 2024-09-10T22:12:50.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b823be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b6841d0 0x55c98b6845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b823be740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b690540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b823be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b690a80 0x55c98b692e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b823be740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c98b5fb290 con 0x55c98b684af0 2024-09-10T22:12:50.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b823be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c98b5ea6c0 con 0x55c98b690a80 2024-09-10T22:12:50.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b823be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c98b5ee4d0 con 0x55c98b6841d0 2024-09-10T22:12:50.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b690540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b690540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34694/0 (socket says 172.21.15.90:34694) 2024-09-10T22:12:50.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b7da00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b6841d0 0x55c98b6845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.979+0000 7f3b7d000640 1 -- 172.21.15.90:0/1683198996 learned_addr learned my addr 172.21.15.90:0/1683198996 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:50.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b77e00640 1 --2- 172.21.15.90:0/1683198996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b690a80 0x55c98b692e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b7d000640 1 -- 172.21.15.90:0/1683198996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b6841d0 msgr2=0x55c98b6845b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:50.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b7d000640 1 --2- 172.21.15.90:0/1683198996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b6841d0 0x55c98b6845b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b7d000640 1 -- 172.21.15.90:0/1683198996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b690a80 msgr2=0x55c98b692e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:50.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b7d000640 1 --2- 172.21.15.90:0/1683198996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b690a80 0x55c98b692e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b7d000640 1 -- 172.21.15.90:0/1683198996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98b577460 con 0x55c98b684af0 2024-09-10T22:12:50.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b77e00640 1 --2- 172.21.15.90:0/1683198996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b690a80 0x55c98b692e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:12:50.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b7d000640 1 --2- 172.21.15.90:0/1683198996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b690540 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0ee1b0 tx=0x7f3b6c0ee670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15ae44087166361 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:50.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b6a000640 1 -- 172.21.15.90:0/1683198996 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0f9070 con 0x55c98b684af0 2024-09-10T22:12:50.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b6a000640 1 -- 172.21.15.90:0/1683198996 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b6c0e6c60 con 0x55c98b684af0 2024-09-10T22:12:50.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.980+0000 7f3b6a000640 1 -- 172.21.15.90:0/1683198996 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0f4b70 con 0x55c98b684af0 2024-09-10T22:12:50.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b75600640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3b600b5580 0x7f3b600b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b75600640 1 -- 172.21.15.90:0/216159555 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9490+0+0 (secure 0 0 0) 0x7f3b6c0e5fd0 con 0x7f3b78104df0 2024-09-10T22:12:50.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b7da00640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3b600b5580 0x7f3b600b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b823be740 1 -- 172.21.15.90:0/1683198996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 msgr2=0x55c98b690540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b823be740 1 --2- 172.21.15.90:0/1683198996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b690540 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0ee1b0 tx=0x7f3b6c0ee670 comp rx=0 tx=0).stop 2024-09-10T22:12:50.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b823be740 1 -- 172.21.15.90:0/1683198996 shutdown_connections 2024-09-10T22:12:50.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b823be740 1 --2- 172.21.15.90:0/1683198996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b690a80 0x55c98b692e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b823be740 1 --2- 172.21.15.90:0/1683198996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b690540 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b823be740 1 --2- 172.21.15.90:0/1683198996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b6841d0 0x55c98b6845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.981+0000 7f3b823be740 1 -- 172.21.15.90:0/1683198996 >> 172.21.15.90:0/1683198996 conn(0x55c98b682000 msgr2=0x55c98b6831a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:50.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.982+0000 7f3b823be740 1 -- 172.21.15.90:0/1683198996 shutdown_connections 2024-09-10T22:12:50.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.982+0000 7f3b823be740 1 -- 172.21.15.90:0/1683198996 wait complete. 2024-09-10T22:12:50.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.982+0000 7f3b7da00640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3b600b5580 0x7f3b600b7a40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3b70005fd0 tx=0x7f3b70009740 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:50.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.982+0000 7f3b823be740 1 -- start start 2024-09-10T22:12:50.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.982+0000 7f3b823be740 1 Processor -- start 2024-09-10T22:12:50.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b823be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b68da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b823be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b684af0 0x55c98b68df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b823be740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b690a80 0x55c98b68e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b823be740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c98b44bbb0 con 0x55c98b690a80 2024-09-10T22:12:50.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b823be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c98b5856c0 con 0x55c98b6841d0 2024-09-10T22:12:50.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b823be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c98b693730 con 0x55c98b684af0 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b823be740 10 client.0 _ll_register_callbacks cb 0x55c98b5a0490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b7da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b690a80 0x55c98b68e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b68da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b68da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:40410/0 (socket says 172.21.15.90:40410) 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b7d000640 1 -- 172.21.15.90:0/2008608292 learned_addr learned my addr 172.21.15.90:0/2008608292 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.983+0000 7f3b77e00640 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b684af0 0x55c98b68df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b7d000640 1 -- 172.21.15.90:0/2008608292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b684af0 msgr2=0x55c98b68df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b7d000640 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b684af0 0x55c98b68df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b7d000640 1 -- 172.21.15.90:0/2008608292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b690a80 msgr2=0x55c98b68e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b7d000640 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b690a80 0x55c98b68e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b7d000640 1 -- 172.21.15.90:0/2008608292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98b348b00 con 0x55c98b6841d0 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b77e00640 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b684af0 0x55c98b68df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:12:50.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b7da00640 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b690a80 0x55c98b68e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b7d000640 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b68da30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0abfb0 tx=0x7f3b6c0f4400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3801cc291c330bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b5ea00640 1 -- 172.21.15.90:0/2008608292 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c020070 con 0x55c98b6841d0 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b5ea00640 1 -- 172.21.15.90:0/2008608292 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9490+0+0 (secure 0 0 0) 0x7f3b6c0a8290 con 0x55c98b6841d0 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.984+0000 7f3b823be740 1 -- 172.21.15.90:0/2008608292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b6c0e6c60 con 0x55c98b6841d0 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b5ea00640 1 -- 172.21.15.90:0/2008608292 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3b6c0a8670 con 0x55c98b6841d0 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b823be740 10 client.35702 fetch_fsmap learned FSMap version 35 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b823be740 1 -- 172.21.15.90:0/2008608292 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c98b3bf960 con 0x55c98b6841d0 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b5ea00640 1 -- 172.21.15.90:0/2008608292 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b6c0e6c60 con 0x55c98b6841d0 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b5ea00640 1 -- 172.21.15.90:0/2008608292 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3b6c010780 con 0x55c98b6841d0 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b823be740 10 client.35702 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b823be740 10 client.35702 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:12:50.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b823be740 20 client.35702 populate_metadata read hostname 'smithi090' 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b823be740 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c98b6a03c0 0x55c98b6a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b823be740 4 client.35702 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.985+0000 7f3b823be740 1 -- 172.21.15.90:0/2008608292 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c98b5fd1e0 con 0x55c98b6a03c0 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.986+0000 7f3b77e00640 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c98b6a03c0 0x55c98b6a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.986+0000 7f3b77e00640 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c98b6a03c0 0x55c98b6a27b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb618610f1f5f07 server_cookie=888fa2c4f9d23391 in_seq=0 out_seq=0 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.986+0000 7f3b5ea00640 10 client.35702 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.987+0000 7f3b5ea00640 1 -- 172.21.15.90:0/2008608292 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c98b5fd1e0 con 0x55c98b6a03c0 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.987+0000 7f3b5ea00640 10 client.35702 handle_command_reply: tid=0 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.987+0000 7f3b823be740 1 client.35702 shutdown 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 -- 172.21.15.90:0/2008608292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 msgr2=0x55c98b68da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b68da30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0abfb0 tx=0x7f3b6c0f4400 comp rx=0 tx=0).stop 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 -- 172.21.15.90:0/2008608292 shutdown_connections 2024-09-10T22:12:50.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c98b6a03c0 0x55c98b6a27b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b690a80 0x55c98b68e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b684af0 0x55c98b68df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b5ea00640 0 client.35702 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:12:50.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 --2- 172.21.15.90:0/2008608292 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b68da30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:50.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 -- 172.21.15.90:0/2008608292 >> 172.21.15.90:0/2008608292 conn(0x55c98b682000 msgr2=0x55c98b687ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:50.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 -- 172.21.15.90:0/2008608292 shutdown_connections 2024-09-10T22:12:50.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 1 -- 172.21.15.90:0/2008608292 wait complete. 2024-09-10T22:12:50.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:50.988+0000 7f3b823be740 20 client.35702 trim_cache size 0 max 16384 2024-09-10T22:12:51.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b823be740 1 Processor -- start 2024-09-10T22:12:51.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b823be740 1 -- start start 2024-09-10T22:12:51.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b823be740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b6841d0 0x55c98b683500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b823be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b684af0 0x55c98b68ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b823be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 0x55c98b68b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b823be740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b1e40 con 0x55c98b6841d0 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b823be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b3980 con 0x55c98b684af0 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b823be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b2510 con 0x55c98b690a80 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b6841d0 0x55c98b683500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b7d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b6841d0 0x55c98b683500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34704/0 (socket says 172.21.15.90:34704) 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b7d000640 1 -- 172.21.15.90:0/801997991 learned_addr learned my addr 172.21.15.90:0/801997991 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.006+0000 7f3b77e00640 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b684af0 0x55c98b68ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b7da00640 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 0x55c98b68b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b7d000640 1 -- 172.21.15.90:0/801997991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 msgr2=0x55c98b68b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b7d000640 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 0x55c98b68b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b7d000640 1 -- 172.21.15.90:0/801997991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b684af0 msgr2=0x55c98b68ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b7d000640 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b684af0 0x55c98b68ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b7d000640 1 -- 172.21.15.90:0/801997991 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98b683a40 con 0x55c98b6841d0 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b7da00640 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 0x55c98b68b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:12:51.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b7d000640 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b6841d0 0x55c98b683500 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0e6390 tx=0x7f3b6c00df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e13dd4f1bfe6a75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:51.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b69600640 1 -- 172.21.15.90:0/801997991 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c020030 con 0x55c98b6841d0 2024-09-10T22:12:51.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b69600640 1 -- 172.21.15.90:0/801997991 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b6c0e95b0 con 0x55c98b6841d0 2024-09-10T22:12:51.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.007+0000 7f3b69600640 1 -- 172.21.15.90:0/801997991 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c0ab790 con 0x55c98b6841d0 2024-09-10T22:12:51.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.008+0000 7f3b823be740 1 -- 172.21.15.90:0/801997991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b6841d0 msgr2=0x55c98b683500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.008+0000 7f3b823be740 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b6841d0 0x55c98b683500 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0e6390 tx=0x7f3b6c00df40 comp rx=0 tx=0).stop 2024-09-10T22:12:51.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.008+0000 7f3b823be740 1 -- 172.21.15.90:0/801997991 shutdown_connections 2024-09-10T22:12:51.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.008+0000 7f3b823be740 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 0x55c98b68b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.008+0000 7f3b823be740 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b684af0 0x55c98b68ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.008+0000 7f3b823be740 1 --2- 172.21.15.90:0/801997991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b6841d0 0x55c98b683500 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.008+0000 7f3b823be740 1 -- 172.21.15.90:0/801997991 >> 172.21.15.90:0/801997991 conn(0x55c98b682000 msgr2=0x55c98b68f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:51.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.008+0000 7f3b823be740 1 -- 172.21.15.90:0/801997991 shutdown_connections 2024-09-10T22:12:51.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.008+0000 7f3b823be740 1 -- 172.21.15.90:0/801997991 wait complete. 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b823be740 1 -- start start 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b823be740 1 Processor -- start 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b823be740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b6989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b823be740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b698f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b823be740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 0x55c98b693980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b823be740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b32b0 con 0x55c98b684af0 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b823be740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781b2be0 con 0x55c98b6841d0 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b823be740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3b781afe40 con 0x55c98b690a80 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b823be740 10 client.0 _ll_register_callbacks cb 0x7f3b6c0f7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.009+0000 7f3b77e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b698f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b77e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b698f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34720/0 (socket says 172.21.15.90:34720) 2024-09-10T22:12:51.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b77e00640 1 -- 172.21.15.90:0/1312751926 learned_addr learned my addr 172.21.15.90:0/1312751926 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b7d000640 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b6989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b7da00640 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 0x55c98b693980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b77e00640 1 -- 172.21.15.90:0/1312751926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 msgr2=0x55c98b693980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b77e00640 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 0x55c98b693980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b77e00640 1 -- 172.21.15.90:0/1312751926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 msgr2=0x55c98b6989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b77e00640 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b6989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b77e00640 1 -- 172.21.15.90:0/1312751926 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c98b699580 con 0x55c98b684af0 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b77e00640 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b698f10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006010 tx=0x7f3b64005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d5d95b3c0b127d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.010+0000 7f3b5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b5ea00640 1 -- 172.21.15.90:0/1312751926 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64035040 con 0x55c98b684af0 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b5ea00640 1 -- 172.21.15.90:0/1312751926 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9490+0+0 (secure 0 0 0) 0x7f3b640046c0 con 0x55c98b684af0 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b823be740 1 -- 172.21.15.90:0/1312751926 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b6c0e95b0 con 0x55c98b684af0 2024-09-10T22:12:51.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b5ea00640 1 -- 172.21.15.90:0/1312751926 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3b640048c0 con 0x55c98b684af0 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b823be740 10 client.16056 fetch_fsmap learned FSMap version 35 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b823be740 1 -- 172.21.15.90:0/1312751926 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c98b70d4f0 con 0x55c98b684af0 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b5ea00640 1 -- 172.21.15.90:0/1312751926 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b6c0e95b0 con 0x55c98b684af0 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b5ea00640 1 -- 172.21.15.90:0/1312751926 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 35) ==== 8370+0+0 (secure 0 0 0) 0x7f3b64003c50 con 0x55c98b684af0 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b823be740 10 client.16056 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b823be740 10 client.16056 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b823be740 20 client.16056 populate_metadata read hostname 'smithi090' 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b823be740 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c98b70dc50 0x55c98b710070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b823be740 4 client.16056 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6a665910", "release": true, "await": true}] 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.011+0000 7f3b823be740 1 -- 172.21.15.90:0/1312751926 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6a665910", "release": true, "await": true}) -- 0x55c98b5f8c90 con 0x55c98b70dc50 2024-09-10T22:12:51.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.012+0000 7f3b7d000640 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c98b70dc50 0x55c98b710070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:12:51.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.012+0000 7f3b5ea00640 10 client.16056 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:12:51.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.012+0000 7f3b7d000640 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c98b70dc50 0x55c98b710070 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75761f53662e895e server_cookie=37a2e1cd6ea22a45 in_seq=0 out_seq=0 2024-09-10T22:12:51.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b5ea00640 1 -- 172.21.15.90:0/1312751926 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c98b5f8c90 con 0x55c98b70dc50 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b5ea00640 10 client.16056 handle_command_reply: tid=0 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b823be740 1 client.16056 shutdown 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b823be740 1 -- 172.21.15.90:0/1312751926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 msgr2=0x55c98b698f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b823be740 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b698f10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006010 tx=0x7f3b64005c00 comp rx=0 tx=0).stop 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b823be740 1 -- 172.21.15.90:0/1312751926 shutdown_connections 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b823be740 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c98b70dc50 0x55c98b710070 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b823be740 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c98b690a80 0x55c98b693980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b823be740 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c98b684af0 0x55c98b698f10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b823be740 1 --2- 172.21.15.90:0/1312751926 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c98b6841d0 0x55c98b6989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b823be740 1 -- 172.21.15.90:0/1312751926 >> 172.21.15.90:0/1312751926 conn(0x55c98b682000 msgr2=0x55c98b68d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.020+0000 7f3b5ea00640 0 client.16056 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.021+0000 7f3b823be740 1 -- 172.21.15.90:0/1312751926 shutdown_connections 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.021+0000 7f3b823be740 1 -- 172.21.15.90:0/1312751926 wait complete. 2024-09-10T22:12:51.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.021+0000 7f3b823be740 20 client.16056 trim_cache size 0 max 16384 2024-09-10T22:12:51.024 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:12:51.024 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 35, 2024-09-10T22:12:51.024 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:12:51.024 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 28, 2024-09-10T22:12:51.024 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:12:51.024 INFO:teuthology.orchestra.run.smithi090.stdout: "6a665910": { 2024-09-10T22:12:51.024 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 28, 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 148.4, 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:12:51.025 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:12:51.026 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:51.026 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:51.026 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:51.026 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:51.026 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:12:51.026 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.021+0000 7f3b7e400640 1 -- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3b600b5580 msgr2=0x7f3b600b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.021+0000 7f3b7e400640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3b600b5580 0x7f3b600b7a40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3b70005fd0 tx=0x7f3b70009740 comp rx=0 tx=0).stop 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 -- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b78104df0 msgr2=0x7f3b7819b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b78104df0 0x7f3b7819b0d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c00d950 tx=0x7f3b6c00de10 comp rx=0 tx=0).stop 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 -- 172.21.15.90:0/216159555 shutdown_connections 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3b600b5580 0x7f3b600b7a40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3b7810b180 0x7f3b781a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3b7810a800 0x7f3b7819b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 --2- 172.21.15.90:0/216159555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b78104df0 0x7f3b7819b0d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 -- 172.21.15.90:0/216159555 >> 172.21.15.90:0/216159555 conn(0x7f3b78100840 msgr2=0x7f3b78102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 -- 172.21.15.90:0/216159555 shutdown_connections 2024-09-10T22:12:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:12:51.022+0000 7f3b7e400640 1 -- 172.21.15.90:0/216159555 wait complete. 2024-09-10T22:12:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:50 smithi090 ceph-mon[30350]: pgmap v692: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-10T22:12:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:50 smithi154 ceph-mon[34495]: pgmap v692: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-10T22:12:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:50 smithi171 ceph-mon[34138]: pgmap v692: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.2 KiB/s wr, 11 op/s 2024-09-10T22:12:51.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107922 1.24 MB/sec execute 343 sec latency 26922.093 ms 2024-09-10T22:12:51.922 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6a665910', total seconds elapsed: 32.4 2024-09-10T22:12:52.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 108276 1.23 MB/sec execute 344 sec latency 22.771 ms 2024-09-10T22:12:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:52 smithi154 ceph-mon[34495]: pgmap v693: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 KiB/s wr, 10 op/s 2024-09-10T22:12:53.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:52 smithi171 ceph-mon[34138]: pgmap v693: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 KiB/s wr, 10 op/s 2024-09-10T22:12:53.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:52 smithi090 ceph-mon[30350]: pgmap v693: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.9 KiB/s wr, 10 op/s 2024-09-10T22:12:53.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 108668 1.23 MB/sec execute 345 sec latency 23.183 ms 2024-09-10T22:12:54.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 109009 1.23 MB/sec execute 346 sec latency 20.369 ms 2024-09-10T22:12:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:54 smithi154 ceph-mon[34495]: pgmap v694: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-09-10T22:12:55.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:54 smithi171 ceph-mon[34138]: pgmap v694: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-09-10T22:12:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:54 smithi090 ceph-mon[30350]: pgmap v694: 129 pgs: 129 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 KiB/s wr, 16 op/s 2024-09-10T22:12:55.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 109473 1.24 MB/sec execute 347 sec latency 19.322 ms 2024-09-10T22:12:56.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:55 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:12:56.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:55 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:12:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:55 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:12:56.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 109776 1.25 MB/sec execute 348 sec latency 204.407 ms 2024-09-10T22:12:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:56 smithi090 ceph-mon[30350]: pgmap v695: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 180 KiB/s wr, 31 op/s 2024-09-10T22:12:57.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:56 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:56 smithi154 ceph-mon[34495]: pgmap v695: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 180 KiB/s wr, 31 op/s 2024-09-10T22:12:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:56 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.228 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:56 smithi171 ceph-mon[34138]: pgmap v695: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 180 KiB/s wr, 31 op/s 2024-09-10T22:12:57.228 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.228 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.228 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.228 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:56 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:57.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 110115 1.25 MB/sec execute 349 sec latency 305.612 ms 2024-09-10T22:12:58.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 110337 1.25 MB/sec execute 350 sec latency 17.082 ms 2024-09-10T22:12:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:58 smithi154 ceph-mon[34495]: pgmap v696: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 343 KiB/s wr, 30 op/s 2024-09-10T22:12:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:58 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:58 smithi171 ceph-mon[34138]: pgmap v696: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 343 KiB/s wr, 30 op/s 2024-09-10T22:12:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:58 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:58 smithi090 ceph-mon[30350]: pgmap v696: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 343 KiB/s wr, 30 op/s 2024-09-10T22:12:59.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:58 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:12:59.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 110526 1.25 MB/sec execute 351 sec latency 17.926 ms 2024-09-10T22:13:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:59 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:13:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:59 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:13:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:12:59 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:13:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:59 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:13:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:59 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:13:00.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:12:59 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:13:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:59 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:13:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:59 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:13:00.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:12:59 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:13:00.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 110824 1.25 MB/sec execute 352 sec latency 21.212 ms 2024-09-10T22:13:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:00 smithi154 ceph-mon[34495]: pgmap v697: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 670 KiB/s wr, 54 op/s 2024-09-10T22:13:01.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:00 smithi171 ceph-mon[34138]: pgmap v697: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 670 KiB/s wr, 54 op/s 2024-09-10T22:13:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:00 smithi090 ceph-mon[30350]: pgmap v697: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 670 KiB/s wr, 54 op/s 2024-09-10T22:13:01.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 110905 1.24 MB/sec execute 353 sec latency 777.518 ms 2024-09-10T22:13:02.483 INFO:tasks.workunit.client.0.smithi090.stdout: 1 110905 1.24 MB/sec execute 354 sec latency 1777.624 ms 2024-09-10T22:13:03.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:02 smithi090 ceph-mon[30350]: pgmap v698: 129 pgs: 129 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 831 KiB/s wr, 70 op/s 2024-09-10T22:13:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:02 smithi154 ceph-mon[34495]: pgmap v698: 129 pgs: 129 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 831 KiB/s wr, 70 op/s 2024-09-10T22:13:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:02 smithi171 ceph-mon[34138]: pgmap v698: 129 pgs: 129 active+clean; 137 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 831 KiB/s wr, 70 op/s 2024-09-10T22:13:03.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 111179 1.24 MB/sec execute 355 sec latency 1920.793 ms 2024-09-10T22:13:04.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 111537 1.25 MB/sec execute 356 sec latency 48.638 ms 2024-09-10T22:13:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:04 smithi154 ceph-mon[34495]: pgmap v699: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 937 KiB/s wr, 96 op/s 2024-09-10T22:13:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:04 smithi154 ceph-mon[34495]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T22:13:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:04 smithi171 ceph-mon[34138]: pgmap v699: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 937 KiB/s wr, 96 op/s 2024-09-10T22:13:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:04 smithi171 ceph-mon[34138]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T22:13:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:04 smithi090 ceph-mon[30350]: pgmap v699: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 937 KiB/s wr, 96 op/s 2024-09-10T22:13:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:04 smithi090 ceph-mon[30350]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T22:13:05.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 111877 1.24 MB/sec execute 357 sec latency 24.250 ms 2024-09-10T22:13:06.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:05 smithi171 ceph-mon[34138]: pgmap v701: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-09-10T22:13:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:05 smithi090 ceph-mon[30350]: pgmap v701: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-09-10T22:13:06.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:05 smithi154 ceph-mon[34495]: pgmap v701: 129 pgs: 129 active+clean; 135 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-09-10T22:13:06.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 112325 1.24 MB/sec execute 358 sec latency 24.501 ms 2024-09-10T22:13:07.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 112603 1.24 MB/sec execute 359 sec latency 19.860 ms 2024-09-10T22:13:08.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 113206 1.25 MB/sec execute 360 sec latency 18.844 ms 2024-09-10T22:13:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:08 smithi090 ceph-mon[30350]: pgmap v702: 129 pgs: 129 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-09-10T22:13:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:08 smithi154 ceph-mon[34495]: pgmap v702: 129 pgs: 129 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-09-10T22:13:09.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:08 smithi171 ceph-mon[34138]: pgmap v702: 129 pgs: 129 active+clean; 136 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.0 MiB/s wr, 109 op/s 2024-09-10T22:13:09.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 113392 1.25 MB/sec execute 361 sec latency 19.353 ms 2024-09-10T22:13:10.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 113787 1.26 MB/sec execute 362 sec latency 18.231 ms 2024-09-10T22:13:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:10 smithi090 ceph-mon[30350]: pgmap v703: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-09-10T22:13:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:10 smithi090 ceph-mon[30350]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T22:13:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:10 smithi154 ceph-mon[34495]: pgmap v703: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-09-10T22:13:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:10 smithi154 ceph-mon[34495]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T22:13:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:10 smithi171 ceph-mon[34138]: pgmap v703: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-09-10T22:13:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:10 smithi171 ceph-mon[34138]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T22:13:11.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 113962 1.26 MB/sec execute 363 sec latency 19.650 ms 2024-09-10T22:13:12.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 114156 1.26 MB/sec execute 364 sec latency 22.933 ms 2024-09-10T22:13:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:12 smithi090 ceph-mon[30350]: pgmap v705: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 950 KiB/s wr, 85 op/s 2024-09-10T22:13:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:12 smithi154 ceph-mon[34495]: pgmap v705: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 950 KiB/s wr, 85 op/s 2024-09-10T22:13:13.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:12 smithi171 ceph-mon[34138]: pgmap v705: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 950 KiB/s wr, 85 op/s 2024-09-10T22:13:13.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 114517 1.26 MB/sec execute 365 sec latency 18.895 ms 2024-09-10T22:13:14.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 114928 1.26 MB/sec execute 366 sec latency 21.716 ms 2024-09-10T22:13:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:14 smithi090 ceph-mon[30350]: pgmap v706: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-10T22:13:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:14 smithi154 ceph-mon[34495]: pgmap v706: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-10T22:13:15.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:14 smithi171 ceph-mon[34138]: pgmap v706: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.2 MiB/s wr, 99 op/s 2024-09-10T22:13:15.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 115235 1.26 MB/sec execute 367 sec latency 25.865 ms 2024-09-10T22:13:16.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 115628 1.26 MB/sec execute 368 sec latency 26.414 ms 2024-09-10T22:13:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:16 smithi154 ceph-mon[34495]: pgmap v707: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-09-10T22:13:17.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:16 smithi171 ceph-mon[34138]: pgmap v707: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-09-10T22:13:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:16 smithi090 ceph-mon[30350]: pgmap v707: 129 pgs: 129 active+clean; 142 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-09-10T22:13:17.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 115974 1.26 MB/sec execute 369 sec latency 25.620 ms 2024-09-10T22:13:18.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 116436 1.27 MB/sec execute 370 sec latency 23.046 ms 2024-09-10T22:13:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:18 smithi154 ceph-mon[34495]: pgmap v708: 129 pgs: 129 active+clean; 141 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-10T22:13:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:18 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:18 smithi171 ceph-mon[34138]: pgmap v708: 129 pgs: 129 active+clean; 141 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-10T22:13:19.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:18 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:19.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:18 smithi090 ceph-mon[30350]: pgmap v708: 129 pgs: 129 active+clean; 141 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.2 MiB/s wr, 105 op/s 2024-09-10T22:13:19.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:18 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:19.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 116850 1.27 MB/sec execute 371 sec latency 16.721 ms 2024-09-10T22:13:20.484 INFO:tasks.workunit.client.0.smithi090.stdout: 1 117033 1.28 MB/sec execute 372 sec latency 385.117 ms 2024-09-10T22:13:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:20 smithi154 ceph-mon[34495]: pgmap v709: 129 pgs: 129 active+clean; 144 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-10T22:13:21.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:20 smithi171 ceph-mon[34138]: pgmap v709: 129 pgs: 129 active+clean; 144 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-10T22:13:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:20 smithi090 ceph-mon[30350]: pgmap v709: 129 pgs: 129 active+clean; 144 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-10T22:13:21.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 117033 1.27 MB/sec execute 373 sec latency 1385.234 ms 2024-09-10T22:13:22.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 117033 1.27 MB/sec execute 374 sec latency 2385.384 ms 2024-09-10T22:13:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:22 smithi154 ceph-mon[34495]: pgmap v710: 129 pgs: 129 active+clean; 144 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-10T22:13:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:22 smithi171 ceph-mon[34138]: pgmap v710: 129 pgs: 129 active+clean; 144 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-10T22:13:23.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:22 smithi090 ceph-mon[30350]: pgmap v710: 129 pgs: 129 active+clean; 144 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.1 MiB/s wr, 106 op/s 2024-09-10T22:13:23.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 117194 1.27 MB/sec execute 375 sec latency 2525.704 ms 2024-09-10T22:13:24.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 117399 1.27 MB/sec execute 376 sec latency 26.524 ms 2024-09-10T22:13:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:24 smithi154 ceph-mon[34495]: pgmap v711: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T22:13:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:24 smithi171 ceph-mon[34138]: pgmap v711: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T22:13:25.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:24 smithi090 ceph-mon[30350]: pgmap v711: 129 pgs: 129 active+clean; 145 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T22:13:25.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 117570 1.26 MB/sec execute 377 sec latency 17.751 ms 2024-09-10T22:13:26.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 117827 1.26 MB/sec execute 378 sec latency 21.072 ms 2024-09-10T22:13:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:26 smithi154 ceph-mon[34495]: pgmap v712: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-09-10T22:13:27.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:26 smithi171 ceph-mon[34138]: pgmap v712: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-09-10T22:13:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:26 smithi090 ceph-mon[30350]: pgmap v712: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 1.0 MiB/s wr, 93 op/s 2024-09-10T22:13:27.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 118183 1.26 MB/sec execute 379 sec latency 22.239 ms 2024-09-10T22:13:28.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 118516 1.27 MB/sec execute 380 sec latency 136.426 ms 2024-09-10T22:13:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:28 smithi154 ceph-mon[34495]: pgmap v713: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 970 KiB/s wr, 77 op/s 2024-09-10T22:13:29.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:28 smithi171 ceph-mon[34138]: pgmap v713: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 970 KiB/s wr, 77 op/s 2024-09-10T22:13:29.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:28 smithi090 ceph-mon[30350]: pgmap v713: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 970 KiB/s wr, 77 op/s 2024-09-10T22:13:29.488 INFO:tasks.workunit.client.0.smithi090.stdout: 1 118858 1.27 MB/sec execute 381 sec latency 24.723 ms 2024-09-10T22:13:30.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 119241 1.26 MB/sec execute 382 sec latency 27.599 ms 2024-09-10T22:13:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:30 smithi154 ceph-mon[34495]: pgmap v714: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T22:13:31.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:30 smithi171 ceph-mon[34138]: pgmap v714: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T22:13:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:30 smithi090 ceph-mon[30350]: pgmap v714: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T22:13:31.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 119565 1.26 MB/sec execute 383 sec latency 19.907 ms 2024-09-10T22:13:32.486 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120060 1.27 MB/sec execute 384 sec latency 21.586 ms 2024-09-10T22:13:33.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:32 smithi090 ceph-mon[30350]: pgmap v715: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 811 KiB/s wr, 71 op/s 2024-09-10T22:13:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:32 smithi154 ceph-mon[34495]: pgmap v715: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 811 KiB/s wr, 71 op/s 2024-09-10T22:13:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:32 smithi171 ceph-mon[34138]: pgmap v715: 129 pgs: 129 active+clean; 148 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 811 KiB/s wr, 71 op/s 2024-09-10T22:13:33.485 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120413 1.28 MB/sec execute 385 sec latency 17.858 ms 2024-09-10T22:13:34.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:33 smithi154 ceph-mon[34495]: pgmap v716: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-10T22:13:34.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:33 smithi171 ceph-mon[34138]: pgmap v716: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-10T22:13:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:33 smithi090 ceph-mon[30350]: pgmap v716: 129 pgs: 129 active+clean; 150 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-09-10T22:13:34.486 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120706 1.28 MB/sec execute 386 sec latency 204.077 ms 2024-09-10T22:13:35.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:34 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:35.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:34 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:34 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:35.486 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120706 1.28 MB/sec execute 387 sec latency 1204.212 ms 2024-09-10T22:13:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:35 smithi154 ceph-mon[34495]: pgmap v717: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-09-10T22:13:36.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:35 smithi171 ceph-mon[34138]: pgmap v717: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-09-10T22:13:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:35 smithi090 ceph-mon[30350]: pgmap v717: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-09-10T22:13:36.486 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120706 1.27 MB/sec execute 388 sec latency 2204.353 ms 2024-09-10T22:13:37.486 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120706 1.27 MB/sec execute 389 sec latency 3204.495 ms 2024-09-10T22:13:38.486 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120773 1.27 MB/sec execute 390 sec latency 3346.190 ms 2024-09-10T22:13:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:38 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:38 smithi171 ceph-mon[34138]: pgmap v718: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 974 KiB/s wr, 86 op/s 2024-09-10T22:13:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:38 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:38 smithi090 ceph-mon[30350]: pgmap v718: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 974 KiB/s wr, 86 op/s 2024-09-10T22:13:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:38 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:38 smithi154 ceph-mon[34495]: pgmap v718: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 974 KiB/s wr, 86 op/s 2024-09-10T22:13:39.486 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120773 1.26 MB/sec execute 391 sec latency 1517.915 ms 2024-09-10T22:13:39.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:39 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:40.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:39 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:40.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:39 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:13:40.486 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120773 1.26 MB/sec execute 392 sec latency 2518.058 ms 2024-09-10T22:13:40.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:40 smithi171 ceph-mon[34138]: pgmap v719: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 925 KiB/s wr, 92 op/s 2024-09-10T22:13:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:40 smithi090 ceph-mon[30350]: pgmap v719: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 925 KiB/s wr, 92 op/s 2024-09-10T22:13:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:40 smithi154 ceph-mon[34495]: pgmap v719: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 925 KiB/s wr, 92 op/s 2024-09-10T22:13:41.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120773 1.26 MB/sec execute 393 sec latency 3518.200 ms 2024-09-10T22:13:42.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120773 1.26 MB/sec execute 394 sec latency 4518.327 ms 2024-09-10T22:13:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:42 smithi154 ceph-mon[34495]: pgmap v720: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 624 KiB/s wr, 60 op/s 2024-09-10T22:13:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:42 smithi171 ceph-mon[34138]: pgmap v720: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 624 KiB/s wr, 60 op/s 2024-09-10T22:13:43.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:42 smithi090 ceph-mon[30350]: pgmap v720: 129 pgs: 129 active+clean; 151 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 624 KiB/s wr, 60 op/s 2024-09-10T22:13:43.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120937 1.25 MB/sec execute 395 sec latency 4656.925 ms 2024-09-10T22:13:44.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 121105 1.25 MB/sec execute 396 sec latency 19.602 ms 2024-09-10T22:13:44.666 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 33.4 2024-09-10T22:13:44.667 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:13:45.074 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:13:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:44 smithi090 ceph-mon[30350]: pgmap v721: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 662 KiB/s wr, 63 op/s 2024-09-10T22:13:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:44 smithi154 ceph-mon[34495]: pgmap v721: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 662 KiB/s wr, 63 op/s 2024-09-10T22:13:45.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:44 smithi171 ceph-mon[34138]: pgmap v721: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 662 KiB/s wr, 63 op/s 2024-09-10T22:13:45.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 121355 1.25 MB/sec execute 397 sec latency 21.450 ms 2024-09-10T22:13:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.724+0000 7fb96b200640 1 -- 172.21.15.90:0/2306914138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb964106d40 msgr2=0x7fb964107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.724+0000 7fb96b200640 1 --2- 172.21.15.90:0/2306914138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb964106d40 0x7fb964107190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb960009ca0 tx=0x7fb96002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.724+0000 7fb96b200640 1 -- 172.21.15.90:0/2306914138 shutdown_connections 2024-09-10T22:13:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.724+0000 7fb96b200640 1 --2- 172.21.15.90:0/2306914138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb964106d40 0x7fb964107190 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.724+0000 7fb96b200640 1 --2- 172.21.15.90:0/2306914138 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb964102220 0x7fb9641066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.724+0000 7fb96b200640 1 --2- 172.21.15.90:0/2306914138 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb964101900 0x7fb964101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.724+0000 7fb96b200640 1 -- 172.21.15.90:0/2306914138 >> 172.21.15.90:0/2306914138 conn(0x7fb9640fd5b0 msgr2=0x7fb9640ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.724+0000 7fb96b200640 1 -- 172.21.15.90:0/2306914138 shutdown_connections 2024-09-10T22:13:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.724+0000 7fb96b200640 1 -- 172.21.15.90:0/2306914138 wait complete. 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.725+0000 7fb96b200640 1 Processor -- start 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.725+0000 7fb96b200640 1 -- start start 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.725+0000 7fb96b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb964101900 0x7fb96419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.725+0000 7fb96b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb964102220 0x7fb96419fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb96b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb964106d40 0x7fb9641a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb96b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb9641190c0 con 0x7fb964102220 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb96b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb964118f40 con 0x7fb964101900 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb96b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb964119240 con 0x7fb964106d40 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb969e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb964101900 0x7fb96419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb969400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb964102220 0x7fb96419fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb969e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb964101900 0x7fb96419f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:57022/0 (socket says 172.21.15.90:57022) 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb969e00640 1 -- 172.21.15.90:0/446910252 learned_addr learned my addr 172.21.15.90:0/446910252 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb96a800640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb964106d40 0x7fb9641a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb969e00640 1 -- 172.21.15.90:0/446910252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb964106d40 msgr2=0x7fb9641a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb969e00640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb964106d40 0x7fb9641a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb969e00640 1 -- 172.21.15.90:0/446910252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb964102220 msgr2=0x7fb96419fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb969e00640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb964102220 0x7fb96419fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.726+0000 7fb969e00640 1 -- 172.21.15.90:0/446910252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9641a5580 con 0x7fb964101900 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.727+0000 7fb969e00640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb964101900 0x7fb96419f4d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb95800e9f0 tx=0x7fb95800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.727+0000 7fb95ea00640 1 -- 172.21.15.90:0/446910252 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95800cd80 con 0x7fb964101900 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.727+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9641a5870 con 0x7fb964101900 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.727+0000 7fb95ea00640 1 -- 172.21.15.90:0/446910252 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb9580108c0 con 0x7fb964101900 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.727+0000 7fb95ea00640 1 -- 172.21.15.90:0/446910252 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9580187c0 con 0x7fb964101900 2024-09-10T22:13:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.727+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9641ac100 con 0x7fb964101900 2024-09-10T22:13:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.728+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96406b1b0 con 0x7fb964101900 2024-09-10T22:13:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.731+0000 7fb95ea00640 1 -- 172.21.15.90:0/446910252 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb958010430 con 0x7fb964101900 2024-09-10T22:13:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.732+0000 7fb95ea00640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb94c0b5580 0x7fb94c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.732+0000 7fb95ea00640 1 -- 172.21.15.90:0/446910252 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7fb9580e66e0 con 0x7fb964101900 2024-09-10T22:13:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.733+0000 7fb969400640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb94c0b5580 0x7fb94c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.733+0000 7fb969400640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb94c0b5580 0x7fb94c0b7a40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb9641073f0 tx=0x7fb954005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.735+0000 7fb95ea00640 1 -- 172.21.15.90:0/446910252 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb9580ac7e0 con 0x7fb964101900 2024-09-10T22:13:46.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:45.999+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9641ac3e0 con 0x7fb964101900 2024-09-10T22:13:46.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.000+0000 7fb95ea00640 1 -- 172.21.15.90:0/446910252 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12000 (secure 0 0 0) 0x7fb9580b1210 con 0x7fb964101900 2024-09-10T22:13:46.002 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:13:46.002 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":39,"btime":"2024-09-10T22:13:38:664800+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:13:38.508804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:13:46.003 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 39 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.002+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb94c0b5580 msgr2=0x7fb94c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.002+0000 7fb96b200640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb94c0b5580 0x7fb94c0b7a40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb9641073f0 tx=0x7fb954005f90 comp rx=0 tx=0).stop 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.002+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb964101900 msgr2=0x7fb96419f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.002+0000 7fb96b200640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb964101900 0x7fb96419f4d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb95800e9f0 tx=0x7fb95800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.003+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 shutdown_connections 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.003+0000 7fb96b200640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb94c0b5580 0x7fb94c0b7a40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.003+0000 7fb96b200640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb964106d40 0x7fb9641a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.003+0000 7fb96b200640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb964102220 0x7fb96419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.003+0000 7fb96b200640 1 --2- 172.21.15.90:0/446910252 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb964101900 0x7fb96419f4d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.003+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 >> 172.21.15.90:0/446910252 conn(0x7fb9640fd5b0 msgr2=0x7fb964104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.003+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 shutdown_connections 2024-09-10T22:13:46.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:46.003+0000 7fb96b200640 1 -- 172.21.15.90:0/446910252 wait complete. 2024-09-10T22:13:46.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 121706 1.25 MB/sec execute 398 sec latency 18.026 ms 2024-09-10T22:13:47.099 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 153.4 --await-for 45.0' 2024-09-10T22:13:47.100 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 153.4 --await-for 45.0 2024-09-10T22:13:47.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:46 smithi090 ceph-mon[30350]: pgmap v722: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 334 KiB/s wr, 40 op/s 2024-09-10T22:13:47.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:46 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/446910252' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:13:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:46 smithi154 ceph-mon[34495]: pgmap v722: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 334 KiB/s wr, 40 op/s 2024-09-10T22:13:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:46 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/446910252' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:13:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:46 smithi171 ceph-mon[34138]: pgmap v722: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 334 KiB/s wr, 40 op/s 2024-09-10T22:13:47.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:46 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/446910252' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:13:47.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122086 1.26 MB/sec execute 399 sec latency 47.124 ms 2024-09-10T22:13:47.531 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.194+0000 7fd935000640 1 -- 172.21.15.90:0/2834193950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 msgr2=0x7fd93010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.194+0000 7fd935000640 1 --2- 172.21.15.90:0/2834193950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 0x7fd93010ac60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd918009900 tx=0x7fd91802a410 comp rx=0 tx=0).stop 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.195+0000 7fd935000640 1 -- 172.21.15.90:0/2834193950 shutdown_connections 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.195+0000 7fd935000640 1 --2- 172.21.15.90:0/2834193950 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd93010b1a0 0x7fd9301119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.195+0000 7fd935000640 1 --2- 172.21.15.90:0/2834193950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 0x7fd93010ac60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.195+0000 7fd935000640 1 --2- 172.21.15.90:0/2834193950 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd930104e10 0x7fd9301051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.195+0000 7fd935000640 1 -- 172.21.15.90:0/2834193950 >> 172.21.15.90:0/2834193950 conn(0x7fd930100840 msgr2=0x7fd930102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.195+0000 7fd935000640 1 -- 172.21.15.90:0/2834193950 shutdown_connections 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.195+0000 7fd935000640 1 -- 172.21.15.90:0/2834193950 wait complete. 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.196+0000 7fd935000640 1 Processor -- start 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.196+0000 7fd935000640 1 -- start start 2024-09-10T22:13:48.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.196+0000 7fd935000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd930104e10 0x7fd93019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.196+0000 7fd935000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 0x7fd93019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.196+0000 7fd935000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93010b1a0 0x7fd9301a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.196+0000 7fd935000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd930114d20 con 0x7fd93010a820 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.196+0000 7fd935000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd930114ba0 con 0x7fd93010b1a0 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.196+0000 7fd935000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd930114ea0 con 0x7fd930104e10 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.196+0000 7fd926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 0x7fd93019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd92fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93010b1a0 0x7fd9301a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 0x7fd93019b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40428/0 (socket says 172.21.15.90:40428) 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd92fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93010b1a0 0x7fd9301a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:36496/0 (socket says 172.21.15.90:36496) 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd926a00640 1 -- 172.21.15.90:0/3143329033 learned_addr learned my addr 172.21.15.90:0/3143329033 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd92f400640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd930104e10 0x7fd93019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd92fe00640 1 -- 172.21.15.90:0/3143329033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd930104e10 msgr2=0x7fd93019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd92fe00640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd930104e10 0x7fd93019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd92fe00640 1 -- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 msgr2=0x7fd93019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd92fe00640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 0x7fd93019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd92fe00640 1 -- 172.21.15.90:0/3143329033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9301a11e0 con 0x7fd93010b1a0 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd926a00640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 0x7fd93019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.197+0000 7fd92fe00640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93010b1a0 0x7fd9301a0a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd92800d6b0 tx=0x7fd92800db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.198+0000 7fd92cc00640 1 -- 172.21.15.90:0/3143329033 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928017070 con 0x7fd93010b1a0 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.198+0000 7fd935000640 1 -- 172.21.15.90:0/3143329033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9301a14d0 con 0x7fd93010b1a0 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.198+0000 7fd92cc00640 1 -- 172.21.15.90:0/3143329033 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd928009cb0 con 0x7fd93010b1a0 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.198+0000 7fd92cc00640 1 -- 172.21.15.90:0/3143329033 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928003bf0 con 0x7fd93010b1a0 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.198+0000 7fd935000640 1 -- 172.21.15.90:0/3143329033 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9301a7b90 con 0x7fd93010b1a0 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.202+0000 7fd9391fd740 1 Processor -- start 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.202+0000 7fd92cc00640 1 -- 172.21.15.90:0/3143329033 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd928009820 con 0x7fd93010b1a0 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.203+0000 7fd9391fd740 1 -- start start 2024-09-10T22:13:48.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.203+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 0x5586da77c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.203+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77cc10 0x5586da788660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.203+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 0x5586da78af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.203+0000 7fd9391fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5586da6f1ca0 con 0x5586da77cc10 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.203+0000 7fd9391fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5586da6e10d0 con 0x5586da788ba0 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.203+0000 7fd9391fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5586da6e4ee0 con 0x5586da77c2f0 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.203+0000 7fd92f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77cc10 0x5586da788660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.203+0000 7fd926a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 0x5586da78af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd92f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77cc10 0x5586da788660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40434/0 (socket says 172.21.15.90:40434) 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd92f400640 1 -- 172.21.15.90:0/3623397736 learned_addr learned my addr 172.21.15.90:0/3623397736 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd926a00640 1 --2- 172.21.15.90:0/3623397736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 0x5586da77c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd92f400640 1 -- 172.21.15.90:0/3623397736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 msgr2=0x5586da77c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:48.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd92f400640 1 --2- 172.21.15.90:0/3623397736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 0x5586da77c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd92f400640 1 -- 172.21.15.90:0/3623397736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 msgr2=0x5586da78af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd92f400640 1 --2- 172.21.15.90:0/3623397736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 0x5586da78af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd92f400640 1 -- 172.21.15.90:0/3623397736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586da693400 con 0x5586da77cc10 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd92cc00640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd9140b5500 0x7fd9140b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.204+0000 7fd92f400640 1 --2- 172.21.15.90:0/3623397736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77cc10 0x5586da788660 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd920009c00 tx=0x7fd92000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8137ff5103457b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.205+0000 7fd92cc00640 1 -- 172.21.15.90:0/3143329033 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7fd9280e5cd0 con 0x7fd93010b1a0 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.205+0000 7fd926a00640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd9140b5500 0x7fd9140b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.205+0000 7fd924c00640 1 -- 172.21.15.90:0/3623397736 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920017bd0 con 0x5586da77cc10 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.205+0000 7fd924c00640 1 -- 172.21.15.90:0/3623397736 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd920003e00 con 0x5586da77cc10 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.205+0000 7fd924c00640 1 -- 172.21.15.90:0/3623397736 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920004720 con 0x5586da77cc10 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.206+0000 7fd9391fd740 1 -- 172.21.15.90:0/3623397736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77cc10 msgr2=0x5586da788660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.206+0000 7fd9391fd740 1 --2- 172.21.15.90:0/3623397736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77cc10 0x5586da788660 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd920009c00 tx=0x7fd92000e720 comp rx=0 tx=0).stop 2024-09-10T22:13:48.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.206+0000 7fd926a00640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd9140b5500 0x7fd9140b79c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd9180096c0 tx=0x7fd918005b50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.206+0000 7fd9391fd740 1 -- 172.21.15.90:0/3623397736 shutdown_connections 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.206+0000 7fd9391fd740 1 --2- 172.21.15.90:0/3623397736 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 0x5586da78af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.206+0000 7fd9391fd740 1 --2- 172.21.15.90:0/3623397736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77cc10 0x5586da788660 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.206+0000 7fd9391fd740 1 --2- 172.21.15.90:0/3623397736 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 0x5586da77c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.206+0000 7fd9391fd740 1 -- 172.21.15.90:0/3623397736 >> 172.21.15.90:0/3623397736 conn(0x5586da77a120 msgr2=0x5586da77b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.206+0000 7fd9391fd740 1 -- 172.21.15.90:0/3623397736 shutdown_connections 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.207+0000 7fd9391fd740 1 -- 172.21.15.90:0/3623397736 wait complete. 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.207+0000 7fd9391fd740 1 -- start start 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.207+0000 7fd9391fd740 1 Processor -- start 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 0x5586da785ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da788ba0 0x5586da7860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da786620 0x5586da794bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd9391fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5586da5427a0 con 0x5586da788ba0 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd9391fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5586da686a10 con 0x5586da786620 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd9391fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5586da78b820 con 0x5586da77c2f0 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd9391fd740 10 client.0 _ll_register_callbacks cb 0x5586da696be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd92fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da786620 0x5586da794bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd92f400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 0x5586da785ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd92fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da786620 0x5586da794bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:36518/0 (socket says 172.21.15.90:36518) 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.208+0000 7fd92fe00640 1 -- 172.21.15.90:0/1039850207 learned_addr learned my addr 172.21.15.90:0/1039850207 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.209+0000 7fd926a00640 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da788ba0 0x5586da7860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.209+0000 7fd92fe00640 1 -- 172.21.15.90:0/1039850207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 msgr2=0x5586da785ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.209+0000 7fd92fe00640 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 0x5586da785ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.209+0000 7fd92fe00640 1 -- 172.21.15.90:0/1039850207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da788ba0 msgr2=0x5586da7860e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:48.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.209+0000 7fd92fe00640 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da788ba0 0x5586da7860e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.209+0000 7fd92fe00640 1 -- 172.21.15.90:0/1039850207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586da44eb90 con 0x5586da786620 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.209+0000 7fd92fe00640 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da786620 0x5586da794bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9280abcb0 tx=0x7fd9280ab670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c57e87a255fe75e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.209+0000 7fd91d600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.210+0000 7fd91d600640 1 -- 172.21.15.90:0/1039850207 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928020070 con 0x5586da786620 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.210+0000 7fd9391fd740 1 -- 172.21.15.90:0/1039850207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd920003e00 con 0x5586da786620 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.210+0000 7fd91d600640 1 -- 172.21.15.90:0/1039850207 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7fd9280a7f90 con 0x5586da786620 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.210+0000 7fd91d600640 1 -- 172.21.15.90:0/1039850207 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd9280d2020 con 0x5586da786620 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.210+0000 7fd9391fd740 10 client.35726 fetch_fsmap learned FSMap version 39 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.210+0000 7fd9391fd740 1 -- 172.21.15.90:0/1039850207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586da7978a0 con 0x5586da786620 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.211+0000 7fd91d600640 1 -- 172.21.15.90:0/1039850207 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd920003e00 con 0x5586da786620 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.211+0000 7fd91d600640 1 -- 172.21.15.90:0/1039850207 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 39) ==== 8402+0+0 (secure 0 0 0) 0x7fd928057b90 con 0x5586da786620 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.211+0000 7fd9391fd740 10 client.35726 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T22:13:48.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.212+0000 7fd9391fd740 10 client.35726 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.212+0000 7fd9391fd740 20 client.35726 populate_metadata read hostname 'smithi090' 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.212+0000 7fd9391fd740 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5586da798dc0 0x5586da79b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.212+0000 7fd9391fd740 4 client.35726 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.212+0000 7fd9391fd740 1 -- 172.21.15.90:0/1039850207 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586da6f3bf0 con 0x5586da798dc0 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.212+0000 7fd92f400640 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5586da798dc0 0x5586da79b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.213+0000 7fd91d600640 10 client.35726 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.213+0000 7fd92f400640 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5586da798dc0 0x5586da79b240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29de80a876281874 server_cookie=604bec6daff00b2c in_seq=0 out_seq=0 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.214+0000 7fd91d600640 1 -- 172.21.15.90:0/1039850207 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5586da6f3bf0 con 0x5586da798dc0 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.214+0000 7fd91d600640 10 client.35726 handle_command_reply: tid=0 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.214+0000 7fd9391fd740 1 client.35726 shutdown 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.214+0000 7fd9391fd740 1 -- 172.21.15.90:0/1039850207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da786620 msgr2=0x5586da794bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.214+0000 7fd9391fd740 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da786620 0x5586da794bb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd9280abcb0 tx=0x7fd9280ab670 comp rx=0 tx=0).stop 2024-09-10T22:13:48.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd9391fd740 1 -- 172.21.15.90:0/1039850207 shutdown_connections 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd9391fd740 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5586da798dc0 0x5586da79b240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd9391fd740 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da786620 0x5586da794bb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd9391fd740 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da788ba0 0x5586da7860e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd9391fd740 1 --2- 172.21.15.90:0/1039850207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da77c2f0 0x5586da785ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd9391fd740 1 -- 172.21.15.90:0/1039850207 >> 172.21.15.90:0/1039850207 conn(0x5586da77a120 msgr2=0x5586da77fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd91d600640 0 client.35726 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd9391fd740 1 -- 172.21.15.90:0/1039850207 shutdown_connections 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd9391fd740 1 -- 172.21.15.90:0/1039850207 wait complete. 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.215+0000 7fd9391fd740 20 client.35726 trim_cache size 0 max 16384 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.236+0000 7fd9391fd740 1 Processor -- start 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd9391fd740 1 -- start start 2024-09-10T22:13:48.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77c2f0 0x5586da780360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da786620 0x5586da782db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 0x5586da7832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd9391fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301ada50 con 0x5586da77c2f0 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd9391fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301af710 con 0x5586da788ba0 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd9391fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301ae120 con 0x5586da786620 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd92f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77c2f0 0x5586da780360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd92f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77c2f0 0x5586da780360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40458/0 (socket says 172.21.15.90:40458) 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd92f400640 1 -- 172.21.15.90:0/2922139345 learned_addr learned my addr 172.21.15.90:0/2922139345 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.237+0000 7fd926a00640 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da786620 0x5586da782db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd92fe00640 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 0x5586da7832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd92f400640 1 -- 172.21.15.90:0/2922139345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da786620 msgr2=0x5586da782db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd92f400640 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da786620 0x5586da782db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd92f400640 1 -- 172.21.15.90:0/2922139345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 msgr2=0x5586da7832f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd92f400640 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 0x5586da7832f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd92f400640 1 -- 172.21.15.90:0/2922139345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586da77aba0 con 0x5586da77c2f0 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd92fe00640 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 0x5586da7832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd92f400640 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77c2f0 0x5586da780360 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x5586da77b000 tx=0x7fd920000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e1cfe694a9b856 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd91fe00640 1 -- 172.21.15.90:0/2922139345 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92003a070 con 0x5586da77c2f0 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd91fe00640 1 -- 172.21.15.90:0/2922139345 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd920003e00 con 0x5586da77c2f0 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.238+0000 7fd91fe00640 1 -- 172.21.15.90:0/2922139345 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9200052a0 con 0x5586da77c2f0 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.239+0000 7fd9391fd740 1 -- 172.21.15.90:0/2922139345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77c2f0 msgr2=0x5586da780360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.239+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77c2f0 0x5586da780360 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x5586da77b000 tx=0x7fd920000f40 comp rx=0 tx=0).stop 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.239+0000 7fd9391fd740 1 -- 172.21.15.90:0/2922139345 shutdown_connections 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.239+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da788ba0 0x5586da7832f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.239+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da786620 0x5586da782db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.239+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2922139345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da77c2f0 0x5586da780360 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.239+0000 7fd9391fd740 1 -- 172.21.15.90:0/2922139345 >> 172.21.15.90:0/2922139345 conn(0x5586da77a120 msgr2=0x5586da793aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.239+0000 7fd9391fd740 1 -- 172.21.15.90:0/2922139345 shutdown_connections 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.239+0000 7fd9391fd740 1 -- 172.21.15.90:0/2922139345 wait complete. 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.240+0000 7fd9391fd740 1 -- start start 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.240+0000 7fd9391fd740 1 Processor -- start 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da77c2f0 0x5586da78cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da786620 0x5586da78d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd9391fd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da788ba0 0x5586da78d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd9391fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301aef80 con 0x5586da786620 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd9391fd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301ae7f0 con 0x5586da77c2f0 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd9391fd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd9301aba50 con 0x5586da788ba0 2024-09-10T22:13:48.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd9391fd740 10 client.0 _ll_register_callbacks cb 0x7fd928009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:13:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da786620 0x5586da78d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd92f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da77c2f0 0x5586da78cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da786620 0x5586da78d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40470/0 (socket says 172.21.15.90:40470) 2024-09-10T22:13:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd926a00640 1 -- 172.21.15.90:0/2025997329 learned_addr learned my addr 172.21.15.90:0/2025997329 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:13:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd92fe00640 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da788ba0 0x5586da78d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd926a00640 1 -- 172.21.15.90:0/2025997329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da788ba0 msgr2=0x5586da78d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd926a00640 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da788ba0 0x5586da78d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd926a00640 1 -- 172.21.15.90:0/2025997329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da77c2f0 msgr2=0x5586da78cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:13:48.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd926a00640 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da77c2f0 0x5586da78cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd926a00640 1 -- 172.21.15.90:0/2025997329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586da77ae30 con 0x5586da786620 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.241+0000 7fd92f400640 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da77c2f0 0x5586da78cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.242+0000 7fd926a00640 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da786620 0x5586da78d050 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd9180057c0 tx=0x7fd918009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b61144ef776a659 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.242+0000 7fd91d600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.242+0000 7fd91d600640 1 -- 172.21.15.90:0/2025997329 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91804ddf0 con 0x5586da786620 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.242+0000 7fd91d600640 1 -- 172.21.15.90:0/2025997329 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9490+0+0 (secure 0 0 0) 0x7fd91803d450 con 0x5586da786620 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.242+0000 7fd9391fd740 1 -- 172.21.15.90:0/2025997329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd920003e00 con 0x5586da786620 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.243+0000 7fd91d600640 1 -- 172.21.15.90:0/2025997329 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd91803d650 con 0x5586da786620 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.243+0000 7fd9391fd740 10 client.16080 fetch_fsmap learned FSMap version 39 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.243+0000 7fd9391fd740 1 -- 172.21.15.90:0/2025997329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586da805ea0 con 0x5586da786620 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.243+0000 7fd91d600640 1 -- 172.21.15.90:0/2025997329 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd920003e00 con 0x5586da786620 2024-09-10T22:13:48.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.243+0000 7fd91d600640 1 -- 172.21.15.90:0/2025997329 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8402+0+0 (secure 0 0 0) 0x7fd918054070 con 0x5586da786620 2024-09-10T22:13:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.244+0000 7fd9391fd740 10 client.16080 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T22:13:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.244+0000 7fd9391fd740 10 client.16080 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:13:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.244+0000 7fd9391fd740 20 client.16080 populate_metadata read hostname 'smithi090' 2024-09-10T22:13:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.244+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5586da8067b0 0x5586da808bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:13:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.244+0000 7fd9391fd740 4 client.16080 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.4, "await_for": 45.0}] 2024-09-10T22:13:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.244+0000 7fd9391fd740 1 -- 172.21.15.90:0/2025997329 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.4, "await_for": 45.0}) -- 0x5586da6ef6a0 con 0x5586da8067b0 2024-09-10T22:13:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.244+0000 7fd92f400640 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5586da8067b0 0x5586da808bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:13:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.245+0000 7fd92f400640 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5586da8067b0 0x5586da808bd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c83fff6ca25ce7d server_cookie=7009cad665a220ac in_seq=0 out_seq=0 2024-09-10T22:13:48.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.245+0000 7fd91d600640 10 client.16080 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:13:48.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd91d600640 1 -- 172.21.15.90:0/2025997329 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5586da6ef6a0 con 0x5586da8067b0 2024-09-10T22:13:48.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd91d600640 10 client.16080 handle_command_reply: tid=0 2024-09-10T22:13:48.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd9391fd740 1 client.16080 shutdown 2024-09-10T22:13:48.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd9391fd740 1 -- 172.21.15.90:0/2025997329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da786620 msgr2=0x5586da78d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da786620 0x5586da78d050 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd9180057c0 tx=0x7fd918009260 comp rx=0 tx=0).stop 2024-09-10T22:13:48.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd9391fd740 1 -- 172.21.15.90:0/2025997329 shutdown_connections 2024-09-10T22:13:48.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5586da8067b0 0x5586da808bd0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5586da788ba0 0x5586da78d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd91d600640 0 client.16080 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:13:48.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5586da786620 0x5586da78d050 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd9391fd740 1 --2- 172.21.15.90:0/2025997329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5586da77c2f0 0x5586da78cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.321+0000 7fd9391fd740 1 -- 172.21.15.90:0/2025997329 >> 172.21.15.90:0/2025997329 conn(0x5586da77a120 msgr2=0x5586da7942c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.322+0000 7fd9391fd740 1 -- 172.21.15.90:0/2025997329 shutdown_connections 2024-09-10T22:13:48.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.322+0000 7fd9391fd740 1 -- 172.21.15.90:0/2025997329 wait complete. 2024-09-10T22:13:48.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.322+0000 7fd9391fd740 20 client.16080 trim_cache size 0 max 16384 2024-09-10T22:13:48.325 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:13:48.325 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 39, 2024-09-10T22:13:48.325 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:13:48.325 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 30, 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "11fb4c5c": { 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 30, 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 153.4, 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:13:48.326 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:13:48.327 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:13:48.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 -- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd9140b5500 msgr2=0x7fd9140b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd9140b5500 0x7fd9140b79c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd9180096c0 tx=0x7fd918005b50 comp rx=0 tx=0).stop 2024-09-10T22:13:48.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 -- 172.21.15.90:0/3143329033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93010b1a0 msgr2=0x7fd9301a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:13:48.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93010b1a0 0x7fd9301a0a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd92800d6b0 tx=0x7fd92800db70 comp rx=0 tx=0).stop 2024-09-10T22:13:48.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 -- 172.21.15.90:0/3143329033 shutdown_connections 2024-09-10T22:13:48.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd9140b5500 0x7fd9140b79c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd93010b1a0 0x7fd9301a0a10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd93010a820 0x7fd93019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 --2- 172.21.15.90:0/3143329033 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd930104e10 0x7fd93019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:13:48.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 -- 172.21.15.90:0/3143329033 >> 172.21.15.90:0/3143329033 conn(0x7fd930100840 msgr2=0x7fd930102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:13:48.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 -- 172.21.15.90:0/3143329033 shutdown_connections 2024-09-10T22:13:48.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:13:48.323+0000 7fd935000640 1 -- 172.21.15.90:0/3143329033 wait complete. 2024-09-10T22:13:48.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.25 MB/sec execute 400 sec latency 252.574 ms 2024-09-10T22:13:48.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:48 smithi090 ceph-mon[30350]: pgmap v723: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 354 KiB/s wr, 42 op/s 2024-09-10T22:13:49.152 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '11fb4c5c', quiesce took 4.5 seconds. Will release after: 28.9 2024-09-10T22:13:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:48 smithi154 ceph-mon[34495]: pgmap v723: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 354 KiB/s wr, 42 op/s 2024-09-10T22:13:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:48 smithi171 ceph-mon[34138]: pgmap v723: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 354 KiB/s wr, 42 op/s 2024-09-10T22:13:49.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.25 MB/sec execute 401 sec latency 1252.640 ms 2024-09-10T22:13:50.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.25 MB/sec execute 402 sec latency 2252.719 ms 2024-09-10T22:13:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:50 smithi154 ceph-mon[34495]: pgmap v724: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 610 KiB/s wr, 76 op/s 2024-09-10T22:13:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:50 smithi171 ceph-mon[34138]: pgmap v724: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 610 KiB/s wr, 76 op/s 2024-09-10T22:13:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:50 smithi090 ceph-mon[30350]: pgmap v724: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 610 KiB/s wr, 76 op/s 2024-09-10T22:13:51.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.24 MB/sec execute 403 sec latency 3252.843 ms 2024-09-10T22:13:52.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.24 MB/sec execute 404 sec latency 4252.954 ms 2024-09-10T22:13:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:52 smithi154 ceph-mon[34495]: pgmap v725: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 545 KiB/s wr, 68 op/s 2024-09-10T22:13:53.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:52 smithi171 ceph-mon[34138]: pgmap v725: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 545 KiB/s wr, 68 op/s 2024-09-10T22:13:53.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:52 smithi090 ceph-mon[30350]: pgmap v725: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 545 KiB/s wr, 68 op/s 2024-09-10T22:13:53.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.24 MB/sec execute 405 sec latency 5253.045 ms 2024-09-10T22:13:54.487 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.23 MB/sec execute 406 sec latency 6253.141 ms 2024-09-10T22:13:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:54 smithi154 ceph-mon[34495]: pgmap v726: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 573 KiB/s wr, 72 op/s 2024-09-10T22:13:55.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:54 smithi171 ceph-mon[34138]: pgmap v726: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 573 KiB/s wr, 72 op/s 2024-09-10T22:13:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:54 smithi090 ceph-mon[30350]: pgmap v726: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 573 KiB/s wr, 72 op/s 2024-09-10T22:13:55.488 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.23 MB/sec execute 407 sec latency 7253.257 ms 2024-09-10T22:13:56.488 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.23 MB/sec execute 408 sec latency 8253.378 ms 2024-09-10T22:13:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:56 smithi154 ceph-mon[34495]: pgmap v727: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 543 KiB/s wr, 69 op/s 2024-09-10T22:13:57.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:56 smithi171 ceph-mon[34138]: pgmap v727: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 543 KiB/s wr, 69 op/s 2024-09-10T22:13:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:56 smithi090 ceph-mon[30350]: pgmap v727: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 543 KiB/s wr, 69 op/s 2024-09-10T22:13:57.488 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.23 MB/sec execute 409 sec latency 9253.448 ms 2024-09-10T22:13:58.488 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.22 MB/sec execute 410 sec latency 10253.571 ms 2024-09-10T22:13:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:58 smithi154 ceph-mon[34495]: pgmap v728: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 500 KiB/s wr, 66 op/s 2024-09-10T22:13:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:58 smithi171 ceph-mon[34138]: pgmap v728: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 500 KiB/s wr, 66 op/s 2024-09-10T22:13:59.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:58 smithi090 ceph-mon[30350]: pgmap v728: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 500 KiB/s wr, 66 op/s 2024-09-10T22:13:59.488 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.22 MB/sec execute 411 sec latency 11253.726 ms 2024-09-10T22:14:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:13:59 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:14:00.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:13:59 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:14:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:13:59 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:14:00.488 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.22 MB/sec execute 412 sec latency 12253.820 ms 2024-09-10T22:14:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:00 smithi154 ceph-mon[34495]: pgmap v729: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 360 KiB/s wr, 57 op/s 2024-09-10T22:14:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:14:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:14:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:14:01.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:00 smithi171 ceph-mon[34138]: pgmap v729: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 360 KiB/s wr, 57 op/s 2024-09-10T22:14:01.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:14:01.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:14:01.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:14:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:00 smithi090 ceph-mon[30350]: pgmap v729: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 360 KiB/s wr, 57 op/s 2024-09-10T22:14:01.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:14:01.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:14:01.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:14:01.488 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.21 MB/sec execute 413 sec latency 13253.966 ms 2024-09-10T22:14:02.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:01 smithi154 ceph-mon[34495]: pgmap v730: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-09-10T22:14:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:01 smithi171 ceph-mon[34138]: pgmap v730: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-09-10T22:14:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:01 smithi090 ceph-mon[30350]: pgmap v730: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 61 KiB/s wr, 21 op/s 2024-09-10T22:14:02.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.21 MB/sec execute 414 sec latency 14254.105 ms 2024-09-10T22:14:03.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.21 MB/sec execute 415 sec latency 15254.264 ms 2024-09-10T22:14:04.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.20 MB/sec execute 416 sec latency 16254.406 ms 2024-09-10T22:14:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:04 smithi154 ceph-mon[34495]: pgmap v731: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 94 KiB/s wr, 35 op/s 2024-09-10T22:14:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:04 smithi171 ceph-mon[34138]: pgmap v731: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 94 KiB/s wr, 35 op/s 2024-09-10T22:14:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:04 smithi090 ceph-mon[30350]: pgmap v731: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 94 KiB/s wr, 35 op/s 2024-09-10T22:14:05.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.20 MB/sec execute 417 sec latency 17254.536 ms 2024-09-10T22:14:06.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.20 MB/sec execute 418 sec latency 18254.624 ms 2024-09-10T22:14:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:06 smithi090 ceph-mon[30350]: pgmap v732: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-09-10T22:14:07.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:06 smithi154 ceph-mon[34495]: pgmap v732: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-09-10T22:14:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:06 smithi171 ceph-mon[34138]: pgmap v732: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-09-10T22:14:07.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.20 MB/sec execute 419 sec latency 19254.707 ms 2024-09-10T22:14:08.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.19 MB/sec execute 420 sec latency 20254.841 ms 2024-09-10T22:14:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:08 smithi154 ceph-mon[34495]: pgmap v733: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 60 KiB/s wr, 28 op/s 2024-09-10T22:14:09.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:08 smithi171 ceph-mon[34138]: pgmap v733: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 60 KiB/s wr, 28 op/s 2024-09-10T22:14:09.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:08 smithi090 ceph-mon[30350]: pgmap v733: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 60 KiB/s wr, 28 op/s 2024-09-10T22:14:09.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.19 MB/sec execute 421 sec latency 21254.984 ms 2024-09-10T22:14:10.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:09 smithi154 ceph-mon[34495]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T22:14:10.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:09 smithi171 ceph-mon[34138]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T22:14:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:09 smithi090 ceph-mon[30350]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T22:14:10.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.19 MB/sec execute 422 sec latency 22255.089 ms 2024-09-10T22:14:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:10 smithi154 ceph-mon[34495]: pgmap v735: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-10T22:14:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:10 smithi171 ceph-mon[34138]: pgmap v735: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-10T22:14:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:10 smithi090 ceph-mon[30350]: pgmap v735: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-10T22:14:11.489 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.18 MB/sec execute 423 sec latency 23255.181 ms 2024-09-10T22:14:12.490 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.18 MB/sec execute 424 sec latency 24255.318 ms 2024-09-10T22:14:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:12 smithi154 ceph-mon[34495]: pgmap v736: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-10T22:14:13.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:12 smithi171 ceph-mon[34138]: pgmap v736: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-10T22:14:13.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:12 smithi090 ceph-mon[30350]: pgmap v736: 129 pgs: 129 active+clean; 153 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 63 KiB/s wr, 32 op/s 2024-09-10T22:14:13.490 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.18 MB/sec execute 425 sec latency 25255.460 ms 2024-09-10T22:14:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:13 smithi171 ceph-mon[34138]: pgmap v737: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T22:14:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:13 smithi090 ceph-mon[30350]: pgmap v737: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T22:14:14.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:13 smithi154 ceph-mon[34495]: pgmap v737: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-09-10T22:14:14.490 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.18 MB/sec execute 426 sec latency 26255.603 ms 2024-09-10T22:14:15.490 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.17 MB/sec execute 427 sec latency 27255.741 ms 2024-09-10T22:14:16.490 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.17 MB/sec execute 428 sec latency 28255.882 ms 2024-09-10T22:14:16.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:16 smithi154 ceph-mon[34495]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T22:14:16.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:16 smithi154 ceph-mon[34495]: pgmap v739: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-09-10T22:14:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:16 smithi171 ceph-mon[34138]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T22:14:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:16 smithi171 ceph-mon[34138]: pgmap v739: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-09-10T22:14:17.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:16 smithi090 ceph-mon[30350]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T22:14:17.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:16 smithi090 ceph-mon[30350]: pgmap v739: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 KiB/s wr, 13 op/s 2024-09-10T22:14:17.490 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.17 MB/sec execute 429 sec latency 29256.071 ms 2024-09-10T22:14:18.053 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:14:18.433 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:14:18.490 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.17 MB/sec execute 430 sec latency 30256.130 ms 2024-09-10T22:14:19.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.020+0000 7f8027200640 1 -- 172.21.15.90:0/1647357799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028104420 msgr2=0x7f802810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:19.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.020+0000 7f8027200640 1 --2- 172.21.15.90:0/1647357799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028104420 0x7f802810c910 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8010009900 tx=0x7f801002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.020+0000 7f8027200640 1 -- 172.21.15.90:0/1647357799 shutdown_connections 2024-09-10T22:14:19.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.020+0000 7f8027200640 1 --2- 172.21.15.90:0/1647357799 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f802810cf60 0x7f802810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.020+0000 7f8027200640 1 --2- 172.21.15.90:0/1647357799 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028104420 0x7f802810c910 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.020+0000 7f8027200640 1 --2- 172.21.15.90:0/1647357799 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8028103b00 0x7f8028103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.020+0000 7f8027200640 1 -- 172.21.15.90:0/1647357799 >> 172.21.15.90:0/1647357799 conn(0x7f80280fd560 msgr2=0x7f80280ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.020+0000 7f8027200640 1 -- 172.21.15.90:0/1647357799 shutdown_connections 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.020+0000 7f8027200640 1 -- 172.21.15.90:0/1647357799 wait complete. 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.021+0000 7f8027200640 1 Processor -- start 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.021+0000 7f8027200640 1 -- start start 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8027200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028103b00 0x7f80281017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8027200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028104420 0x7f8028101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8027200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f802810cf60 0x7f8028102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8027200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8028112b90 con 0x7f8028103b00 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8027200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8028112a10 con 0x7f8028104420 2024-09-10T22:14:19.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8027200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8028112d10 con 0x7f802810cf60 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8025e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028103b00 0x7f80281017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8025e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028103b00 0x7f80281017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52816/0 (socket says 172.21.15.90:52816) 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8025e00640 1 -- 172.21.15.90:0/2469210963 learned_addr learned my addr 172.21.15.90:0/2469210963 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f801d400640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028104420 0x7f8028101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8026800640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f802810cf60 0x7f8028102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8025e00640 1 -- 172.21.15.90:0/2469210963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f802810cf60 msgr2=0x7f8028102230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8025e00640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f802810cf60 0x7f8028102230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8025e00640 1 -- 172.21.15.90:0/2469210963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028104420 msgr2=0x7f8028101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8025e00640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028104420 0x7f8028101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.022+0000 7f8025e00640 1 -- 172.21.15.90:0/2469210963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80281a7500 con 0x7f8028103b00 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.023+0000 7f8025e00640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028103b00 0x7f80281017b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f801800b4a0 tx=0x7f801800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.023+0000 7f801f400640 1 -- 172.21.15.90:0/2469210963 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018014070 con 0x7f8028103b00 2024-09-10T22:14:19.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.023+0000 7f801f400640 1 -- 172.21.15.90:0/2469210963 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8018010aa0 con 0x7f8028103b00 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.023+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80281a77f0 con 0x7f8028103b00 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.023+0000 7f801f400640 1 -- 172.21.15.90:0/2469210963 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018017b70 con 0x7f8028103b00 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.023+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80281a7d80 con 0x7f8028103b00 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.024+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8028103f60 con 0x7f8028103b00 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.026+0000 7f801f400640 1 -- 172.21.15.90:0/2469210963 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8018010610 con 0x7f8028103b00 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.028+0000 7f801f400640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f800c0b5500 0x7f800c0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.028+0000 7f801f400640 1 -- 172.21.15.90:0/2469210963 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9490+0+0 (secure 0 0 0) 0x7f80180e5ce0 con 0x7f8028103b00 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.028+0000 7f801d400640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f800c0b5500 0x7f800c0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.029+0000 7f801d400640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f800c0b5500 0x7f800c0b79c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80100057c0 tx=0x7f8010005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.031+0000 7f801f400640 1 -- 172.21.15.90:0/2469210963 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f80180ac430 con 0x7f8028103b00 2024-09-10T22:14:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:18 smithi154 ceph-mon[34495]: pgmap v740: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T22:14:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:18 smithi171 ceph-mon[34138]: pgmap v740: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T22:14:19.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.250+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80281a8110 con 0x7f8028103b00 2024-09-10T22:14:19.252 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:18 smithi090 ceph-mon[30350]: pgmap v740: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-09-10T22:14:19.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.251+0000 7f801f400640 1 -- 172.21.15.90:0/2469210963 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+12000 (secure 0 0 0) 0x7f80180abd30 con 0x7f8028103b00 2024-09-10T22:14:19.253 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:14:19.253 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":39,"btime":"2024-09-10T22:13:38:664800+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:13:38.508804+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:14:19.254 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 39 2024-09-10T22:14:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f800c0b5500 msgr2=0x7f800c0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f800c0b5500 0x7f800c0b79c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80100057c0 tx=0x7f8010005e80 comp rx=0 tx=0).stop 2024-09-10T22:14:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028103b00 msgr2=0x7f80281017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028103b00 0x7f80281017b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f801800b4a0 tx=0x7f801800b960 comp rx=0 tx=0).stop 2024-09-10T22:14:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 shutdown_connections 2024-09-10T22:14:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f800c0b5500 0x7f800c0b79c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f802810cf60 0x7f8028102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8028104420 0x7f8028101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 --2- 172.21.15.90:0/2469210963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8028103b00 0x7f80281017b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 >> 172.21.15.90:0/2469210963 conn(0x7f80280fd560 msgr2=0x7f802810f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 shutdown_connections 2024-09-10T22:14:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:19.254+0000 7f8027200640 1 -- 172.21.15.90:0/2469210963 wait complete. 2024-09-10T22:14:19.490 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.16 MB/sec execute 431 sec latency 31256.221 ms 2024-09-10T22:14:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:19 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2469210963' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:14:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:19 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2469210963' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:14:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:19 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2469210963' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:14:20.302 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 11fb4c5c --release --await' 2024-09-10T22:14:20.302 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 11fb4c5c --release --await 2024-09-10T22:14:20.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122325 1.16 MB/sec execute 432 sec latency 32256.289 ms 2024-09-10T22:14:20.672 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:14:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:20 smithi090 ceph-mon[30350]: pgmap v741: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-09-10T22:14:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:20 smithi154 ceph-mon[34495]: pgmap v741: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-09-10T22:14:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:20 smithi171 ceph-mon[34138]: pgmap v741: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.8 KiB/s wr, 10 op/s 2024-09-10T22:14:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.255+0000 7f992de00640 1 -- 172.21.15.90:0/3357654676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9928109150 msgr2=0x7f992810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.255+0000 7f992de00640 1 --2- 172.21.15.90:0/3357654676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9928109150 0x7f992810f950 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f9920009ca0 tx=0x7f992002a8b0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.256+0000 7f992de00640 1 -- 172.21.15.90:0/3357654676 shutdown_connections 2024-09-10T22:14:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.256+0000 7f992de00640 1 --2- 172.21.15.90:0/3357654676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9928109150 0x7f992810f950 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.256+0000 7f992de00640 1 --2- 172.21.15.90:0/3357654676 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f99281087d0 0x7f9928108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.256+0000 7f992de00640 1 --2- 172.21.15.90:0/3357654676 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9928102dc0 0x7f99281031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.256+0000 7f992de00640 1 -- 172.21.15.90:0/3357654676 >> 172.21.15.90:0/3357654676 conn(0x7f99280fe830 msgr2=0x7f9928100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.256+0000 7f992de00640 1 -- 172.21.15.90:0/3357654676 shutdown_connections 2024-09-10T22:14:21.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.256+0000 7f992de00640 1 -- 172.21.15.90:0/3357654676 wait complete. 2024-09-10T22:14:21.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.257+0000 7f992de00640 1 Processor -- start 2024-09-10T22:14:21.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.257+0000 7f992de00640 1 -- start start 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.257+0000 7f992de00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9928102dc0 0x7f992819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.257+0000 7f992de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99281087d0 0x7f992819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.257+0000 7f992de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9928109150 0x7f99281a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.257+0000 7f992de00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9928112cc0 con 0x7f99281087d0 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.257+0000 7f992de00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9928112b40 con 0x7f9928109150 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.257+0000 7f992de00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9928112e40 con 0x7f9928102dc0 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.257+0000 7f9926000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99281087d0 0x7f992819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9926000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99281087d0 0x7f992819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52830/0 (socket says 172.21.15.90:52830) 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9926000640 1 -- 172.21.15.90:0/2076576654 learned_addr learned my addr 172.21.15.90:0/2076576654 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9927400640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9928109150 0x7f99281a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9926a00640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9928102dc0 0x7f992819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9926000640 1 -- 172.21.15.90:0/2076576654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9928102dc0 msgr2=0x7f992819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9926000640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9928102dc0 0x7f992819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9926000640 1 -- 172.21.15.90:0/2076576654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9928109150 msgr2=0x7f99281a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9926000640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9928109150 0x7f99281a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9926000640 1 -- 172.21.15.90:0/2076576654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99281a13c0 con 0x7f99281087d0 2024-09-10T22:14:21.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.258+0000 7f9926000640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99281087d0 0x7f992819b850 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f991400c920 tx=0x7f991400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.259+0000 7f992ca00640 1 -- 172.21.15.90:0/2076576654 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914007c70 con 0x7f99281087d0 2024-09-10T22:14:21.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.259+0000 7f992ca00640 1 -- 172.21.15.90:0/2076576654 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9914007e10 con 0x7f99281087d0 2024-09-10T22:14:21.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.259+0000 7f992de00640 1 -- 172.21.15.90:0/2076576654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99281a16b0 con 0x7f99281087d0 2024-09-10T22:14:21.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.259+0000 7f992ca00640 1 -- 172.21.15.90:0/2076576654 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914018710 con 0x7f99281087d0 2024-09-10T22:14:21.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.259+0000 7f992de00640 1 -- 172.21.15.90:0/2076576654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99281a7d70 con 0x7f99281087d0 2024-09-10T22:14:21.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.262+0000 7f99309c9740 1 Processor -- start 2024-09-10T22:14:21.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.262+0000 7f99309c9740 1 -- start start 2024-09-10T22:14:21.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.262+0000 7f992ca00640 1 -- 172.21.15.90:0/2076576654 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f991400f450 con 0x7f99281087d0 2024-09-10T22:14:21.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f99309c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45941ff0 0x557c459423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f99309c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45942910 0x557c4594e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f99309c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c45950c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f99309c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c458b9290 con 0x557c45942910 2024-09-10T22:14:21.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f99309c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c458a86c0 con 0x557c4594e8a0 2024-09-10T22:14:21.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f99309c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557c458ac4d0 con 0x557c45941ff0 2024-09-10T22:14:21.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45942910 0x557c4594e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9927400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45941ff0 0x557c459423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45942910 0x557c4594e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52844/0 (socket says 172.21.15.90:52844) 2024-09-10T22:14:21.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9926a00640 1 -- 172.21.15.90:0/3342286994 learned_addr learned my addr 172.21.15.90:0/3342286994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:14:21.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9926000640 1 --2- 172.21.15.90:0/3342286994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c45950c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9926a00640 1 -- 172.21.15.90:0/3342286994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45941ff0 msgr2=0x557c459423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:21.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9926a00640 1 --2- 172.21.15.90:0/3342286994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45941ff0 0x557c459423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9926a00640 1 -- 172.21.15.90:0/3342286994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 msgr2=0x557c45950c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:21.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9926a00640 1 --2- 172.21.15.90:0/3342286994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c45950c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.263+0000 7f9926a00640 1 -- 172.21.15.90:0/3342286994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c45835460 con 0x557c45942910 2024-09-10T22:14:21.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.264+0000 7f9926a00640 1 --2- 172.21.15.90:0/3342286994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45942910 0x557c4594e360 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f991800bd80 tx=0x7f99180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f70c0662949ee4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.264+0000 7f9913e00640 1 -- 172.21.15.90:0/3342286994 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918005220 con 0x557c45942910 2024-09-10T22:14:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.264+0000 7f9913e00640 1 -- 172.21.15.90:0/3342286994 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9918004c30 con 0x557c45942910 2024-09-10T22:14:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.264+0000 7f9913e00640 1 -- 172.21.15.90:0/3342286994 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99180116d0 con 0x557c45942910 2024-09-10T22:14:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.264+0000 7f992ca00640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f990c0b5450 0x7f990c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f992ca00640 1 -- 172.21.15.90:0/2076576654 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9490+0+0 (secure 0 0 0) 0x7f99140e64c0 con 0x7f99281087d0 2024-09-10T22:14:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f9927400640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f990c0b5450 0x7f990c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f99309c9740 1 -- 172.21.15.90:0/3342286994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45942910 msgr2=0x557c4594e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f99309c9740 1 --2- 172.21.15.90:0/3342286994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45942910 0x557c4594e360 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f991800bd80 tx=0x7f99180107a0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f99309c9740 1 -- 172.21.15.90:0/3342286994 shutdown_connections 2024-09-10T22:14:21.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f99309c9740 1 --2- 172.21.15.90:0/3342286994 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c45950c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f99309c9740 1 --2- 172.21.15.90:0/3342286994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45942910 0x557c4594e360 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f99309c9740 1 --2- 172.21.15.90:0/3342286994 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45941ff0 0x557c459423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f99309c9740 1 -- 172.21.15.90:0/3342286994 >> 172.21.15.90:0/3342286994 conn(0x557c4593fe20 msgr2=0x557c45940fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f99309c9740 1 -- 172.21.15.90:0/3342286994 shutdown_connections 2024-09-10T22:14:21.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f99309c9740 1 -- 172.21.15.90:0/3342286994 wait complete. 2024-09-10T22:14:21.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.265+0000 7f9927400640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f990c0b5450 0x7f990c0b7910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9920006010 tx=0x7f9920005fa0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.266+0000 7f99309c9740 1 -- start start 2024-09-10T22:14:21.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.266+0000 7f99309c9740 1 Processor -- start 2024-09-10T22:14:21.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.266+0000 7f99309c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c4594b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.266+0000 7f99309c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c4594be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.266+0000 7f99309c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c4594c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.266+0000 7f99309c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557c45709bb0 con 0x557c45941ff0 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.266+0000 7f99309c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557c458436c0 con 0x557c4594e8a0 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.266+0000 7f99309c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557c45951550 con 0x557c45942910 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f99309c9740 10 client.0 _ll_register_callbacks cb 0x557c4585e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c4594b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c4594b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52846/0 (socket says 172.21.15.90:52846) 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9926a00640 1 -- 172.21.15.90:0/1509301171 learned_addr learned my addr 172.21.15.90:0/1509301171 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9927400640 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c4594c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9926000640 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c4594be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9926a00640 1 -- 172.21.15.90:0/1509301171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 msgr2=0x557c4594be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9926a00640 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c4594be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9926a00640 1 -- 172.21.15.90:0/1509301171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 msgr2=0x557c4594c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:21.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9926a00640 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c4594c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9926a00640 1 -- 172.21.15.90:0/1509301171 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c45606b00 con 0x557c45941ff0 2024-09-10T22:14:21.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.267+0000 7f9927400640 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c4594c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:14:21.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.268+0000 7f9926a00640 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c4594b8d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9918011010 tx=0x7f99180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300f012011ea67d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.268+0000 7f9910c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:14:21.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.268+0000 7f9910c00640 1 -- 172.21.15.90:0/1509301171 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918029060 con 0x557c45941ff0 2024-09-10T22:14:21.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.268+0000 7f9910c00640 1 -- 172.21.15.90:0/1509301171 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9490+0+0 (secure 0 0 0) 0x7f9918004ff0 con 0x557c45941ff0 2024-09-10T22:14:21.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.268+0000 7f99309c9740 1 -- 172.21.15.90:0/1509301171 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9918004c30 con 0x557c45941ff0 2024-09-10T22:14:21.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.268+0000 7f9910c00640 1 -- 172.21.15.90:0/1509301171 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9918030050 con 0x557c45941ff0 2024-09-10T22:14:21.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.268+0000 7f99309c9740 10 client.16098 fetch_fsmap learned FSMap version 39 2024-09-10T22:14:21.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.268+0000 7f99309c9740 1 -- 172.21.15.90:0/1509301171 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557c4567d960 con 0x557c45941ff0 2024-09-10T22:14:21.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.269+0000 7f9910c00640 1 -- 172.21.15.90:0/1509301171 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9918004c30 con 0x557c45941ff0 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.269+0000 7f9910c00640 1 -- 172.21.15.90:0/1509301171 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8402+0+0 (secure 0 0 0) 0x7f99180119d0 con 0x557c45941ff0 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.269+0000 7f99309c9740 10 client.16098 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.269+0000 7f99309c9740 10 client.16098 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.269+0000 7f99309c9740 20 client.16098 populate_metadata read hostname 'smithi090' 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.269+0000 7f99309c9740 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557c4595e430 0x557c45960820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.269+0000 7f99309c9740 4 client.16098 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.269+0000 7f99309c9740 1 -- 172.21.15.90:0/1509301171 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c458bb1e0 con 0x557c4595e430 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.269+0000 7f9926000640 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557c4595e430 0x557c45960820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.270+0000 7f9926000640 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557c4595e430 0x557c45960820 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59ab6adea3743e4 server_cookie=825ddb5b36634a92 in_seq=0 out_seq=0 2024-09-10T22:14:21.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.270+0000 7f9910c00640 10 client.16098 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:14:21.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f9910c00640 1 -- 172.21.15.90:0/1509301171 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c458bb1e0 con 0x557c4595e430 2024-09-10T22:14:21.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f9910c00640 10 client.16098 handle_command_reply: tid=0 2024-09-10T22:14:21.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 client.16098 shutdown 2024-09-10T22:14:21.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 -- 172.21.15.90:0/1509301171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 msgr2=0x557c4594b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c4594b8d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9918011010 tx=0x7f99180048f0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 -- 172.21.15.90:0/1509301171 shutdown_connections 2024-09-10T22:14:21.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557c4595e430 0x557c45960820 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c4594c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c4594be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 --2- 172.21.15.90:0/1509301171 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c4594b8d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 -- 172.21.15.90:0/1509301171 >> 172.21.15.90:0/1509301171 conn(0x557c4593fe20 msgr2=0x557c459459c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f9910c00640 0 client.16098 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:14:21.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 -- 172.21.15.90:0/1509301171 shutdown_connections 2024-09-10T22:14:21.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.271+0000 7f99309c9740 1 -- 172.21.15.90:0/1509301171 wait complete. 2024-09-10T22:14:21.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.272+0000 7f99309c9740 20 client.16098 trim_cache size 0 max 16384 2024-09-10T22:14:21.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.289+0000 7f99309c9740 1 Processor -- start 2024-09-10T22:14:21.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.289+0000 7f99309c9740 1 -- start start 2024-09-10T22:14:21.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f99309c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c45941ff0 0x557c4593eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f99309c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c45948ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f99309c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c4594e8a0 0x557c45949020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f99309c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b2060 con 0x557c4594e8a0 2024-09-10T22:14:21.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f99309c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b3d20 con 0x557c45941ff0 2024-09-10T22:14:21.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f99309c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b2730 con 0x557c45942910 2024-09-10T22:14:21.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9927400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c4594e8a0 0x557c45949020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9927400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c4594e8a0 0x557c45949020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52856/0 (socket says 172.21.15.90:52856) 2024-09-10T22:14:21.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9927400640 1 -- 172.21.15.90:0/2199713530 learned_addr learned my addr 172.21.15.90:0/2199713530 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:14:21.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9926a00640 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c45941ff0 0x557c4593eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9926000640 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c45948ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9927400640 1 -- 172.21.15.90:0/2199713530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 msgr2=0x557c45948ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:21.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9927400640 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c45948ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9927400640 1 -- 172.21.15.90:0/2199713530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c45941ff0 msgr2=0x557c4593eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:21.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9927400640 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c45941ff0 0x557c4593eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9927400640 1 -- 172.21.15.90:0/2199713530 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c45942740 con 0x557c4594e8a0 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.290+0000 7f9926a00640 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c45941ff0 0x557c4593eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.291+0000 7f9927400640 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c4594e8a0 0x557c45949020 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9920035000 tx=0x7f9920041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83be40fc70420f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.291+0000 7f9913400640 1 -- 172.21.15.90:0/2199713530 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920030be0 con 0x557c4594e8a0 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.291+0000 7f9913400640 1 -- 172.21.15.90:0/2199713530 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9920036050 con 0x557c4594e8a0 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.291+0000 7f9913400640 1 -- 172.21.15.90:0/2199713530 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920004640 con 0x557c4594e8a0 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 -- 172.21.15.90:0/2199713530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c4594e8a0 msgr2=0x557c45949020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c4594e8a0 0x557c45949020 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9920035000 tx=0x7f9920041000 comp rx=0 tx=0).stop 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 -- 172.21.15.90:0/2199713530 shutdown_connections 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c4594e8a0 0x557c45949020 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c45948ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 --2- 172.21.15.90:0/2199713530 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c45941ff0 0x557c4593eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 -- 172.21.15.90:0/2199713530 >> 172.21.15.90:0/2199713530 conn(0x557c4593fe20 msgr2=0x557c4593e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 -- 172.21.15.90:0/2199713530 shutdown_connections 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 -- 172.21.15.90:0/2199713530 wait complete. 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.292+0000 7f99309c9740 1 -- start start 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f99309c9740 1 Processor -- start 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f99309c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c45953030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f99309c9740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c459535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f99309c9740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c459ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f99309c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b3590 con 0x557c45941ff0 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f99309c9740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b2e00 con 0x557c4594e8a0 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f99309c9740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f99281b0060 con 0x557c45942910 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f99309c9740 10 client.0 _ll_register_callbacks cb 0x7f99140e9e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f9926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c45953030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f9926a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c45953030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52860/0 (socket says 172.21.15.90:52860) 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.293+0000 7f9926a00640 1 -- 172.21.15.90:0/2707472966 learned_addr learned my addr 172.21.15.90:0/2707472966 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9926000640 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c459535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9926a00640 1 -- 172.21.15.90:0/2707472966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 msgr2=0x557c459535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9926a00640 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c459535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9926a00640 1 -- 172.21.15.90:0/2707472966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 msgr2=0x557c459ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9926a00640 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c459ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9926a00640 1 -- 172.21.15.90:0/2707472966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c459ca7f0 con 0x557c45941ff0 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9926000640 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c459535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9926a00640 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c45953030 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9918010ca0 tx=0x7f9918010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd87dde890103ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9910c00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9910c00640 1 -- 172.21.15.90:0/2707472966 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918029030 con 0x557c45941ff0 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9910c00640 1 -- 172.21.15.90:0/2707472966 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9490+0+0 (secure 0 0 0) 0x7f9918004960 con 0x557c45941ff0 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f99309c9740 1 -- 172.21.15.90:0/2707472966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9920036050 con 0x557c45941ff0 2024-09-10T22:14:21.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f9910c00640 1 -- 172.21.15.90:0/2707472966 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f991800be70 con 0x557c45941ff0 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f99309c9740 10 client.16104 fetch_fsmap learned FSMap version 39 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.294+0000 7f99309c9740 1 -- 172.21.15.90:0/2707472966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557c459cbb30 con 0x557c45941ff0 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.295+0000 7f9910c00640 1 -- 172.21.15.90:0/2707472966 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9920036050 con 0x557c45941ff0 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.295+0000 7f9910c00640 1 -- 172.21.15.90:0/2707472966 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 39) ==== 8402+0+0 (secure 0 0 0) 0x7f991801b880 con 0x557c45941ff0 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.295+0000 7f99309c9740 10 client.16104 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.295+0000 7f99309c9740 10 client.16104 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.295+0000 7f99309c9740 20 client.16104 populate_metadata read hostname 'smithi090' 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.295+0000 7f99309c9740 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557c459cc3c0 0x557c459ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.295+0000 7f99309c9740 4 client.16104 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "11fb4c5c", "release": true, "await": true}] 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.295+0000 7f99309c9740 1 -- 172.21.15.90:0/2707472966 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "11fb4c5c", "release": true, "await": true}) -- 0x557c458b6c90 con 0x557c459cc3c0 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.295+0000 7f9926000640 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557c459cc3c0 0x557c459ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.296+0000 7f9926000640 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557c459cc3c0 0x557c459ce7e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b6a722805302765 server_cookie=740167d9408e109c in_seq=0 out_seq=0 2024-09-10T22:14:21.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.296+0000 7f9910c00640 10 client.16104 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:14:21.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f9910c00640 1 -- 172.21.15.90:0/2707472966 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557c458b6c90 con 0x557c459cc3c0 2024-09-10T22:14:21.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f9910c00640 10 client.16104 handle_command_reply: tid=0 2024-09-10T22:14:21.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 client.16104 shutdown 2024-09-10T22:14:21.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 -- 172.21.15.90:0/2707472966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 msgr2=0x557c45953030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c45953030 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9918010ca0 tx=0x7f9918010de0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 -- 172.21.15.90:0/2707472966 shutdown_connections 2024-09-10T22:14:21.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557c459cc3c0 0x557c459ce7e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557c4594e8a0 0x557c459ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557c45942910 0x557c459535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 --2- 172.21.15.90:0/2707472966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557c45941ff0 0x557c45953030 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f9910c00640 0 client.16104 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:14:21.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 -- 172.21.15.90:0/2707472966 >> 172.21.15.90:0/2707472966 conn(0x557c4593fe20 msgr2=0x557c4593edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 -- 172.21.15.90:0/2707472966 shutdown_connections 2024-09-10T22:14:21.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 1 -- 172.21.15.90:0/2707472966 wait complete. 2024-09-10T22:14:21.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.305+0000 7f99309c9740 20 client.16104 trim_cache size 0 max 16384 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 39, 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 32, 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "11fb4c5c": { 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 32, 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:14:21.309 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 153.4, 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:14:21.310 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:14:21.311 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:14:21.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 -- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f990c0b5450 msgr2=0x7f990c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f990c0b5450 0x7f990c0b7910 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9920006010 tx=0x7f9920005fa0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 -- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99281087d0 msgr2=0x7f992819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:14:21.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99281087d0 0x7f992819b850 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f991400c920 tx=0x7f991400cde0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 -- 172.21.15.90:0/2076576654 shutdown_connections 2024-09-10T22:14:21.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f990c0b5450 0x7f990c0b7910 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9928109150 0x7f99281a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99281087d0 0x7f992819b850 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 --2- 172.21.15.90:0/2076576654 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9928102dc0 0x7f992819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:14:21.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 -- 172.21.15.90:0/2076576654 >> 172.21.15.90:0/2076576654 conn(0x7f99280fe830 msgr2=0x7f9928075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:14:21.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 -- 172.21.15.90:0/2076576654 shutdown_connections 2024-09-10T22:14:21.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:14:21.306+0000 7f992de00640 1 -- 172.21.15.90:0/2076576654 wait complete. 2024-09-10T22:14:21.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122408 1.16 MB/sec execute 433 sec latency 33074.323 ms 2024-09-10T22:14:22.235 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '11fb4c5c', total seconds elapsed: 37.6 2024-09-10T22:14:22.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 122782 1.16 MB/sec execute 434 sec latency 45.621 ms 2024-09-10T22:14:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:22 smithi154 ceph-mon[34495]: pgmap v742: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-10T22:14:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:22 smithi171 ceph-mon[34138]: pgmap v742: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-10T22:14:23.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:22 smithi090 ceph-mon[30350]: pgmap v742: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 4.7 KiB/s wr, 10 op/s 2024-09-10T22:14:23.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 123107 1.16 MB/sec execute 435 sec latency 18.906 ms 2024-09-10T22:14:24.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 123608 1.17 MB/sec execute 436 sec latency 15.745 ms 2024-09-10T22:14:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:24 smithi154 ceph-mon[34495]: pgmap v743: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-10T22:14:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:24 smithi171 ceph-mon[34138]: pgmap v743: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-10T22:14:25.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:24 smithi090 ceph-mon[30350]: pgmap v743: 129 pgs: 129 active+clean; 152 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-09-10T22:14:25.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 123961 1.17 MB/sec execute 437 sec latency 19.658 ms 2024-09-10T22:14:26.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124249 1.17 MB/sec execute 438 sec latency 253.910 ms 2024-09-10T22:14:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:26 smithi154 ceph-mon[34495]: pgmap v744: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 309 KiB/s wr, 28 op/s 2024-09-10T22:14:27.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:26 smithi171 ceph-mon[34138]: pgmap v744: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 309 KiB/s wr, 28 op/s 2024-09-10T22:14:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:26 smithi090 ceph-mon[30350]: pgmap v744: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 309 KiB/s wr, 28 op/s 2024-09-10T22:14:27.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124249 1.17 MB/sec execute 439 sec latency 1254.051 ms 2024-09-10T22:14:28.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124316 1.17 MB/sec execute 440 sec latency 1386.887 ms 2024-09-10T22:14:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:28 smithi154 ceph-mon[34495]: pgmap v745: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 402 KiB/s wr, 33 op/s 2024-09-10T22:14:29.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:28 smithi171 ceph-mon[34138]: pgmap v745: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 402 KiB/s wr, 33 op/s 2024-09-10T22:14:29.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:28 smithi090 ceph-mon[30350]: pgmap v745: 129 pgs: 129 active+clean; 156 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 402 KiB/s wr, 33 op/s 2024-09-10T22:14:29.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124316 1.16 MB/sec execute 441 sec latency 1547.819 ms 2024-09-10T22:14:30.491 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124316 1.16 MB/sec execute 442 sec latency 2547.907 ms 2024-09-10T22:14:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:30 smithi154 ceph-mon[34495]: pgmap v746: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 593 KiB/s wr, 47 op/s 2024-09-10T22:14:31.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:30 smithi171 ceph-mon[34138]: pgmap v746: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 593 KiB/s wr, 47 op/s 2024-09-10T22:14:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:30 smithi090 ceph-mon[30350]: pgmap v746: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 593 KiB/s wr, 47 op/s 2024-09-10T22:14:31.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124316 1.16 MB/sec execute 443 sec latency 3548.031 ms 2024-09-10T22:14:32.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124316 1.16 MB/sec execute 444 sec latency 4548.167 ms 2024-09-10T22:14:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:32 smithi154 ceph-mon[34495]: pgmap v747: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 593 KiB/s wr, 47 op/s 2024-09-10T22:14:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:32 smithi171 ceph-mon[34138]: pgmap v747: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 593 KiB/s wr, 47 op/s 2024-09-10T22:14:33.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:32 smithi090 ceph-mon[30350]: pgmap v747: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 593 KiB/s wr, 47 op/s 2024-09-10T22:14:33.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124489 1.16 MB/sec execute 445 sec latency 4680.507 ms 2024-09-10T22:14:34.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:33 smithi154 ceph-mon[34495]: pgmap v748: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 599 KiB/s wr, 49 op/s 2024-09-10T22:14:34.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:33 smithi171 ceph-mon[34138]: pgmap v748: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 599 KiB/s wr, 49 op/s 2024-09-10T22:14:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:33 smithi090 ceph-mon[30350]: pgmap v748: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 599 KiB/s wr, 49 op/s 2024-09-10T22:14:34.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124659 1.15 MB/sec execute 446 sec latency 21.258 ms 2024-09-10T22:14:35.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124929 1.15 MB/sec execute 447 sec latency 21.197 ms 2024-09-10T22:14:36.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 125286 1.15 MB/sec execute 448 sec latency 18.754 ms 2024-09-10T22:14:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:36 smithi154 ceph-mon[34495]: pgmap v749: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 779 KiB/s wr, 59 op/s 2024-09-10T22:14:37.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:36 smithi171 ceph-mon[34138]: pgmap v749: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 779 KiB/s wr, 59 op/s 2024-09-10T22:14:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:36 smithi090 ceph-mon[30350]: pgmap v749: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 779 KiB/s wr, 59 op/s 2024-09-10T22:14:37.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 125660 1.16 MB/sec execute 449 sec latency 22.558 ms 2024-09-10T22:14:38.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 126017 1.16 MB/sec execute 450 sec latency 23.975 ms 2024-09-10T22:14:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:38 smithi090 ceph-mon[30350]: pgmap v750: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 646 KiB/s wr, 56 op/s 2024-09-10T22:14:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:38 smithi154 ceph-mon[34495]: pgmap v750: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 646 KiB/s wr, 56 op/s 2024-09-10T22:14:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:38 smithi171 ceph-mon[34138]: pgmap v750: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 646 KiB/s wr, 56 op/s 2024-09-10T22:14:39.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 126401 1.15 MB/sec execute 451 sec latency 27.444 ms 2024-09-10T22:14:40.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 126740 1.16 MB/sec execute 452 sec latency 21.961 ms 2024-09-10T22:14:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:40 smithi154 ceph-mon[34495]: pgmap v751: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 715 KiB/s wr, 74 op/s 2024-09-10T22:14:41.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:40 smithi171 ceph-mon[34138]: pgmap v751: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 715 KiB/s wr, 74 op/s 2024-09-10T22:14:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:40 smithi090 ceph-mon[30350]: pgmap v751: 129 pgs: 129 active+clean; 159 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 715 KiB/s wr, 74 op/s 2024-09-10T22:14:41.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 127196 1.16 MB/sec execute 453 sec latency 35.261 ms 2024-09-10T22:14:42.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 127519 1.17 MB/sec execute 454 sec latency 20.801 ms 2024-09-10T22:14:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:42 smithi154 ceph-mon[34495]: pgmap v752: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 642 KiB/s wr, 74 op/s 2024-09-10T22:14:43.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:42 smithi171 ceph-mon[34138]: pgmap v752: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 642 KiB/s wr, 74 op/s 2024-09-10T22:14:43.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:42 smithi090 ceph-mon[30350]: pgmap v752: 129 pgs: 129 active+clean; 160 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 642 KiB/s wr, 74 op/s 2024-09-10T22:14:43.492 INFO:tasks.workunit.client.0.smithi090.stdout: 1 127800 1.17 MB/sec execute 455 sec latency 372.362 ms 2024-09-10T22:14:44.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 127800 1.17 MB/sec execute 456 sec latency 1372.478 ms 2024-09-10T22:14:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:44 smithi154 ceph-mon[34495]: pgmap v753: 129 pgs: 129 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 951 KiB/s wr, 86 op/s 2024-09-10T22:14:45.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:44 smithi171 ceph-mon[34138]: pgmap v753: 129 pgs: 129 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 951 KiB/s wr, 86 op/s 2024-09-10T22:14:45.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:44 smithi090 ceph-mon[30350]: pgmap v753: 129 pgs: 129 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 951 KiB/s wr, 86 op/s 2024-09-10T22:14:45.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 127800 1.17 MB/sec execute 457 sec latency 2372.586 ms 2024-09-10T22:14:46.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 127800 1.16 MB/sec execute 458 sec latency 3372.742 ms 2024-09-10T22:14:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:46 smithi154 ceph-mon[34495]: pgmap v754: 129 pgs: 129 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-09-10T22:14:47.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:46 smithi171 ceph-mon[34138]: pgmap v754: 129 pgs: 129 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-09-10T22:14:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:46 smithi090 ceph-mon[30350]: pgmap v754: 129 pgs: 129 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-09-10T22:14:47.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 127800 1.16 MB/sec execute 459 sec latency 4372.883 ms 2024-09-10T22:14:48.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 127983 1.16 MB/sec execute 460 sec latency 4506.649 ms 2024-09-10T22:14:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:48 smithi090 ceph-mon[30350]: pgmap v755: 129 pgs: 129 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 935 KiB/s wr, 85 op/s 2024-09-10T22:14:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:48 smithi154 ceph-mon[34495]: pgmap v755: 129 pgs: 129 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 935 KiB/s wr, 85 op/s 2024-09-10T22:14:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:48 smithi171 ceph-mon[34138]: pgmap v755: 129 pgs: 129 active+clean; 163 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 935 KiB/s wr, 85 op/s 2024-09-10T22:14:49.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 128155 1.16 MB/sec execute 461 sec latency 17.151 ms 2024-09-10T22:14:50.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 128346 1.16 MB/sec execute 462 sec latency 21.826 ms 2024-09-10T22:14:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:50 smithi154 ceph-mon[34495]: pgmap v756: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 915 KiB/s wr, 81 op/s 2024-09-10T22:14:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:50 smithi171 ceph-mon[34138]: pgmap v756: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 915 KiB/s wr, 81 op/s 2024-09-10T22:14:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:50 smithi090 ceph-mon[30350]: pgmap v756: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 915 KiB/s wr, 81 op/s 2024-09-10T22:14:51.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 128702 1.16 MB/sec execute 463 sec latency 20.851 ms 2024-09-10T22:14:52.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 129052 1.16 MB/sec execute 464 sec latency 19.456 ms 2024-09-10T22:14:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:52 smithi154 ceph-mon[34495]: pgmap v757: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 862 KiB/s wr, 65 op/s 2024-09-10T22:14:53.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:52 smithi171 ceph-mon[34138]: pgmap v757: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 862 KiB/s wr, 65 op/s 2024-09-10T22:14:53.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:52 smithi090 ceph-mon[30350]: pgmap v757: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 862 KiB/s wr, 65 op/s 2024-09-10T22:14:53.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 129388 1.16 MB/sec execute 465 sec latency 25.511 ms 2024-09-10T22:14:54.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:53 smithi171 ceph-mon[34138]: pgmap v758: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 892 KiB/s wr, 71 op/s 2024-09-10T22:14:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:53 smithi090 ceph-mon[30350]: pgmap v758: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 892 KiB/s wr, 71 op/s 2024-09-10T22:14:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:53 smithi154 ceph-mon[34495]: pgmap v758: 129 pgs: 129 active+clean; 164 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 892 KiB/s wr, 71 op/s 2024-09-10T22:14:54.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 129791 1.16 MB/sec execute 466 sec latency 24.671 ms 2024-09-10T22:14:55.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 130147 1.16 MB/sec execute 467 sec latency 25.178 ms 2024-09-10T22:14:56.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 130606 1.17 MB/sec execute 468 sec latency 19.175 ms 2024-09-10T22:14:57.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:56 smithi154 ceph-mon[34495]: pgmap v759: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 804 KiB/s wr, 84 op/s 2024-09-10T22:14:57.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:56 smithi171 ceph-mon[34138]: pgmap v759: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 804 KiB/s wr, 84 op/s 2024-09-10T22:14:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:56 smithi090 ceph-mon[30350]: pgmap v759: 129 pgs: 129 active+clean; 165 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 804 KiB/s wr, 84 op/s 2024-09-10T22:14:57.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131032 1.17 MB/sec execute 469 sec latency 19.203 ms 2024-09-10T22:14:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:57 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:14:58.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:57 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:14:58.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:57 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:14:58.493 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131295 1.17 MB/sec execute 470 sec latency 21.447 ms 2024-09-10T22:14:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:14:58 smithi154 ceph-mon[34495]: pgmap v760: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 744 KiB/s wr, 79 op/s 2024-09-10T22:14:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:14:58 smithi171 ceph-mon[34138]: pgmap v760: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 744 KiB/s wr, 79 op/s 2024-09-10T22:14:59.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:14:58 smithi090 ceph-mon[30350]: pgmap v760: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 375 KiB/s rd, 744 KiB/s wr, 79 op/s 2024-09-10T22:14:59.494 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131345 1.17 MB/sec execute 471 sec latency 923.221 ms 2024-09-10T22:15:00.494 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131345 1.17 MB/sec execute 472 sec latency 1923.346 ms 2024-09-10T22:15:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:00 smithi154 ceph-mon[34495]: pgmap v761: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-10T22:15:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:00 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:15:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:00 smithi171 ceph-mon[34138]: pgmap v761: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-10T22:15:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:00 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:15:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:00 smithi090 ceph-mon[30350]: pgmap v761: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-10T22:15:01.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:00 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:15:01.494 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131345 1.17 MB/sec execute 473 sec latency 2923.422 ms 2024-09-10T22:15:02.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:01 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:15:02.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:01 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:15:02.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:01 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:15:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:01 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:15:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:01 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:15:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:01 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:15:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:01 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:15:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:01 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:15:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:01 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:15:02.494 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131345 1.16 MB/sec execute 474 sec latency 3923.555 ms 2024-09-10T22:15:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:02 smithi154 ceph-mon[34495]: pgmap v762: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.0 MiB/s wr, 94 op/s 2024-09-10T22:15:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:02 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:15:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:02 smithi171 ceph-mon[34138]: pgmap v762: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.0 MiB/s wr, 94 op/s 2024-09-10T22:15:03.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:02 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:15:03.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:02 smithi090 ceph-mon[30350]: pgmap v762: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 1.0 MiB/s wr, 94 op/s 2024-09-10T22:15:03.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:02 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:15:03.494 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131412 1.16 MB/sec execute 475 sec latency 4054.249 ms 2024-09-10T22:15:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:03 smithi154 ceph-mon[34495]: pgmap v763: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 969 KiB/s wr, 102 op/s 2024-09-10T22:15:04.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:03 smithi171 ceph-mon[34138]: pgmap v763: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 969 KiB/s wr, 102 op/s 2024-09-10T22:15:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:03 smithi090 ceph-mon[30350]: pgmap v763: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 969 KiB/s wr, 102 op/s 2024-09-10T22:15:04.494 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131412 1.16 MB/sec execute 476 sec latency 1527.026 ms 2024-09-10T22:15:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:04 smithi154 ceph-mon[34495]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T22:15:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:04 smithi171 ceph-mon[34138]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T22:15:05.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:04 smithi090 ceph-mon[30350]: osdmap e101: 12 total, 12 up, 12 in 2024-09-10T22:15:05.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131412 1.16 MB/sec execute 477 sec latency 2527.184 ms 2024-09-10T22:15:06.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:05 smithi154 ceph-mon[34495]: pgmap v765: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 765 KiB/s wr, 77 op/s 2024-09-10T22:15:06.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:05 smithi171 ceph-mon[34138]: pgmap v765: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 765 KiB/s wr, 77 op/s 2024-09-10T22:15:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:05 smithi090 ceph-mon[30350]: pgmap v765: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 765 KiB/s wr, 77 op/s 2024-09-10T22:15:06.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131412 1.15 MB/sec execute 478 sec latency 3527.305 ms 2024-09-10T22:15:07.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131412 1.15 MB/sec execute 479 sec latency 4527.455 ms 2024-09-10T22:15:08.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131583 1.15 MB/sec execute 480 sec latency 4658.454 ms 2024-09-10T22:15:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:08 smithi154 ceph-mon[34495]: pgmap v766: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 624 KiB/s wr, 66 op/s 2024-09-10T22:15:09.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:08 smithi171 ceph-mon[34138]: pgmap v766: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 624 KiB/s wr, 66 op/s 2024-09-10T22:15:09.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:08 smithi090 ceph-mon[30350]: pgmap v766: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 624 KiB/s wr, 66 op/s 2024-09-10T22:15:09.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 131749 1.15 MB/sec execute 481 sec latency 21.109 ms 2024-09-10T22:15:10.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 132002 1.15 MB/sec execute 482 sec latency 22.956 ms 2024-09-10T22:15:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:10 smithi154 ceph-mon[34495]: pgmap v767: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 394 KiB/s wr, 56 op/s 2024-09-10T22:15:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:10 smithi154 ceph-mon[34495]: osdmap e102: 12 total, 12 up, 12 in 2024-09-10T22:15:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:10 smithi171 ceph-mon[34138]: pgmap v767: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 394 KiB/s wr, 56 op/s 2024-09-10T22:15:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:10 smithi171 ceph-mon[34138]: osdmap e102: 12 total, 12 up, 12 in 2024-09-10T22:15:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:10 smithi090 ceph-mon[30350]: pgmap v767: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 394 KiB/s wr, 56 op/s 2024-09-10T22:15:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:10 smithi090 ceph-mon[30350]: osdmap e102: 12 total, 12 up, 12 in 2024-09-10T22:15:11.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 132359 1.15 MB/sec execute 483 sec latency 17.946 ms 2024-09-10T22:15:12.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 132661 1.15 MB/sec execute 484 sec latency 269.858 ms 2024-09-10T22:15:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:12 smithi154 ceph-mon[34495]: pgmap v769: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 417 KiB/s wr, 42 op/s 2024-09-10T22:15:13.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:12 smithi171 ceph-mon[34138]: pgmap v769: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 417 KiB/s wr, 42 op/s 2024-09-10T22:15:13.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:12 smithi090 ceph-mon[30350]: pgmap v769: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 417 KiB/s wr, 42 op/s 2024-09-10T22:15:13.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 132850 1.15 MB/sec execute 485 sec latency 593.697 ms 2024-09-10T22:15:14.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 133199 1.15 MB/sec execute 486 sec latency 24.754 ms 2024-09-10T22:15:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:14 smithi154 ceph-mon[34495]: pgmap v770: 129 pgs: 129 active+clean; 170 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 520 KiB/s wr, 56 op/s 2024-09-10T22:15:15.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:14 smithi171 ceph-mon[34138]: pgmap v770: 129 pgs: 129 active+clean; 170 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 520 KiB/s wr, 56 op/s 2024-09-10T22:15:15.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:14 smithi090 ceph-mon[30350]: pgmap v770: 129 pgs: 129 active+clean; 170 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 520 KiB/s wr, 56 op/s 2024-09-10T22:15:15.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 133619 1.15 MB/sec execute 487 sec latency 27.204 ms 2024-09-10T22:15:16.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 133886 1.15 MB/sec execute 488 sec latency 17.927 ms 2024-09-10T22:15:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:16 smithi154 ceph-mon[34495]: pgmap v771: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 765 KiB/s wr, 79 op/s 2024-09-10T22:15:17.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:16 smithi171 ceph-mon[34138]: pgmap v771: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 765 KiB/s wr, 79 op/s 2024-09-10T22:15:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:16 smithi090 ceph-mon[30350]: pgmap v771: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 765 KiB/s wr, 79 op/s 2024-09-10T22:15:17.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 134484 1.16 MB/sec execute 489 sec latency 20.173 ms 2024-09-10T22:15:18.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 134764 1.16 MB/sec execute 490 sec latency 21.224 ms 2024-09-10T22:15:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:18 smithi154 ceph-mon[34495]: pgmap v772: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 827 KiB/s wr, 87 op/s 2024-09-10T22:15:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:18 smithi171 ceph-mon[34138]: pgmap v772: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 827 KiB/s wr, 87 op/s 2024-09-10T22:15:19.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:18 smithi090 ceph-mon[30350]: pgmap v772: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 827 KiB/s wr, 87 op/s 2024-09-10T22:15:19.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 134772 1.16 MB/sec execute 491 sec latency 981.229 ms 2024-09-10T22:15:20.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 134772 1.16 MB/sec execute 492 sec latency 1981.298 ms 2024-09-10T22:15:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:20 smithi154 ceph-mon[34495]: pgmap v773: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 991 KiB/s wr, 90 op/s 2024-09-10T22:15:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:20 smithi171 ceph-mon[34138]: pgmap v773: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 991 KiB/s wr, 90 op/s 2024-09-10T22:15:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:20 smithi090 ceph-mon[30350]: pgmap v773: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 991 KiB/s wr, 90 op/s 2024-09-10T22:15:21.495 INFO:tasks.workunit.client.0.smithi090.stdout: 1 134914 1.16 MB/sec execute 493 sec latency 2452.677 ms 2024-09-10T22:15:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:21 smithi171 ceph-mon[34138]: pgmap v774: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 997 KiB/s wr, 90 op/s 2024-09-10T22:15:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:21 smithi090 ceph-mon[30350]: pgmap v774: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 997 KiB/s wr, 90 op/s 2024-09-10T22:15:22.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:21 smithi154 ceph-mon[34495]: pgmap v774: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 997 KiB/s wr, 90 op/s 2024-09-10T22:15:22.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 134958 1.16 MB/sec execute 494 sec latency 836.641 ms 2024-09-10T22:15:23.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 135130 1.16 MB/sec execute 495 sec latency 969.149 ms 2024-09-10T22:15:24.264 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T22:15:24.264 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.4 2024-09-10T22:15:24.265 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:15:24.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 135301 1.15 MB/sec execute 496 sec latency 18.629 ms 2024-09-10T22:15:24.680 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:15:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:24 smithi090 ceph-mon[30350]: pgmap v775: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 964 KiB/s wr, 86 op/s 2024-09-10T22:15:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:24 smithi154 ceph-mon[34495]: pgmap v775: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 964 KiB/s wr, 86 op/s 2024-09-10T22:15:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:24 smithi171 ceph-mon[34138]: pgmap v775: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 964 KiB/s wr, 86 op/s 2024-09-10T22:15:25.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.367+0000 7f14e9c00640 1 -- 172.21.15.90:0/2412139065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14e4104fa0 msgr2=0x7f14e4105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.367+0000 7f14e9c00640 1 --2- 172.21.15.90:0/2412139065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14e4104fa0 0x7f14e4105380 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009900 tx=0x7f14d402a410 comp rx=0 tx=0).stop 2024-09-10T22:15:25.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.367+0000 7f14e9c00640 1 -- 172.21.15.90:0/2412139065 shutdown_connections 2024-09-10T22:15:25.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.367+0000 7f14e9c00640 1 --2- 172.21.15.90:0/2412139065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14e410b330 0x7f14e4111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.367+0000 7f14e9c00640 1 --2- 172.21.15.90:0/2412139065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14e410a9b0 0x7f14e410adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.367+0000 7f14e9c00640 1 --2- 172.21.15.90:0/2412139065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14e4104fa0 0x7f14e4105380 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.367+0000 7f14e9c00640 1 -- 172.21.15.90:0/2412139065 >> 172.21.15.90:0/2412139065 conn(0x7f14e4100a50 msgr2=0x7f14e4102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.367+0000 7f14e9c00640 1 -- 172.21.15.90:0/2412139065 shutdown_connections 2024-09-10T22:15:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.367+0000 7f14e9c00640 1 -- 172.21.15.90:0/2412139065 wait complete. 2024-09-10T22:15:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.369+0000 7f14e9c00640 1 Processor -- start 2024-09-10T22:15:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.369+0000 7f14e9c00640 1 -- start start 2024-09-10T22:15:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.369+0000 7f14e9c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14e4104fa0 0x7f14e419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.369+0000 7f14e9c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14e410a9b0 0x7f14e419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.369+0000 7f14e9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14e410b330 0x7f14e41a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.369+0000 7f14e9c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4114e10 con 0x7f14e410b330 2024-09-10T22:15:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.369+0000 7f14e9c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4114c90 con 0x7f14e4104fa0 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.369+0000 7f14e9c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f14e4114f90 con 0x7f14e410a9b0 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e2000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14e410a9b0 0x7f14e419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14e410b330 0x7f14e41a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e2000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14e410a9b0 0x7f14e419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:45394/0 (socket says 172.21.15.90:45394) 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e2000640 1 -- 172.21.15.90:0/1835925147 learned_addr learned my addr 172.21.15.90:0/1835925147 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e2a00640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14e4104fa0 0x7f14e419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e2000640 1 -- 172.21.15.90:0/1835925147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14e4104fa0 msgr2=0x7f14e419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e2000640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14e4104fa0 0x7f14e419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e2000640 1 -- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14e410b330 msgr2=0x7f14e41a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e2000640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14e410b330 0x7f14e41a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e2000640 1 -- 172.21.15.90:0/1835925147 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e41a1470 con 0x7f14e410a9b0 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.370+0000 7f14e3400640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14e410b330 0x7f14e41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:15:25.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.371+0000 7f14e2000640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14e410a9b0 0x7f14e419b900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14d000e9f0 tx=0x7f14d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:25.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.371+0000 7f14db400640 1 -- 172.21.15.90:0/1835925147 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d000cd80 con 0x7f14e410a9b0 2024-09-10T22:15:25.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.371+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e41a1760 con 0x7f14e410a9b0 2024-09-10T22:15:25.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.371+0000 7f14db400640 1 -- 172.21.15.90:0/1835925147 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14d00108c0 con 0x7f14e410a9b0 2024-09-10T22:15:25.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.371+0000 7f14db400640 1 -- 172.21.15.90:0/1835925147 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d00187c0 con 0x7f14e410a9b0 2024-09-10T22:15:25.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.371+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e41a7ff0 con 0x7f14e410a9b0 2024-09-10T22:15:25.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.372+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e4106250 con 0x7f14e410a9b0 2024-09-10T22:15:25.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.375+0000 7f14db400640 1 -- 172.21.15.90:0/1835925147 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f14d0010430 con 0x7f14e410a9b0 2024-09-10T22:15:25.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.377+0000 7f14db400640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f14c80b5450 0x7f14c80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:25.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.377+0000 7f14db400640 1 -- 172.21.15.90:0/1835925147 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7f14d00e6800 con 0x7f14e410a9b0 2024-09-10T22:15:25.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.377+0000 7f14e2a00640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f14c80b5450 0x7f14c80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:25.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.378+0000 7f14e2a00640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f14c80b5450 0x7f14c80b7910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f14d40057c0 tx=0x7f14d40058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:25.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.378+0000 7f14db400640 1 -- 172.21.15.90:0/1835925147 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f14d00ac900 con 0x7f14e410a9b0 2024-09-10T22:15:25.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 135576 1.15 MB/sec execute 497 sec latency 22.418 ms 2024-09-10T22:15:25.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.648+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14e4105380 con 0x7f14e410a9b0 2024-09-10T22:15:25.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.649+0000 7f14db400640 1 -- 172.21.15.90:0/1835925147 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+12003 (secure 0 0 0) 0x7f14d00b1330 con 0x7f14e410a9b0 2024-09-10T22:15:25.651 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:15:25.652 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"btime":"2024-09-10T22:15:01:892115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:15:00.940095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:15:25.652 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 41 2024-09-10T22:15:25.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.651+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f14c80b5450 msgr2=0x7f14c80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.651+0000 7f14e9c00640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f14c80b5450 0x7f14c80b7910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f14d40057c0 tx=0x7f14d40058d0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.651+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14e410a9b0 msgr2=0x7f14e419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.651+0000 7f14e9c00640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14e410a9b0 0x7f14e419b900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f14d000e9f0 tx=0x7f14d000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.651+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 shutdown_connections 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.651+0000 7f14e9c00640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f14c80b5450 0x7f14c80b7910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.651+0000 7f14e9c00640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14e410b330 0x7f14e41a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.651+0000 7f14e9c00640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f14e410a9b0 0x7f14e419b900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.651+0000 7f14e9c00640 1 --2- 172.21.15.90:0/1835925147 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f14e4104fa0 0x7f14e419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.652+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 >> 172.21.15.90:0/1835925147 conn(0x7f14e4100a50 msgr2=0x7f14e4101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.652+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 shutdown_connections 2024-09-10T22:15:25.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:25.652+0000 7f14e9c00640 1 -- 172.21.15.90:0/1835925147 wait complete. 2024-09-10T22:15:26.069 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:25 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1835925147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:15:26.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:25 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1835925147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:15:26.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:25 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1835925147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:15:26.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 135924 1.15 MB/sec execute 498 sec latency 24.089 ms 2024-09-10T22:15:26.523 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0' 2024-09-10T22:15:26.523 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0 2024-09-10T22:15:26.950 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:15:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:26 smithi154 ceph-mon[34495]: pgmap v776: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 997 KiB/s wr, 91 op/s 2024-09-10T22:15:27.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:26 smithi171 ceph-mon[34138]: pgmap v776: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 997 KiB/s wr, 91 op/s 2024-09-10T22:15:27.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:26 smithi090 ceph-mon[30350]: pgmap v776: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 997 KiB/s wr, 91 op/s 2024-09-10T22:15:27.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136289 1.16 MB/sec execute 499 sec latency 25.492 ms 2024-09-10T22:15:27.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.594+0000 7fac6d600640 1 -- 172.21.15.90:0/1248447554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810b1a0 msgr2=0x7fac681119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.594+0000 7fac6d600640 1 --2- 172.21.15.90:0/1248447554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810b1a0 0x7fac681119a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fac60009ca0 tx=0x7fac6002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.595+0000 7fac6d600640 1 -- 172.21.15.90:0/1248447554 shutdown_connections 2024-09-10T22:15:27.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.595+0000 7fac6d600640 1 --2- 172.21.15.90:0/1248447554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810b1a0 0x7fac681119a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.595+0000 7fac6d600640 1 --2- 172.21.15.90:0/1248447554 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac6810a820 0x7fac6810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.595+0000 7fac6d600640 1 --2- 172.21.15.90:0/1248447554 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac68104e10 0x7fac681051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.595+0000 7fac6d600640 1 -- 172.21.15.90:0/1248447554 >> 172.21.15.90:0/1248447554 conn(0x7fac68100840 msgr2=0x7fac68102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:27.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.595+0000 7fac6d600640 1 -- 172.21.15.90:0/1248447554 shutdown_connections 2024-09-10T22:15:27.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.595+0000 7fac6d600640 1 -- 172.21.15.90:0/1248447554 wait complete. 2024-09-10T22:15:27.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.596+0000 7fac6d600640 1 Processor -- start 2024-09-10T22:15:27.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.596+0000 7fac6d600640 1 -- start start 2024-09-10T22:15:27.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac6d600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac68104e10 0x7fac6819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac6d600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a820 0x7fac6819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac6d600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6810b1a0 0x7fac681a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac6d600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac68114d60 con 0x7fac6810a820 2024-09-10T22:15:27.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac6d600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fac68114be0 con 0x7fac68104e10 2024-09-10T22:15:27.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac6d600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fac68114ee0 con 0x7fac6810b1a0 2024-09-10T22:15:27.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac67400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a820 0x7fac6819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac67e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac68104e10 0x7fac6819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac67400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a820 0x7fac6819b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56720/0 (socket says 172.21.15.90:56720) 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac67e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac68104e10 0x7fac6819b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:57118/0 (socket says 172.21.15.90:57118) 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac67400640 1 -- 172.21.15.90:0/3185412421 learned_addr learned my addr 172.21.15.90:0/3185412421 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.597+0000 7fac6cc00640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6810b1a0 0x7fac681a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.598+0000 7fac6cc00640 1 -- 172.21.15.90:0/3185412421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac68104e10 msgr2=0x7fac6819b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.598+0000 7fac6cc00640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac68104e10 0x7fac6819b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.598+0000 7fac6cc00640 1 -- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a820 msgr2=0x7fac6819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.598+0000 7fac6cc00640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a820 0x7fac6819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.598+0000 7fac6cc00640 1 -- 172.21.15.90:0/3185412421 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac681a1200 con 0x7fac6810b1a0 2024-09-10T22:15:27.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.598+0000 7fac67400640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a820 0x7fac6819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:15:27.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.598+0000 7fac6cc00640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6810b1a0 0x7fac681a0a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fac60031040 tx=0x7fac60003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:27.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.599+0000 7fac64c00640 1 -- 172.21.15.90:0/3185412421 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60032c10 con 0x7fac6810b1a0 2024-09-10T22:15:27.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.599+0000 7fac64c00640 1 -- 172.21.15.90:0/3185412421 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac60032db0 con 0x7fac6810b1a0 2024-09-10T22:15:27.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.599+0000 7fac64c00640 1 -- 172.21.15.90:0/3185412421 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac60030ab0 con 0x7fac6810b1a0 2024-09-10T22:15:27.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.599+0000 7fac6d600640 1 -- 172.21.15.90:0/3185412421 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac681a1490 con 0x7fac6810b1a0 2024-09-10T22:15:27.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.599+0000 7fac6d600640 1 -- 172.21.15.90:0/3185412421 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac681a7bb0 con 0x7fac6810b1a0 2024-09-10T22:15:27.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.603+0000 7fac64c00640 1 -- 172.21.15.90:0/3185412421 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fac60030c50 con 0x7fac6810b1a0 2024-09-10T22:15:27.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.603+0000 7fac71669740 1 Processor -- start 2024-09-10T22:15:27.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.603+0000 7fac71669740 1 -- start start 2024-09-10T22:15:27.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.603+0000 7fac71669740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe12f0 0x55eaffbe16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.603+0000 7fac71669740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe1c10 0x55eaffbf5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.603+0000 7fac71669740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 0x55eaffbf87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.603+0000 7fac71669740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eaffb56ca0 con 0x55eaffbe12f0 2024-09-10T22:15:27.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.603+0000 7fac71669740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55eaffb460d0 con 0x55eaffbf63b0 2024-09-10T22:15:27.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.603+0000 7fac71669740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eaffb49ee0 con 0x55eaffbe1c10 2024-09-10T22:15:27.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe12f0 0x55eaffbe16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe1c10 0x55eaffbf5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe1c10 0x55eaffbf5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:45410/0 (socket says 172.21.15.90:45410) 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67e00640 1 -- 172.21.15.90:0/548764065 learned_addr learned my addr 172.21.15.90:0/548764065 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67400640 1 --2- 172.21.15.90:0/548764065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 0x55eaffbf87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67e00640 1 -- 172.21.15.90:0/548764065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 msgr2=0x55eaffbf87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67e00640 1 --2- 172.21.15.90:0/548764065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 0x55eaffbf87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67e00640 1 -- 172.21.15.90:0/548764065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe12f0 msgr2=0x55eaffbe16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67e00640 1 --2- 172.21.15.90:0/548764065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe12f0 0x55eaffbe16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.604+0000 7fac67e00640 1 -- 172.21.15.90:0/548764065 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaffaf8400 con 0x55eaffbe1c10 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.605+0000 7fac67400640 1 --2- 172.21.15.90:0/548764065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 0x55eaffbf87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.605+0000 7fac67e00640 1 --2- 172.21.15.90:0/548764065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe1c10 0x55eaffbf5e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac58009cf0 tx=0x7fac58011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6264bec3fefc67d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.605+0000 7fac64c00640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac4c0b5580 0x7fac4c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.605+0000 7fac5d200640 1 -- 172.21.15.90:0/548764065 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58004230 con 0x55eaffbe1c10 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.605+0000 7fac64c00640 1 -- 172.21.15.90:0/3185412421 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7fac60107380 con 0x7fac6810b1a0 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.605+0000 7fac5d200640 1 -- 172.21.15.90:0/548764065 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac580043d0 con 0x55eaffbe1c10 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.605+0000 7fac5d200640 1 -- 172.21.15.90:0/548764065 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58019750 con 0x55eaffbe1c10 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.605+0000 7fac67400640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac4c0b5580 0x7fac4c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.606+0000 7fac71669740 1 -- 172.21.15.90:0/548764065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe1c10 msgr2=0x55eaffbf5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.606+0000 7fac71669740 1 --2- 172.21.15.90:0/548764065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe1c10 0x55eaffbf5e70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac58009cf0 tx=0x7fac58011810 comp rx=0 tx=0).stop 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.606+0000 7fac67400640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac4c0b5580 0x7fac4c0b7a40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fac54005f60 tx=0x7fac54009d00 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.607+0000 7fac71669740 1 -- 172.21.15.90:0/548764065 shutdown_connections 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.607+0000 7fac71669740 1 --2- 172.21.15.90:0/548764065 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 0x55eaffbf87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.607+0000 7fac71669740 1 --2- 172.21.15.90:0/548764065 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe1c10 0x55eaffbf5e70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.607+0000 7fac71669740 1 --2- 172.21.15.90:0/548764065 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe12f0 0x55eaffbe16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.607+0000 7fac71669740 1 -- 172.21.15.90:0/548764065 >> 172.21.15.90:0/548764065 conn(0x55eaffbdf120 msgr2=0x55eaffbe02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.607+0000 7fac71669740 1 -- 172.21.15.90:0/548764065 shutdown_connections 2024-09-10T22:15:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.607+0000 7fac71669740 1 -- 172.21.15.90:0/548764065 wait complete. 2024-09-10T22:15:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.607+0000 7fac71669740 1 -- start start 2024-09-10T22:15:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.608+0000 7fac71669740 1 Processor -- start 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.608+0000 7fac71669740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 0x55eaffbf5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.608+0000 7fac71669740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbf3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.608+0000 7fac71669740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 0x55eaffbf3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.608+0000 7fac71669740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eaff9a77a0 con 0x55eaffbe1c10 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.608+0000 7fac71669740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55eaffaeba10 con 0x55eaffbf63b0 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.608+0000 7fac71669740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55eaffbf9030 con 0x55eaffbe12f0 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.608+0000 7fac71669740 10 client.0 _ll_register_callbacks cb 0x55eaffafbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.608+0000 7fac67400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbf3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac67400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbf3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56734/0 (socket says 172.21.15.90:56734) 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac67400640 1 -- 172.21.15.90:0/3408760785 learned_addr learned my addr 172.21.15.90:0/3408760785 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac6cc00640 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 0x55eaffbf3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac67e00640 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 0x55eaffbf5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac67400640 1 -- 172.21.15.90:0/3408760785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 msgr2=0x55eaffbf5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac67400640 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 0x55eaffbf5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac67400640 1 -- 172.21.15.90:0/3408760785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 msgr2=0x55eaffbf3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac67400640 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 0x55eaffbf3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac67400640 1 -- 172.21.15.90:0/3408760785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaff8b3b90 con 0x55eaffbe1c10 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.609+0000 7fac67400640 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbf3100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fac5400ff10 tx=0x7fac54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169ca3dbb1214a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.610+0000 7fac51600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.610+0000 7fac51600640 1 -- 172.21.15.90:0/3408760785 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac5401fdd0 con 0x55eaffbe1c10 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.610+0000 7fac71669740 1 -- 172.21.15.90:0/3408760785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac580043d0 con 0x55eaffbe1c10 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.610+0000 7fac51600640 1 -- 172.21.15.90:0/3408760785 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7fac54005020 con 0x55eaffbe1c10 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.610+0000 7fac51600640 1 -- 172.21.15.90:0/3408760785 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fac54035050 con 0x55eaffbe1c10 2024-09-10T22:15:27.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.610+0000 7fac71669740 10 client.16122 fetch_fsmap learned FSMap version 41 2024-09-10T22:15:27.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.610+0000 7fac71669740 1 -- 172.21.15.90:0/3408760785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eaffc049a0 con 0x55eaffbe1c10 2024-09-10T22:15:27.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.611+0000 7fac51600640 1 -- 172.21.15.90:0/3408760785 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac580043d0 con 0x55eaffbe1c10 2024-09-10T22:15:27.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.611+0000 7fac51600640 1 -- 172.21.15.90:0/3408760785 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8410+0+0 (secure 0 0 0) 0x7fac5402a420 con 0x55eaffbe1c10 2024-09-10T22:15:27.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.611+0000 7fac71669740 10 client.16122 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T22:15:27.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.611+0000 7fac71669740 10 client.16122 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:15:27.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.611+0000 7fac71669740 20 client.16122 populate_metadata read hostname 'smithi090' 2024-09-10T22:15:27.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.611+0000 7fac71669740 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55eaffc05f20 0x55eaffc08380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.611+0000 7fac71669740 4 client.16122 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:15:27.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.611+0000 7fac71669740 1 -- 172.21.15.90:0/3408760785 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaffb58bf0 con 0x55eaffc05f20 2024-09-10T22:15:27.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.612+0000 7fac67e00640 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55eaffc05f20 0x55eaffc08380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.612+0000 7fac67e00640 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55eaffc05f20 0x55eaffc08380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53f0ad9a6c93679 server_cookie=dce45e03b622ac3a in_seq=0 out_seq=0 2024-09-10T22:15:27.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.612+0000 7fac51600640 10 client.16122 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:27.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac51600640 1 -- 172.21.15.90:0/3408760785 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eaffb58bf0 con 0x55eaffc05f20 2024-09-10T22:15:27.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac51600640 10 client.16122 handle_command_reply: tid=0 2024-09-10T22:15:27.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac71669740 1 client.16122 shutdown 2024-09-10T22:15:27.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac71669740 1 -- 172.21.15.90:0/3408760785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 msgr2=0x55eaffbf3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac71669740 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbf3100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fac5400ff10 tx=0x7fac54004730 comp rx=0 tx=0).stop 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac71669740 1 -- 172.21.15.90:0/3408760785 shutdown_connections 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac71669740 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55eaffc05f20 0x55eaffc08380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac71669740 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbf63b0 0x55eaffbf3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac71669740 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbf3100 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac71669740 1 --2- 172.21.15.90:0/3408760785 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 0x55eaffbf5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac71669740 1 -- 172.21.15.90:0/3408760785 >> 172.21.15.90:0/3408760785 conn(0x55eaffbdf120 msgr2=0x55eaffbe4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.614+0000 7fac51600640 0 client.16122 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.615+0000 7fac71669740 1 -- 172.21.15.90:0/3408760785 shutdown_connections 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.615+0000 7fac71669740 1 -- 172.21.15.90:0/3408760785 wait complete. 2024-09-10T22:15:27.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.615+0000 7fac71669740 20 client.16122 trim_cache size 0 max 16384 2024-09-10T22:15:27.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.634+0000 7fac71669740 1 Processor -- start 2024-09-10T22:15:27.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.634+0000 7fac71669740 1 -- start start 2024-09-10T22:15:27.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.634+0000 7fac71669740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 0x55eaffbf7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.634+0000 7fac71669740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe1c10 0x55eaffc02260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac71669740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbf63b0 0x55eaffbe06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac71669740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac681ad9f0 con 0x55eaffbf63b0 2024-09-10T22:15:27.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac71669740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fac681af710 con 0x55eaffbe1c10 2024-09-10T22:15:27.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac71669740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fac681ae0c0 con 0x55eaffbe12f0 2024-09-10T22:15:27.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac67e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 0x55eaffbf7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac67e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 0x55eaffbf7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:45424/0 (socket says 172.21.15.90:45424) 2024-09-10T22:15:27.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac67e00640 1 -- 172.21.15.90:0/2080704260 learned_addr learned my addr 172.21.15.90:0/2080704260 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:27.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac67400640 1 --2- 172.21.15.90:0/2080704260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe1c10 0x55eaffc02260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac6cc00640 1 --2- 172.21.15.90:0/2080704260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbf63b0 0x55eaffbe06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac6cc00640 1 -- 172.21.15.90:0/2080704260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 msgr2=0x55eaffbf7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:27.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac6cc00640 1 --2- 172.21.15.90:0/2080704260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 0x55eaffbf7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac6cc00640 1 -- 172.21.15.90:0/2080704260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe1c10 msgr2=0x55eaffc02260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:27.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac6cc00640 1 --2- 172.21.15.90:0/2080704260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe1c10 0x55eaffc02260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.635+0000 7fac6cc00640 1 -- 172.21.15.90:0/2080704260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eaffbdef50 con 0x55eaffbf63b0 2024-09-10T22:15:27.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.636+0000 7fac6cc00640 1 --2- 172.21.15.90:0/2080704260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbf63b0 0x55eaffbe06a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fac601046a0 tx=0x7fac60009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eec1fdb84909408 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:27.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.636+0000 7fac53e00640 1 -- 172.21.15.90:0/2080704260 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac600ccf70 con 0x55eaffbf63b0 2024-09-10T22:15:27.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.636+0000 7fac53e00640 1 -- 172.21.15.90:0/2080704260 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac60036040 con 0x55eaffbf63b0 2024-09-10T22:15:27.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.636+0000 7fac53e00640 1 -- 172.21.15.90:0/2080704260 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac600d6960 con 0x55eaffbf63b0 2024-09-10T22:15:27.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.637+0000 7fac71669740 1 -- 172.21.15.90:0/2080704260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbf63b0 msgr2=0x55eaffbe06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.637+0000 7fac71669740 1 --2- 172.21.15.90:0/2080704260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbf63b0 0x55eaffbe06a0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fac601046a0 tx=0x7fac60009640 comp rx=0 tx=0).stop 2024-09-10T22:15:27.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.637+0000 7fac71669740 1 -- 172.21.15.90:0/2080704260 shutdown_connections 2024-09-10T22:15:27.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.637+0000 7fac71669740 1 --2- 172.21.15.90:0/2080704260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbf63b0 0x55eaffbe06a0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.637+0000 7fac71669740 1 --2- 172.21.15.90:0/2080704260 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe1c10 0x55eaffc02260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.637+0000 7fac71669740 1 --2- 172.21.15.90:0/2080704260 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbe12f0 0x55eaffbf7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.637+0000 7fac71669740 1 -- 172.21.15.90:0/2080704260 >> 172.21.15.90:0/2080704260 conn(0x55eaffbdf120 msgr2=0x55eaffbf4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:27.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.637+0000 7fac71669740 1 -- 172.21.15.90:0/2080704260 shutdown_connections 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.637+0000 7fac71669740 1 -- 172.21.15.90:0/2080704260 wait complete. 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac71669740 1 -- start start 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac71669740 1 Processor -- start 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac71669740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe12f0 0x55eaffbfe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac71669740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbfb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac71669740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbf63b0 0x55eaffbfb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac71669740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac681aef80 con 0x55eaffbe1c10 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac71669740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fac681ae7f0 con 0x55eaffbe12f0 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac71669740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fac681aba30 con 0x55eaffbf63b0 2024-09-10T22:15:27.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac71669740 10 client.0 _ll_register_callbacks cb 0x7fac54009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:15:27.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.638+0000 7fac67e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe12f0 0x55eaffbfe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac6cc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbf63b0 0x55eaffbfb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac67e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe12f0 0x55eaffbfe880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:57180/0 (socket says 172.21.15.90:57180) 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac67e00640 1 -- 172.21.15.90:0/2457278576 learned_addr learned my addr 172.21.15.90:0/2457278576 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac67400640 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbfb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac67e00640 1 -- 172.21.15.90:0/2457278576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbf63b0 msgr2=0x55eaffbfb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac67e00640 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbf63b0 0x55eaffbfb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac67e00640 1 -- 172.21.15.90:0/2457278576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 msgr2=0x55eaffbfb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac67e00640 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbfb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac67e00640 1 -- 172.21.15.90:0/2457278576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaffbfbe10 con 0x55eaffbe12f0 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac67e00640 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe12f0 0x55eaffbfe880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac5801b040 tx=0x7fac58009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6ddb659506a51bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.639+0000 7fac51600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.640+0000 7fac51600640 1 -- 172.21.15.90:0/2457278576 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58005510 con 0x55eaffbe12f0 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.640+0000 7fac51600640 1 -- 172.21.15.90:0/2457278576 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7fac580035f0 con 0x55eaffbe12f0 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.640+0000 7fac71669740 1 -- 172.21.15.90:0/2457278576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac60036040 con 0x55eaffbe12f0 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.640+0000 7fac51600640 1 -- 172.21.15.90:0/2457278576 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fac5803b050 con 0x55eaffbe12f0 2024-09-10T22:15:27.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.640+0000 7fac71669740 10 client.35768 fetch_fsmap learned FSMap version 41 2024-09-10T22:15:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.640+0000 7fac71669740 1 -- 172.21.15.90:0/2457278576 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eaffc73cb0 con 0x55eaffbe12f0 2024-09-10T22:15:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.640+0000 7fac51600640 1 -- 172.21.15.90:0/2457278576 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fac60036040 con 0x55eaffbe12f0 2024-09-10T22:15:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.640+0000 7fac51600640 1 -- 172.21.15.90:0/2457278576 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 41) ==== 8410+0+0 (secure 0 0 0) 0x7fac58019690 con 0x55eaffbe12f0 2024-09-10T22:15:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.641+0000 7fac71669740 10 client.35768 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T22:15:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.641+0000 7fac71669740 10 client.35768 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:15:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.641+0000 7fac71669740 20 client.35768 populate_metadata read hostname 'smithi090' 2024-09-10T22:15:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.641+0000 7fac71669740 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55eaffc742d0 0x55eaffc766f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.641+0000 7fac71669740 4 client.35768 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}] 2024-09-10T22:15:27.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.641+0000 7fac71669740 1 -- 172.21.15.90:0/2457278576 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}) -- 0x55eaffb546a0 con 0x55eaffc742d0 2024-09-10T22:15:27.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.641+0000 7fac67400640 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55eaffc742d0 0x55eaffc766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:27.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.642+0000 7fac67400640 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55eaffc742d0 0x55eaffc766f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9be11ff342a6232 server_cookie=84984d9e43c09d39 in_seq=0 out_seq=0 2024-09-10T22:15:27.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.642+0000 7fac51600640 10 client.35768 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:27.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.696+0000 7fac51600640 1 -- 172.21.15.90:0/2457278576 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55eaffb546a0 con 0x55eaffc742d0 2024-09-10T22:15:27.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.696+0000 7fac51600640 10 client.35768 handle_command_reply: tid=0 2024-09-10T22:15:27.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.696+0000 7fac71669740 1 client.35768 shutdown 2024-09-10T22:15:27.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 -- 172.21.15.90:0/2457278576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe12f0 msgr2=0x55eaffbfe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe12f0 0x55eaffbfe880 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac5801b040 tx=0x7fac58009310 comp rx=0 tx=0).stop 2024-09-10T22:15:27.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 -- 172.21.15.90:0/2457278576 shutdown_connections 2024-09-10T22:15:27.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55eaffc742d0 0x55eaffc766f0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55eaffbf63b0 0x55eaffbfb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eaffbe1c10 0x55eaffbfb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 --2- 172.21.15.90:0/2457278576 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55eaffbe12f0 0x55eaffbfe880 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 -- 172.21.15.90:0/2457278576 >> 172.21.15.90:0/2457278576 conn(0x55eaffbdf120 msgr2=0x55eaffbddae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:27.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac51600640 0 client.35768 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:27.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 -- 172.21.15.90:0/2457278576 shutdown_connections 2024-09-10T22:15:27.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.697+0000 7fac71669740 1 -- 172.21.15.90:0/2457278576 wait complete. 2024-09-10T22:15:27.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac71669740 20 client.35768 trim_cache size 0 max 16384 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 41, 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 34, 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "23d806cd": { 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 34, 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:15:27.701 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 145.4, 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:27.702 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:15:27.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 -- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac4c0b5580 msgr2=0x7fac4c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac4c0b5580 0x7fac4c0b7a40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fac54005f60 tx=0x7fac54009d00 comp rx=0 tx=0).stop 2024-09-10T22:15:27.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 -- 172.21.15.90:0/3185412421 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6810b1a0 msgr2=0x7fac681a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:27.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6810b1a0 0x7fac681a0a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fac60031040 tx=0x7fac60003f30 comp rx=0 tx=0).stop 2024-09-10T22:15:27.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 -- 172.21.15.90:0/3185412421 shutdown_connections 2024-09-10T22:15:27.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fac4c0b5580 0x7fac4c0b7a40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fac6810b1a0 0x7fac681a0a30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac6810a820 0x7fac6819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 --2- 172.21.15.90:0/3185412421 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fac68104e10 0x7fac6819b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:27.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 -- 172.21.15.90:0/3185412421 >> 172.21.15.90:0/3185412421 conn(0x7fac68100840 msgr2=0x7fac68103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:27.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 -- 172.21.15.90:0/3185412421 shutdown_connections 2024-09-10T22:15:27.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:27.699+0000 7fac6d600640 1 -- 172.21.15.90:0/3185412421 wait complete. 2024-09-10T22:15:28.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.15 MB/sec execute 500 sec latency 773.799 ms 2024-09-10T22:15:28.648 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '23d806cd', quiesce took 4.4 seconds. Will release after: 21.0 2024-09-10T22:15:29.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:28 smithi154 ceph-mon[34495]: pgmap v777: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 836 KiB/s wr, 62 op/s 2024-09-10T22:15:29.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:28 smithi171 ceph-mon[34138]: pgmap v777: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 836 KiB/s wr, 62 op/s 2024-09-10T22:15:29.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:28 smithi090 ceph-mon[30350]: pgmap v777: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 836 KiB/s wr, 62 op/s 2024-09-10T22:15:29.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.15 MB/sec execute 501 sec latency 1774.177 ms 2024-09-10T22:15:30.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.15 MB/sec execute 502 sec latency 2774.317 ms 2024-09-10T22:15:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:30 smithi154 ceph-mon[34495]: pgmap v778: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-09-10T22:15:31.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:30 smithi171 ceph-mon[34138]: pgmap v778: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-09-10T22:15:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:30 smithi090 ceph-mon[30350]: pgmap v778: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-09-10T22:15:31.496 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.15 MB/sec execute 503 sec latency 3774.392 ms 2024-09-10T22:15:32.497 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.15 MB/sec execute 504 sec latency 4774.515 ms 2024-09-10T22:15:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:32 smithi154 ceph-mon[34495]: pgmap v779: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 782 KiB/s wr, 79 op/s 2024-09-10T22:15:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:32 smithi171 ceph-mon[34138]: pgmap v779: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 782 KiB/s wr, 79 op/s 2024-09-10T22:15:33.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:32 smithi090 ceph-mon[30350]: pgmap v779: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 782 KiB/s wr, 79 op/s 2024-09-10T22:15:33.497 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.14 MB/sec execute 505 sec latency 5774.667 ms 2024-09-10T22:15:34.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:33 smithi154 ceph-mon[34495]: pgmap v780: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 697 KiB/s wr, 72 op/s 2024-09-10T22:15:34.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:33 smithi171 ceph-mon[34138]: pgmap v780: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 697 KiB/s wr, 72 op/s 2024-09-10T22:15:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:33 smithi090 ceph-mon[30350]: pgmap v780: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 697 KiB/s wr, 72 op/s 2024-09-10T22:15:34.497 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.14 MB/sec execute 506 sec latency 6774.802 ms 2024-09-10T22:15:35.497 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.14 MB/sec execute 507 sec latency 7774.942 ms 2024-09-10T22:15:36.497 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.14 MB/sec execute 508 sec latency 8775.098 ms 2024-09-10T22:15:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:36 smithi154 ceph-mon[34495]: pgmap v781: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 620 KiB/s wr, 70 op/s 2024-09-10T22:15:37.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:36 smithi171 ceph-mon[34138]: pgmap v781: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 620 KiB/s wr, 70 op/s 2024-09-10T22:15:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:36 smithi090 ceph-mon[30350]: pgmap v781: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 620 KiB/s wr, 70 op/s 2024-09-10T22:15:37.497 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.13 MB/sec execute 509 sec latency 9775.237 ms 2024-09-10T22:15:38.497 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.13 MB/sec execute 510 sec latency 10775.356 ms 2024-09-10T22:15:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:38 smithi154 ceph-mon[34495]: pgmap v782: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 436 KiB/s wr, 47 op/s 2024-09-10T22:15:39.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:38 smithi171 ceph-mon[34138]: pgmap v782: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 436 KiB/s wr, 47 op/s 2024-09-10T22:15:39.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:38 smithi090 ceph-mon[30350]: pgmap v782: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 436 KiB/s wr, 47 op/s 2024-09-10T22:15:39.498 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.13 MB/sec execute 511 sec latency 11775.503 ms 2024-09-10T22:15:40.498 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.13 MB/sec execute 512 sec latency 12775.639 ms 2024-09-10T22:15:41.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:40 smithi154 ceph-mon[34495]: pgmap v783: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 368 KiB/s wr, 52 op/s 2024-09-10T22:15:41.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:40 smithi171 ceph-mon[34138]: pgmap v783: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 368 KiB/s wr, 52 op/s 2024-09-10T22:15:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:40 smithi090 ceph-mon[30350]: pgmap v783: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 368 KiB/s wr, 52 op/s 2024-09-10T22:15:41.498 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.13 MB/sec execute 513 sec latency 13775.762 ms 2024-09-10T22:15:42.498 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.12 MB/sec execute 514 sec latency 14775.855 ms 2024-09-10T22:15:43.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:42 smithi154 ceph-mon[34495]: pgmap v784: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-09-10T22:15:43.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:42 smithi171 ceph-mon[34138]: pgmap v784: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-09-10T22:15:43.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:42 smithi090 ceph-mon[30350]: pgmap v784: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-09-10T22:15:43.498 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.12 MB/sec execute 515 sec latency 15775.938 ms 2024-09-10T22:15:44.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:43 smithi171 ceph-mon[34138]: pgmap v785: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-10T22:15:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:43 smithi090 ceph-mon[30350]: pgmap v785: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-10T22:15:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:43 smithi154 ceph-mon[34495]: pgmap v785: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 44 KiB/s wr, 13 op/s 2024-09-10T22:15:44.498 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.12 MB/sec execute 516 sec latency 16776.063 ms 2024-09-10T22:15:45.498 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.12 MB/sec execute 517 sec latency 17776.208 ms 2024-09-10T22:15:46.499 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.11 MB/sec execute 518 sec latency 18776.350 ms 2024-09-10T22:15:47.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:46 smithi154 ceph-mon[34495]: pgmap v786: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-09-10T22:15:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:46 smithi171 ceph-mon[34138]: pgmap v786: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-09-10T22:15:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:46 smithi090 ceph-mon[30350]: pgmap v786: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-09-10T22:15:47.499 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.11 MB/sec execute 519 sec latency 19776.492 ms 2024-09-10T22:15:48.499 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.11 MB/sec execute 520 sec latency 20776.658 ms 2024-09-10T22:15:49.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:48 smithi154 ceph-mon[34495]: pgmap v787: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-10T22:15:49.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:48 smithi171 ceph-mon[34138]: pgmap v787: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-10T22:15:49.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:48 smithi090 ceph-mon[30350]: pgmap v787: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-10T22:15:49.499 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.11 MB/sec execute 521 sec latency 21776.731 ms 2024-09-10T22:15:49.650 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:15:50.038 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:15:50.499 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.11 MB/sec execute 522 sec latency 22776.798 ms 2024-09-10T22:15:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.627+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3588147272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 msgr2=0x7fdd5810ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:50.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.627+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3588147272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 0x7fdd5810ac40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009900 tx=0x7fdd4402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.627+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3588147272 shutdown_connections 2024-09-10T22:15:50.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.627+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3588147272 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd5810b180 0x7fdd58111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.627+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3588147272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 0x7fdd5810ac40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.627+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3588147272 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd58104df0 0x7fdd581051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.627+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3588147272 >> 172.21.15.90:0/3588147272 conn(0x7fdd58100840 msgr2=0x7fdd58102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:50.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.627+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3588147272 shutdown_connections 2024-09-10T22:15:50.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.627+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3588147272 wait complete. 2024-09-10T22:15:50.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.628+0000 7fdd5cc00640 1 Processor -- start 2024-09-10T22:15:50.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.628+0000 7fdd5cc00640 1 -- start start 2024-09-10T22:15:50.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.628+0000 7fdd5cc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd58104df0 0x7fdd5819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:50.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.628+0000 7fdd5cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 0x7fdd5819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:50.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.628+0000 7fdd5cc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5810b180 0x7fdd581a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:50.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.628+0000 7fdd5cc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58114d00 con 0x7fdd5810a800 2024-09-10T22:15:50.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.628+0000 7fdd5cc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58114b80 con 0x7fdd5810b180 2024-09-10T22:15:50.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.628+0000 7fdd5cc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58114e80 con 0x7fdd58104df0 2024-09-10T22:15:50.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd56a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 0x7fdd5819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:50.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd57400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd58104df0 0x7fdd5819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:50.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd56a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 0x7fdd5819b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50324/0 (socket says 172.21.15.90:50324) 2024-09-10T22:15:50.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd56a00640 1 -- 172.21.15.90:0/3307001917 learned_addr learned my addr 172.21.15.90:0/3307001917 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:50.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd57400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd58104df0 0x7fdd5819b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:48754/0 (socket says 172.21.15.90:48754) 2024-09-10T22:15:50.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd57e00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5810b180 0x7fdd581a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:50.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd56a00640 1 -- 172.21.15.90:0/3307001917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd58104df0 msgr2=0x7fdd5819b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:50.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd56a00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd58104df0 0x7fdd5819b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd56a00640 1 -- 172.21.15.90:0/3307001917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5810b180 msgr2=0x7fdd581a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:50.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd56a00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5810b180 0x7fdd581a09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd56a00640 1 -- 172.21.15.90:0/3307001917 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd581a11a0 con 0x7fdd5810a800 2024-09-10T22:15:50.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.629+0000 7fdd56a00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 0x7fdd5819b630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdd4402a910 tx=0x7fdd44003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:50.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.630+0000 7fdd4fe00640 1 -- 172.21.15.90:0/3307001917 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4403f070 con 0x7fdd5810a800 2024-09-10T22:15:50.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.630+0000 7fdd4fe00640 1 -- 172.21.15.90:0/3307001917 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd44003e40 con 0x7fdd5810a800 2024-09-10T22:15:50.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.630+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd581a1430 con 0x7fdd5810a800 2024-09-10T22:15:50.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.630+0000 7fdd4fe00640 1 -- 172.21.15.90:0/3307001917 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44035030 con 0x7fdd5810a800 2024-09-10T22:15:50.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.630+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd581a7b50 con 0x7fdd5810a800 2024-09-10T22:15:50.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.631+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd581051d0 con 0x7fdd5810a800 2024-09-10T22:15:50.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.633+0000 7fdd4fe00640 1 -- 172.21.15.90:0/3307001917 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdd44031400 con 0x7fdd5810a800 2024-09-10T22:15:50.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.634+0000 7fdd4fe00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd3c0b5580 0x7fdd3c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:50.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.634+0000 7fdd4fe00640 1 -- 172.21.15.90:0/3307001917 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7fdd441071b0 con 0x7fdd5810a800 2024-09-10T22:15:50.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.635+0000 7fdd57400640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd3c0b5580 0x7fdd3c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:50.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.635+0000 7fdd57400640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd3c0b5580 0x7fdd3c0b7a40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd48007ec0 tx=0x7fdd480075b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:50.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.636+0000 7fdd4fe00640 1 -- 172.21.15.90:0/3307001917 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdd440cd2b0 con 0x7fdd5810a800 2024-09-10T22:15:50.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.861+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd581a7de0 con 0x7fdd5810a800 2024-09-10T22:15:50.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.862+0000 7fdd4fe00640 1 -- 172.21.15.90:0/3307001917 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+12003 (secure 0 0 0) 0x7fdd440d1ce0 con 0x7fdd5810a800 2024-09-10T22:15:50.865 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:15:50.865 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"btime":"2024-09-10T22:15:01:892115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:15:00.940095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:15:50.865 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 41 2024-09-10T22:15:50.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd3c0b5580 msgr2=0x7fdd3c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:50.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd3c0b5580 0x7fdd3c0b7a40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd48007ec0 tx=0x7fdd480075b0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 msgr2=0x7fdd5819b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:50.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 0x7fdd5819b630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fdd4402a910 tx=0x7fdd44003c20 comp rx=0 tx=0).stop 2024-09-10T22:15:50.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 shutdown_connections 2024-09-10T22:15:50.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdd3c0b5580 0x7fdd3c0b7a40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdd5810b180 0x7fdd581a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd5810a800 0x7fdd5819b630 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 --2- 172.21.15.90:0/3307001917 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdd58104df0 0x7fdd5819b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 >> 172.21.15.90:0/3307001917 conn(0x7fdd58100840 msgr2=0x7fdd58108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 shutdown_connections 2024-09-10T22:15:50.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:50.865+0000 7fdd5cc00640 1 -- 172.21.15.90:0/3307001917 wait complete. 2024-09-10T22:15:51.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:50 smithi154 ceph-mon[34495]: pgmap v788: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-09-10T22:15:51.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:50 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3307001917' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:15:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:50 smithi171 ceph-mon[34138]: pgmap v788: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-09-10T22:15:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:50 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3307001917' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:15:51.265 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:50 smithi090 ceph-mon[30350]: pgmap v788: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-09-10T22:15:51.265 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:50 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3307001917' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:15:51.499 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.10 MB/sec execute 523 sec latency 23776.929 ms 2024-09-10T22:15:51.716 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 23d806cd --release --await' 2024-09-10T22:15:51.716 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 23d806cd --release --await 2024-09-10T22:15:52.089 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:15:52.499 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136329 1.10 MB/sec execute 524 sec latency 24777.018 ms 2024-09-10T22:15:52.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.687+0000 7f9971a00640 1 -- 172.21.15.90:0/3860006329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c10a800 msgr2=0x7f996c10ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:52.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.687+0000 7f9971a00640 1 --2- 172.21.15.90:0/3860006329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c10a800 0x7f996c10ac40 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9958009900 tx=0x7f995802a410 comp rx=0 tx=0).stop 2024-09-10T22:15:52.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.687+0000 7f9971a00640 1 -- 172.21.15.90:0/3860006329 shutdown_connections 2024-09-10T22:15:52.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.687+0000 7f9971a00640 1 --2- 172.21.15.90:0/3860006329 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c10b180 0x7f996c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.687+0000 7f9971a00640 1 --2- 172.21.15.90:0/3860006329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c10a800 0x7f996c10ac40 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.687+0000 7f9971a00640 1 --2- 172.21.15.90:0/3860006329 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996c104df0 0x7f996c1051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.687+0000 7f9971a00640 1 -- 172.21.15.90:0/3860006329 >> 172.21.15.90:0/3860006329 conn(0x7f996c100840 msgr2=0x7f996c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.687+0000 7f9971a00640 1 -- 172.21.15.90:0/3860006329 shutdown_connections 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.687+0000 7f9971a00640 1 -- 172.21.15.90:0/3860006329 wait complete. 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.688+0000 7f9971a00640 1 Processor -- start 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.688+0000 7f9971a00640 1 -- start start 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.688+0000 7f9971a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c104df0 0x7f996c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.688+0000 7f9971a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996c10a800 0x7f996c19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c10b180 0x7f996c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f996c114d40 con 0x7f996c10b180 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f996c114bc0 con 0x7f996c104df0 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f996c114ec0 con 0x7f996c10a800 2024-09-10T22:15:52.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f996be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c104df0 0x7f996c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c10b180 0x7f996c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f996be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c104df0 0x7f996c19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:42780/0 (socket says 172.21.15.90:42780) 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f996b400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996c10a800 0x7f996c19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f996be00640 1 -- 172.21.15.90:0/3408601778 learned_addr learned my addr 172.21.15.90:0/3408601778 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971000640 1 -- 172.21.15.90:0/3408601778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996c10a800 msgr2=0x7f996c19b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971000640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996c10a800 0x7f996c19b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971000640 1 -- 172.21.15.90:0/3408601778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c104df0 msgr2=0x7f996c19b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971000640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c104df0 0x7f996c19b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f9971000640 1 -- 172.21.15.90:0/3408601778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f996c1a11e0 con 0x7f996c10b180 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.689+0000 7f996b400640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996c10a800 0x7f996c19b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.690+0000 7f9971000640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c10b180 0x7f996c1a0a10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f996400da10 tx=0x7f996400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.690+0000 7f9968c00640 1 -- 172.21.15.90:0/3408601778 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996400bd70 con 0x7f996c10b180 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.690+0000 7f9968c00640 1 -- 172.21.15.90:0/3408601778 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9964003e00 con 0x7f996c10b180 2024-09-10T22:15:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.690+0000 7f9968c00640 1 -- 172.21.15.90:0/3408601778 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99640106c0 con 0x7f996c10b180 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.690+0000 7f9971a00640 1 -- 172.21.15.90:0/3408601778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f996c1a14d0 con 0x7f996c10b180 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.690+0000 7f9971a00640 1 -- 172.21.15.90:0/3408601778 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f996c1a7b90 con 0x7f996c10b180 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.693+0000 7f99759f1740 1 Processor -- start 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.693+0000 7f9968c00640 1 -- 172.21.15.90:0/3408601778 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9964024060 con 0x7f996c10b180 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f99759f1740 1 -- start start 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f99759f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f99759f1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead19af0 0x557fead25540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f99759f1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead27e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f99759f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557feac90290 con 0x557fead191d0 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f99759f1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557feac7f6c0 con 0x557fead25a80 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f99759f1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557feac834d0 con 0x557fead19af0 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f996b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f996b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead195b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50354/0 (socket says 172.21.15.90:50354) 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.694+0000 7f996b400640 1 -- 172.21.15.90:0/1566738322 learned_addr learned my addr 172.21.15.90:0/1566738322 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f996b400640 1 -- 172.21.15.90:0/1566738322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead19af0 msgr2=0x557fead25540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f996b400640 1 --2- 172.21.15.90:0/1566738322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead19af0 0x557fead25540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f996b400640 1 -- 172.21.15.90:0/1566738322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 msgr2=0x557fead27e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f996b400640 1 --2- 172.21.15.90:0/1566738322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead27e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f996b400640 1 -- 172.21.15.90:0/1566738322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557feac0c460 con 0x557fead191d0 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f996b400640 1 --2- 172.21.15.90:0/1566738322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead195b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9958033000 tx=0x7f9958004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e56dd6bf86b0876 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f9968c00640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99500b5500 0x7f99500b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f9961600640 1 -- 172.21.15.90:0/1566738322 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958035040 con 0x557fead191d0 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f9961600640 1 -- 172.21.15.90:0/1566738322 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9958003d80 con 0x557fead191d0 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f9961600640 1 -- 172.21.15.90:0/1566738322 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995803bcf0 con 0x557fead191d0 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f9968c00640 1 -- 172.21.15.90:0/3408601778 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7f99640e6470 con 0x7f996c10b180 2024-09-10T22:15:52.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.695+0000 7f996be00640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99500b5500 0x7f99500b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f99759f1740 1 -- 172.21.15.90:0/1566738322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 msgr2=0x557fead195b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f996be00640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99500b5500 0x7f99500b79c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f995c009810 tx=0x7f995c0097a0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f99759f1740 1 --2- 172.21.15.90:0/1566738322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead195b0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9958033000 tx=0x7f9958004610 comp rx=0 tx=0).stop 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f99759f1740 1 -- 172.21.15.90:0/1566738322 shutdown_connections 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f99759f1740 1 --2- 172.21.15.90:0/1566738322 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead27e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f99759f1740 1 --2- 172.21.15.90:0/1566738322 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead19af0 0x557fead25540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f99759f1740 1 --2- 172.21.15.90:0/1566738322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead195b0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f99759f1740 1 -- 172.21.15.90:0/1566738322 >> 172.21.15.90:0/1566738322 conn(0x557fead17000 msgr2=0x557fead181a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f99759f1740 1 -- 172.21.15.90:0/1566738322 shutdown_connections 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.696+0000 7f99759f1740 1 -- 172.21.15.90:0/1566738322 wait complete. 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.697+0000 7f99759f1740 1 -- start start 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.697+0000 7f99759f1740 1 Processor -- start 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.697+0000 7f99759f1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 0x557fead22ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.697+0000 7f99759f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead19af0 0x557fead23020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f99759f1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead23560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f99759f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557feaae0bb0 con 0x557fead19af0 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f99759f1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x557feac1a6c0 con 0x557fead25a80 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f99759f1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x557fead286d0 con 0x557fead191d0 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f99759f1740 10 client.0 _ll_register_callbacks cb 0x557feac35490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f996b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead19af0 0x557fead23020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f996b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead19af0 0x557fead23020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50368/0 (socket says 172.21.15.90:50368) 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f996b400640 1 -- 172.21.15.90:0/2697316528 learned_addr learned my addr 172.21.15.90:0/2697316528 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f9971000640 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead23560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f996be00640 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 0x557fead22ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f996b400640 1 -- 172.21.15.90:0/2697316528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 msgr2=0x557fead22ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f996b400640 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 0x557fead22ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f996b400640 1 -- 172.21.15.90:0/2697316528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 msgr2=0x557fead23560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:52.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f996b400640 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead23560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.698+0000 7f996b400640 1 -- 172.21.15.90:0/2697316528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fea9ddb00 con 0x557fead19af0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.699+0000 7f996b400640 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead19af0 0x557fead23020 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9958004010 tx=0x7f995802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cb702b6937c5ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.699+0000 7f9956000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.699+0000 7f9956000640 1 -- 172.21.15.90:0/2697316528 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9958035040 con 0x557fead19af0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.699+0000 7f9956000640 1 -- 172.21.15.90:0/2697316528 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7f99580036f0 con 0x557fead19af0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.699+0000 7f99759f1740 1 -- 172.21.15.90:0/2697316528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9958003d80 con 0x557fead19af0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.699+0000 7f9956000640 1 -- 172.21.15.90:0/2697316528 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9958057050 con 0x557fead19af0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.699+0000 7f99759f1740 10 client.16146 fetch_fsmap learned FSMap version 41 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.699+0000 7f99759f1740 1 -- 172.21.15.90:0/2697316528 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557feaa54960 con 0x557fead19af0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.700+0000 7f9956000640 1 -- 172.21.15.90:0/2697316528 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9958003d80 con 0x557fead19af0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.700+0000 7f9956000640 1 -- 172.21.15.90:0/2697316528 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8410+0+0 (secure 0 0 0) 0x7f9958045c60 con 0x557fead19af0 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.700+0000 7f99759f1740 10 client.16146 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T22:15:52.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.700+0000 7f99759f1740 10 client.16146 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.700+0000 7f99759f1740 20 client.16146 populate_metadata read hostname 'smithi090' 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.700+0000 7f99759f1740 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557fead354d0 0x557fead378c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.700+0000 7f99759f1740 4 client.16146 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.700+0000 7f99759f1740 1 -- 172.21.15.90:0/2697316528 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557feac921e0 con 0x557fead354d0 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.701+0000 7f996be00640 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557fead354d0 0x557fead378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.701+0000 7f996be00640 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557fead354d0 0x557fead378c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb6202edd1fc9f4 server_cookie=abe54ee687ab7fa4 in_seq=0 out_seq=0 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.701+0000 7f9956000640 10 client.16146 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f9956000640 1 -- 172.21.15.90:0/2697316528 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557feac921e0 con 0x557fead354d0 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f9956000640 10 client.16146 handle_command_reply: tid=0 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f99759f1740 1 client.16146 shutdown 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f99759f1740 1 -- 172.21.15.90:0/2697316528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead19af0 msgr2=0x557fead23020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f99759f1740 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead19af0 0x557fead23020 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9958004010 tx=0x7f995802a410 comp rx=0 tx=0).stop 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f99759f1740 1 -- 172.21.15.90:0/2697316528 shutdown_connections 2024-09-10T22:15:52.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f99759f1740 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557fead354d0 0x557fead378c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f99759f1740 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead23560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f99759f1740 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead19af0 0x557fead23020 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f9956000640 0 client.16146 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f99759f1740 1 --2- 172.21.15.90:0/2697316528 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 0x557fead22ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.702+0000 7f99759f1740 1 -- 172.21.15.90:0/2697316528 >> 172.21.15.90:0/2697316528 conn(0x557fead17000 msgr2=0x557fead1cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.703+0000 7f99759f1740 1 -- 172.21.15.90:0/2697316528 shutdown_connections 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.703+0000 7f99759f1740 1 -- 172.21.15.90:0/2697316528 wait complete. 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.703+0000 7f99759f1740 20 client.16146 trim_cache size 0 max 16384 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.720+0000 7f99759f1740 1 Processor -- start 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.720+0000 7f99759f1740 1 -- start start 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.720+0000 7f99759f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead1fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f99759f1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead19af0 0x557fead20230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f99759f1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead3cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f99759f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b1c70 con 0x557fead191d0 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f99759f1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b3990 con 0x557fead25a80 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f99759f1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b2340 con 0x557fead19af0 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead1fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead1fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50376/0 (socket says 172.21.15.90:50376) 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996be00640 1 -- 172.21.15.90:0/1910766146 learned_addr learned my addr 172.21.15.90:0/1910766146 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996b400640 1 --2- 172.21.15.90:0/1910766146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead19af0 0x557fead20230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f9971000640 1 --2- 172.21.15.90:0/1910766146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead3cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996be00640 1 -- 172.21.15.90:0/1910766146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead19af0 msgr2=0x557fead20230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996be00640 1 --2- 172.21.15.90:0/1910766146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead19af0 0x557fead20230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996be00640 1 -- 172.21.15.90:0/1910766146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 msgr2=0x557fead3cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996be00640 1 --2- 172.21.15.90:0/1910766146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead3cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996be00640 1 -- 172.21.15.90:0/1910766146 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fead16e30 con 0x557fead191d0 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.721+0000 7f996be00640 1 --2- 172.21.15.90:0/1910766146 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead1fcf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f995c012fd0 tx=0x7f995c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6e5e86a6488405 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f9960c00640 1 -- 172.21.15.90:0/1910766146 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c02a670 con 0x557fead191d0 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f9960c00640 1 -- 172.21.15.90:0/1910766146 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f995c02aca0 con 0x557fead191d0 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f9960c00640 1 -- 172.21.15.90:0/1910766146 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c02fc70 con 0x557fead191d0 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f99759f1740 1 -- 172.21.15.90:0/1910766146 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 msgr2=0x557fead1fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f99759f1740 1 --2- 172.21.15.90:0/1910766146 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead1fcf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f995c012fd0 tx=0x7f995c014000 comp rx=0 tx=0).stop 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f99759f1740 1 -- 172.21.15.90:0/1910766146 shutdown_connections 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f99759f1740 1 --2- 172.21.15.90:0/1910766146 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead25a80 0x557fead3cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f99759f1740 1 --2- 172.21.15.90:0/1910766146 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead19af0 0x557fead20230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f99759f1740 1 --2- 172.21.15.90:0/1910766146 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead191d0 0x557fead1fcf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f99759f1740 1 -- 172.21.15.90:0/1910766146 >> 172.21.15.90:0/1910766146 conn(0x557fead17000 msgr2=0x557fead31140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.722+0000 7f99759f1740 1 -- 172.21.15.90:0/1910766146 shutdown_connections 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.723+0000 7f99759f1740 1 -- 172.21.15.90:0/1910766146 wait complete. 2024-09-10T22:15:52.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.723+0000 7f99759f1740 1 -- start start 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.723+0000 7f99759f1740 1 Processor -- start 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.723+0000 7f99759f1740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 0x557fead2ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f99759f1740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead19af0 0x557fead2b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f99759f1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead25a80 0x557feada2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f99759f1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b3200 con 0x557fead25a80 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f99759f1740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1b2a70 con 0x557fead19af0 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f99759f1740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f996c1afc70 con 0x557fead191d0 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f99759f1740 10 client.0 _ll_register_callbacks cb 0x7f99580095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f9971000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead25a80 0x557feada2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f996b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead19af0 0x557fead2b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f9971000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead25a80 0x557feada2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50384/0 (socket says 172.21.15.90:50384) 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f9971000640 1 -- 172.21.15.90:0/3315192073 learned_addr learned my addr 172.21.15.90:0/3315192073 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:52.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f996be00640 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 0x557fead2ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f9971000640 1 -- 172.21.15.90:0/3315192073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 msgr2=0x557fead2ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f9971000640 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 0x557fead2ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f9971000640 1 -- 172.21.15.90:0/3315192073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead19af0 msgr2=0x557fead2b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f9971000640 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead19af0 0x557fead2b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.724+0000 7f9971000640 1 -- 172.21.15.90:0/3315192073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fead17a80 con 0x557fead25a80 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f9971000640 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead25a80 0x557feada2160 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f99640e3df0 tx=0x7f99640abc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c2c0f9330e3c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f9956000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f9956000640 1 -- 172.21.15.90:0/3315192073 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99640acc20 con 0x557fead25a80 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f9956000640 1 -- 172.21.15.90:0/3315192073 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7f99640a8790 con 0x557fead25a80 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f99759f1740 1 -- 172.21.15.90:0/3315192073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f995c02aca0 con 0x557fead25a80 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f9956000640 1 -- 172.21.15.90:0/3315192073 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f99640ac520 con 0x557fead25a80 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f99759f1740 10 client.16152 fetch_fsmap learned FSMap version 41 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f99759f1740 1 -- 172.21.15.90:0/3315192073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557feada3870 con 0x557fead25a80 2024-09-10T22:15:52.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f9956000640 1 -- 172.21.15.90:0/3315192073 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f995c02aca0 con 0x557fead25a80 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f9956000640 1 -- 172.21.15.90:0/3315192073 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8410+0+0 (secure 0 0 0) 0x7f9964093050 con 0x557fead25a80 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f99759f1740 10 client.16152 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f99759f1740 10 client.16152 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f99759f1740 20 client.16152 populate_metadata read hostname 'smithi090' 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f99759f1740 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557feada40b0 0x557feada64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f99759f1740 4 client.16152 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "23d806cd", "release": true, "await": true}] 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.725+0000 7f99759f1740 1 -- 172.21.15.90:0/3315192073 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "23d806cd", "release": true, "await": true}) -- 0x557feac8dc90 con 0x557feada40b0 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.726+0000 7f996be00640 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557feada40b0 0x557feada64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.727+0000 7f996be00640 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557feada40b0 0x557feada64d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3127251bbd4ec3d0 server_cookie=f652cab391ac4130 in_seq=0 out_seq=0 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.727+0000 7f9956000640 10 client.16152 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.734+0000 7f9956000640 1 -- 172.21.15.90:0/3315192073 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557feac8dc90 con 0x557feada40b0 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.734+0000 7f9956000640 10 client.16152 handle_command_reply: tid=0 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.734+0000 7f99759f1740 1 client.16152 shutdown 2024-09-10T22:15:52.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 -- 172.21.15.90:0/3315192073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead25a80 msgr2=0x557feada2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:52.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead25a80 0x557feada2160 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f99640e3df0 tx=0x7f99640abc90 comp rx=0 tx=0).stop 2024-09-10T22:15:52.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 -- 172.21.15.90:0/3315192073 shutdown_connections 2024-09-10T22:15:52.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x557feada40b0 0x557feada64d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fead25a80 0x557feada2160 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x557fead19af0 0x557fead2b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 --2- 172.21.15.90:0/3315192073 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x557fead191d0 0x557fead2ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 -- 172.21.15.90:0/3315192073 >> 172.21.15.90:0/3315192073 conn(0x557fead17000 msgr2=0x557fead1c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:52.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f9956000640 0 client.16152 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:52.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 -- 172.21.15.90:0/3315192073 shutdown_connections 2024-09-10T22:15:52.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 1 -- 172.21.15.90:0/3315192073 wait complete. 2024-09-10T22:15:52.761 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:15:52.761 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 41, 2024-09-10T22:15:52.761 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:15:52.761 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 36, 2024-09-10T22:15:52.761 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:15:52.761 INFO:teuthology.orchestra.run.smithi090.stdout: "23d806cd": { 2024-09-10T22:15:52.761 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 36, 2024-09-10T22:15:52.761 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 145.4, 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:52.762 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:52.763 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:52.763 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:52.763 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:15:52.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.735+0000 7f99759f1740 20 client.16152 trim_cache size 0 max 16384 2024-09-10T22:15:52.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 -- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99500b5500 msgr2=0x7f99500b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:52.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99500b5500 0x7f99500b79c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f995c009810 tx=0x7f995c0097a0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 -- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c10b180 msgr2=0x7f996c1a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:52.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c10b180 0x7f996c1a0a10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f996400da10 tx=0x7f996400ded0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 -- 172.21.15.90:0/3408601778 shutdown_connections 2024-09-10T22:15:52.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f99500b5500 0x7f99500b79c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f996c10b180 0x7f996c1a0a10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f996c10a800 0x7f996c19b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 --2- 172.21.15.90:0/3408601778 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f996c104df0 0x7f996c19b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:52.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 -- 172.21.15.90:0/3408601778 >> 172.21.15.90:0/3408601778 conn(0x7f996c100840 msgr2=0x7f996c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:52.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 -- 172.21.15.90:0/3408601778 shutdown_connections 2024-09-10T22:15:52.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:52.736+0000 7f9971a00640 1 -- 172.21.15.90:0/3408601778 wait complete. 2024-09-10T22:15:53.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:52 smithi154 ceph-mon[34495]: pgmap v789: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-10T22:15:53.243 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:52 smithi090 ceph-mon[30350]: pgmap v789: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-10T22:15:53.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:52 smithi171 ceph-mon[34138]: pgmap v789: 129 pgs: 129 active+clean; 178 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 KiB/s wr, 9 op/s 2024-09-10T22:15:53.499 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136596 1.10 MB/sec execute 525 sec latency 25016.917 ms 2024-09-10T22:15:53.662 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '23d806cd', total seconds elapsed: 29.4 2024-09-10T22:15:53.662 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.4 2024-09-10T22:15:53.662 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:15:54.078 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:15:54.223 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:53 smithi090 ceph-mon[30350]: pgmap v790: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-09-10T22:15:54.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:53 smithi171 ceph-mon[34138]: pgmap v790: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-09-10T22:15:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:53 smithi154 ceph-mon[34495]: pgmap v790: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-09-10T22:15:54.499 INFO:tasks.workunit.client.0.smithi090.stdout: 1 136997 1.10 MB/sec execute 526 sec latency 27.306 ms 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.683+0000 7f567be00640 1 -- 172.21.15.90:0/1005387374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f567c0754e0 msgr2=0x7f567c0758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.683+0000 7f567be00640 1 --2- 172.21.15.90:0/1005387374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f567c0754e0 0x7f567c0758c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f566c009900 tx=0x7f566c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.683+0000 7f567be00640 1 -- 172.21.15.90:0/1005387374 shutdown_connections 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.683+0000 7f567be00640 1 --2- 172.21.15.90:0/1005387374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f567c10f200 0x7f567c1115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.683+0000 7f567be00640 1 --2- 172.21.15.90:0/1005387374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567c075e00 0x7f567c10ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.683+0000 7f567be00640 1 --2- 172.21.15.90:0/1005387374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f567c0754e0 0x7f567c0758c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.683+0000 7f567be00640 1 -- 172.21.15.90:0/1005387374 >> 172.21.15.90:0/1005387374 conn(0x7f567c0fe660 msgr2=0x7f567c100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.683+0000 7f567be00640 1 -- 172.21.15.90:0/1005387374 shutdown_connections 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.683+0000 7f567be00640 1 -- 172.21.15.90:0/1005387374 wait complete. 2024-09-10T22:15:54.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.684+0000 7f567be00640 1 Processor -- start 2024-09-10T22:15:54.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.684+0000 7f567be00640 1 -- start start 2024-09-10T22:15:54.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.684+0000 7f567be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f567c0754e0 0x7f567c19b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:54.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.684+0000 7f567be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567c075e00 0x7f567c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:54.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.684+0000 7f567be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f567c10f200 0x7f567c1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:54.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f567c114c50 con 0x7f567c10f200 2024-09-10T22:15:54.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567be00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f567c114ad0 con 0x7f567c075e00 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567be00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f567c114dd0 con 0x7f567c0754e0 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f567c0754e0 0x7f567c19b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f567c10f200 0x7f567c1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567aa00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f567c0754e0 0x7f567c19b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:48810/0 (socket says 172.21.15.90:48810) 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f5673e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567c075e00 0x7f567c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567aa00640 1 -- 172.21.15.90:0/2667793449 learned_addr learned my addr 172.21.15.90:0/2667793449 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567aa00640 1 -- 172.21.15.90:0/2667793449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567c075e00 msgr2=0x7f567c19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567aa00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567c075e00 0x7f567c19b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567aa00640 1 -- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f567c10f200 msgr2=0x7f567c1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567aa00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f567c10f200 0x7f567c1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567aa00640 1 -- 172.21.15.90:0/2667793449 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c1a1210 con 0x7f567c0754e0 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f567b400640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f567c10f200 0x7f567c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:15:54.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.685+0000 7f5673e00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567c075e00 0x7f567c19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:15:54.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.686+0000 7f567aa00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f567c0754e0 0x7f567c19b160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f566c009230 tx=0x7f566c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:54.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.686+0000 7f5673400640 1 -- 172.21.15.90:0/2667793449 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c03f030 con 0x7f567c0754e0 2024-09-10T22:15:54.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.686+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c1a14a0 con 0x7f567c0754e0 2024-09-10T22:15:54.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.686+0000 7f5673400640 1 -- 172.21.15.90:0/2667793449 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f566c003a80 con 0x7f567c0754e0 2024-09-10T22:15:54.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.686+0000 7f5673400640 1 -- 172.21.15.90:0/2667793449 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c035070 con 0x7f567c0754e0 2024-09-10T22:15:54.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.686+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c1a7bc0 con 0x7f567c0754e0 2024-09-10T22:15:54.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.687+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567c0758c0 con 0x7f567c0754e0 2024-09-10T22:15:54.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.691+0000 7f5673400640 1 -- 172.21.15.90:0/2667793449 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f566c003c30 con 0x7f567c0754e0 2024-09-10T22:15:54.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.693+0000 7f5673400640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56600b5500 0x7f56600b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:54.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.693+0000 7f5673400640 1 -- 172.21.15.90:0/2667793449 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7f566c107070 con 0x7f567c0754e0 2024-09-10T22:15:54.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.693+0000 7f5673e00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56600b5500 0x7f56600b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:54.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.694+0000 7f5673400640 1 -- 172.21.15.90:0/2667793449 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f566c0cd170 con 0x7f567c0754e0 2024-09-10T22:15:54.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.694+0000 7f5673e00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56600b5500 0x7f56600b79c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f567c1117a0 tx=0x7f5664005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:54.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.968+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f567c1a7e20 con 0x7f567c0754e0 2024-09-10T22:15:54.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.969+0000 7f5673400640 1 -- 172.21.15.90:0/2667793449 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+12003 (secure 0 0 0) 0x7f566c0d1ba0 con 0x7f567c0754e0 2024-09-10T22:15:54.971 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:15:54.971 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"btime":"2024-09-10T22:15:01:892115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:15:00.940095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:15:54.971 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 41 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56600b5500 msgr2=0x7f56600b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56600b5500 0x7f56600b79c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f567c1117a0 tx=0x7f5664005f90 comp rx=0 tx=0).stop 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f567c0754e0 msgr2=0x7f567c19b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f567c0754e0 0x7f567c19b160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f566c009230 tx=0x7f566c004270 comp rx=0 tx=0).stop 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 shutdown_connections 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f56600b5500 0x7f56600b79c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f567c10f200 0x7f567c1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f567c075e00 0x7f567c19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 --2- 172.21.15.90:0/2667793449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f567c0754e0 0x7f567c19b160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 >> 172.21.15.90:0/2667793449 conn(0x7f567c0fe660 msgr2=0x7f567c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 shutdown_connections 2024-09-10T22:15:54.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:54.972+0000 7f567be00640 1 -- 172.21.15.90:0/2667793449 wait complete. 2024-09-10T22:15:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:55 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2667793449' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:15:55.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:55 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2667793449' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:15:55.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:55 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2667793449' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:15:55.504 INFO:tasks.workunit.client.0.smithi090.stdout: 1 137295 1.10 MB/sec execute 527 sec latency 19.168 ms 2024-09-10T22:15:55.967 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0' 2024-09-10T22:15:55.967 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 145.4 --await-for 45.0 2024-09-10T22:15:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:56 smithi090 ceph-mon[30350]: pgmap v791: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 58 KiB/s wr, 19 op/s 2024-09-10T22:15:56.394 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:15:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:56 smithi154 ceph-mon[34495]: pgmap v791: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 58 KiB/s wr, 19 op/s 2024-09-10T22:15:56.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:56 smithi171 ceph-mon[34138]: pgmap v791: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 58 KiB/s wr, 19 op/s 2024-09-10T22:15:56.504 INFO:tasks.workunit.client.0.smithi090.stdout: 1 137790 1.11 MB/sec execute 528 sec latency 18.160 ms 2024-09-10T22:15:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.029+0000 7fdbcfe00640 1 -- 172.21.15.90:0/1086338805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd0102020 msgr2=0x7fdbd010ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.029+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/1086338805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd0102020 0x7fdbd010ed70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009900 tx=0x7fdbb802a480 comp rx=0 tx=0).stop 2024-09-10T22:15:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.029+0000 7fdbcfe00640 1 -- 172.21.15.90:0/1086338805 shutdown_connections 2024-09-10T22:15:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.029+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/1086338805 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd010f2b0 0x7fdbd01116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.029+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/1086338805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd0102020 0x7fdbd010ed70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.029+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/1086338805 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd0101700 0x7fdbd0101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.029+0000 7fdbcfe00640 1 -- 172.21.15.90:0/1086338805 >> 172.21.15.90:0/1086338805 conn(0x7fdbd00fd5b0 msgr2=0x7fdbd00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:57.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.029+0000 7fdbcfe00640 1 -- 172.21.15.90:0/1086338805 shutdown_connections 2024-09-10T22:15:57.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.029+0000 7fdbcfe00640 1 -- 172.21.15.90:0/1086338805 wait complete. 2024-09-10T22:15:57.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.030+0000 7fdbcfe00640 1 Processor -- start 2024-09-10T22:15:57.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.030+0000 7fdbcfe00640 1 -- start start 2024-09-10T22:15:57.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcfe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd0101700 0x7fdbd019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd0102020 0x7fdbd019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcfe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd010f2b0 0x7fdbd01a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcfe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0114cf0 con 0x7fdbd0102020 2024-09-10T22:15:57.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcfe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0114b70 con 0x7fdbd0101700 2024-09-10T22:15:57.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcfe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0114e70 con 0x7fdbd010f2b0 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd0101700 0x7fdbd019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd0101700 0x7fdbd019b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:42840/0 (socket says 172.21.15.90:42840) 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcea00640 1 -- 172.21.15.90:0/400544060 learned_addr learned my addr 172.21.15.90:0/400544060 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbc7e00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd0102020 0x7fdbd019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcf400640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd010f2b0 0x7fdbd01a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcea00640 1 -- 172.21.15.90:0/400544060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd010f2b0 msgr2=0x7fdbd01a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcea00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd010f2b0 0x7fdbd01a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcea00640 1 -- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd0102020 msgr2=0x7fdbd019b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcea00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd0102020 0x7fdbd019b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcea00640 1 -- 172.21.15.90:0/400544060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd01a1260 con 0x7fdbd0101700 2024-09-10T22:15:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.031+0000 7fdbcf400640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd010f2b0 0x7fdbd01a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.032+0000 7fdbcea00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd0101700 0x7fdbd019b1b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000e9f0 tx=0x7fdbc000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.032+0000 7fdbc7400640 1 -- 172.21.15.90:0/400544060 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc000cd80 con 0x7fdbd0101700 2024-09-10T22:15:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.032+0000 7fdbc7400640 1 -- 172.21.15.90:0/400544060 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc00054b0 con 0x7fdbd0101700 2024-09-10T22:15:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.032+0000 7fdbcfe00640 1 -- 172.21.15.90:0/400544060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd01a1550 con 0x7fdbd0101700 2024-09-10T22:15:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.032+0000 7fdbc7400640 1 -- 172.21.15.90:0/400544060 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0010430 con 0x7fdbd0101700 2024-09-10T22:15:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.032+0000 7fdbcfe00640 1 -- 172.21.15.90:0/400544060 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd01a7c10 con 0x7fdbd0101700 2024-09-10T22:15:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.036+0000 7fdbd8405740 1 Processor -- start 2024-09-10T22:15:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.036+0000 7fdbc7400640 1 -- 172.21.15.90:0/400544060 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fdbc0005020 con 0x7fdbd0101700 2024-09-10T22:15:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.036+0000 7fdbd8405740 1 -- start start 2024-09-10T22:15:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.036+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 0x55cb0b1f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.036+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b201660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.036+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b201ba0 0x55cb0b203f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.036+0000 7fdbd8405740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cb0b16aca0 con 0x55cb0b1f52f0 2024-09-10T22:15:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.036+0000 7fdbd8405740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cb0b15a0d0 con 0x55cb0b201ba0 2024-09-10T22:15:57.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.036+0000 7fdbd8405740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55cb0b15dee0 con 0x55cb0b1f5c10 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbcf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 0x55cb0b1f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbcea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b201660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbcf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 0x55cb0b1f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50414/0 (socket says 172.21.15.90:50414) 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b201ba0 0x55cb0b203f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbcf400640 1 -- 172.21.15.90:0/563842594 learned_addr learned my addr 172.21.15.90:0/563842594 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbc7e00640 1 -- 172.21.15.90:0/563842594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 msgr2=0x55cb0b201660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbc7e00640 1 --2- 172.21.15.90:0/563842594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b201660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbc7e00640 1 -- 172.21.15.90:0/563842594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 msgr2=0x55cb0b1f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbc7e00640 1 --2- 172.21.15.90:0/563842594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 0x55cb0b1f56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.037+0000 7fdbc7e00640 1 -- 172.21.15.90:0/563842594 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb0b10c400 con 0x55cb0b201ba0 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.038+0000 7fdbc7e00640 1 --2- 172.21.15.90:0/563842594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b201ba0 0x55cb0b203f90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbb802a9a0 tx=0x7fdbb8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a57f05e856c02402 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:57.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.038+0000 7fdbc7400640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdbb40b5500 0x7fdbb40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.038+0000 7fdbbfe00640 1 -- 172.21.15.90:0/563842594 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8040070 con 0x55cb0b201ba0 2024-09-10T22:15:57.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.038+0000 7fdbbfe00640 1 -- 172.21.15.90:0/563842594 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbb8004450 con 0x55cb0b201ba0 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.038+0000 7fdbbfe00640 1 -- 172.21.15.90:0/563842594 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8035040 con 0x55cb0b201ba0 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.038+0000 7fdbc7400640 1 -- 172.21.15.90:0/400544060 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7fdbc00e6880 con 0x7fdbd0101700 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.038+0000 7fdbcf400640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdbb40b5500 0x7fdbb40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbd8405740 1 -- 172.21.15.90:0/563842594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b201ba0 msgr2=0x55cb0b203f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbd8405740 1 --2- 172.21.15.90:0/563842594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b201ba0 0x55cb0b203f90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdbb802a9a0 tx=0x7fdbb8000bc0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbd8405740 1 -- 172.21.15.90:0/563842594 shutdown_connections 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbd8405740 1 --2- 172.21.15.90:0/563842594 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b201ba0 0x55cb0b203f90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbd8405740 1 --2- 172.21.15.90:0/563842594 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b201660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbd8405740 1 --2- 172.21.15.90:0/563842594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 0x55cb0b1f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbd8405740 1 -- 172.21.15.90:0/563842594 >> 172.21.15.90:0/563842594 conn(0x55cb0b1f3120 msgr2=0x55cb0b1f42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbcf400640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdbb40b5500 0x7fdbb40b79c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8011670 tx=0x7fdbc8009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbd8405740 1 -- 172.21.15.90:0/563842594 shutdown_connections 2024-09-10T22:15:57.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.039+0000 7fdbd8405740 1 -- 172.21.15.90:0/563842594 wait complete. 2024-09-10T22:15:57.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.040+0000 7fdbd8405740 1 -- start start 2024-09-10T22:15:57.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.040+0000 7fdbd8405740 1 Processor -- start 2024-09-10T22:15:57.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.040+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f52f0 0x55cb0b2014a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.040+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b1fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.040+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b201ba0 0x55cb0b1fef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.040+0000 7fdbd8405740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cb0afbb7a0 con 0x55cb0b201ba0 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.040+0000 7fdbd8405740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55cb0b0ffa10 con 0x55cb0b1f52f0 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.040+0000 7fdbd8405740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55cb0b2047c0 con 0x55cb0b1f5c10 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbd8405740 10 client.0 _ll_register_callbacks cb 0x55cb0b10fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbcf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b201ba0 0x55cb0b1fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b1fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b1fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:48848/0 (socket says 172.21.15.90:48848) 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbcea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f52f0 0x55cb0b2014a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbc7e00640 1 -- 172.21.15.90:0/1745944945 learned_addr learned my addr 172.21.15.90:0/1745944945 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbc7e00640 1 -- 172.21.15.90:0/1745944945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f52f0 msgr2=0x55cb0b2014a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbc7e00640 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f52f0 0x55cb0b2014a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbc7e00640 1 -- 172.21.15.90:0/1745944945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b201ba0 msgr2=0x55cb0b1fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbc7e00640 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b201ba0 0x55cb0b1fef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.041+0000 7fdbc7e00640 1 -- 172.21.15.90:0/1745944945 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0aec7b90 con 0x55cb0b1f5c10 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.042+0000 7fdbc7e00640 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b1fe8f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009430 tx=0x7fdbb8004e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c3e7e53851c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.042+0000 7fdbbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.042+0000 7fdbbcc00640 1 -- 172.21.15.90:0/1745944945 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8035070 con 0x55cb0b1f5c10 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.042+0000 7fdbbcc00640 1 -- 172.21.15.90:0/1745944945 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7fdbb80048d0 con 0x55cb0b1f5c10 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.042+0000 7fdbd8405740 1 -- 172.21.15.90:0/1745944945 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbb8004450 con 0x55cb0b1f5c10 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbbcc00640 1 -- 172.21.15.90:0/1745944945 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbb8058050 con 0x55cb0b1f5c10 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbd8405740 10 client.25777 fetch_fsmap learned FSMap version 41 2024-09-10T22:15:57.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbd8405740 1 -- 172.21.15.90:0/1745944945 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cb0b210110 con 0x55cb0b1f5c10 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbbcc00640 1 -- 172.21.15.90:0/1745944945 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbb8004450 con 0x55cb0b1f5c10 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbbcc00640 1 -- 172.21.15.90:0/1745944945 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 41) ==== 8410+0+0 (secure 0 0 0) 0x7fdbb8003b60 con 0x55cb0b1f5c10 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbd8405740 10 client.25777 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbd8405740 10 client.25777 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbd8405740 20 client.25777 populate_metadata read hostname 'smithi090' 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbd8405740 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55cb0b211460 0x55cb0b213850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbd8405740 4 client.25777 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.043+0000 7fdbd8405740 1 -- 172.21.15.90:0/1745944945 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb0b16cbf0 con 0x55cb0b211460 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.044+0000 7fdbcea00640 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55cb0b211460 0x55cb0b213850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.044+0000 7fdbbcc00640 10 client.25777 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:57.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.044+0000 7fdbcea00640 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55cb0b211460 0x55cb0b213850 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c00c0531ad711c server_cookie=70c0b1d8b2b6edfb in_seq=0 out_seq=0 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbbcc00640 1 -- 172.21.15.90:0/1745944945 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cb0b16cbf0 con 0x55cb0b211460 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbbcc00640 10 client.25777 handle_command_reply: tid=0 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbd8405740 1 client.25777 shutdown 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbd8405740 1 -- 172.21.15.90:0/1745944945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 msgr2=0x55cb0b1fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbd8405740 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b1fe8f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009430 tx=0x7fdbb8004e90 comp rx=0 tx=0).stop 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbd8405740 1 -- 172.21.15.90:0/1745944945 shutdown_connections 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbd8405740 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55cb0b211460 0x55cb0b213850 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbd8405740 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b201ba0 0x55cb0b1fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbd8405740 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f5c10 0x55cb0b1fe8f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbd8405740 1 --2- 172.21.15.90:0/1745944945 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f52f0 0x55cb0b2014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbbcc00640 0 client.25777 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.045+0000 7fdbd8405740 1 -- 172.21.15.90:0/1745944945 >> 172.21.15.90:0/1745944945 conn(0x55cb0b1f3120 msgr2=0x55cb0b1f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.046+0000 7fdbd8405740 1 -- 172.21.15.90:0/1745944945 shutdown_connections 2024-09-10T22:15:57.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.046+0000 7fdbd8405740 1 -- 172.21.15.90:0/1745944945 wait complete. 2024-09-10T22:15:57.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.046+0000 7fdbd8405740 20 client.25777 trim_cache size 0 max 16384 2024-09-10T22:15:57.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.068+0000 7fdbd8405740 1 Processor -- start 2024-09-10T22:15:57.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbd8405740 1 -- start start 2024-09-10T22:15:57.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f3120 0x55cb0b1f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 0x55cb0b1fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 0x55cb0b202f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbd8405740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01b1d60 con 0x55cb0b1f52f0 2024-09-10T22:15:57.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbd8405740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01b3a80 con 0x55cb0b1f5c10 2024-09-10T22:15:57.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbd8405740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01b2430 con 0x55cb0b1f3120 2024-09-10T22:15:57.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbcea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f3120 0x55cb0b1f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbcf400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 0x55cb0b202f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbc7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 0x55cb0b1fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbcea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f3120 0x55cb0b1f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:48864/0 (socket says 172.21.15.90:48864) 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.069+0000 7fdbcea00640 1 -- 172.21.15.90:0/2459859522 learned_addr learned my addr 172.21.15.90:0/2459859522 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.070+0000 7fdbcea00640 1 -- 172.21.15.90:0/2459859522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 msgr2=0x55cb0b202f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.070+0000 7fdbcea00640 1 --2- 172.21.15.90:0/2459859522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 0x55cb0b202f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.070+0000 7fdbcea00640 1 -- 172.21.15.90:0/2459859522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 msgr2=0x55cb0b1fdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.070+0000 7fdbcea00640 1 --2- 172.21.15.90:0/2459859522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 0x55cb0b1fdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.070+0000 7fdbcea00640 1 -- 172.21.15.90:0/2459859522 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb0b1fa230 con 0x55cb0b1f3120 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.070+0000 7fdbcea00640 1 --2- 172.21.15.90:0/2459859522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f3120 0x55cb0b1f2af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00e3c00 tx=0x7fdbc00a8040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e5eae31dab776f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.070+0000 7fdbbf400640 1 -- 172.21.15.90:0/2459859522 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00e9e60 con 0x55cb0b1f3120 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.071+0000 7fdbbf400640 1 -- 172.21.15.90:0/2459859522 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc00ac3e0 con 0x55cb0b1f3120 2024-09-10T22:15:57.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.071+0000 7fdbbf400640 1 -- 172.21.15.90:0/2459859522 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00acfd0 con 0x55cb0b1f3120 2024-09-10T22:15:57.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.071+0000 7fdbd8405740 1 -- 172.21.15.90:0/2459859522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f3120 msgr2=0x55cb0b1f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:57.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.071+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2459859522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f3120 0x55cb0b1f2af0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00e3c00 tx=0x7fdbc00a8040 comp rx=0 tx=0).stop 2024-09-10T22:15:57.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.071+0000 7fdbd8405740 1 -- 172.21.15.90:0/2459859522 shutdown_connections 2024-09-10T22:15:57.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.071+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2459859522 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 0x55cb0b202f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.071+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2459859522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f52f0 0x55cb0b1fdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.071+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2459859522 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f3120 0x55cb0b1f2af0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.071+0000 7fdbd8405740 1 -- 172.21.15.90:0/2459859522 >> 172.21.15.90:0/2459859522 conn(0x55cb0b1fe4c0 msgr2=0x55cb0b1fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.072+0000 7fdbd8405740 1 -- 172.21.15.90:0/2459859522 shutdown_connections 2024-09-10T22:15:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.072+0000 7fdbd8405740 1 -- 172.21.15.90:0/2459859522 wait complete. 2024-09-10T22:15:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.072+0000 7fdbd8405740 1 -- start start 2024-09-10T22:15:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.072+0000 7fdbd8405740 1 Processor -- start 2024-09-10T22:15:57.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f3120 0x55cb0b20a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f52f0 0x55cb0b204a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbd8405740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 0x55cb0b204f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbd8405740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01b32f0 con 0x55cb0b1f3120 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbd8405740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01b2b60 con 0x55cb0b1f5c10 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbd8405740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd01afd60 con 0x55cb0b1f52f0 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbd8405740 10 client.0 _ll_register_callbacks cb 0x7fdbc8009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbcea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f3120 0x55cb0b20a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbcf400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 0x55cb0b204f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbcea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f3120 0x55cb0b20a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50446/0 (socket says 172.21.15.90:50446) 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbcea00640 1 -- 172.21.15.90:0/2880313719 learned_addr learned my addr 172.21.15.90:0/2880313719 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.073+0000 7fdbc7e00640 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f52f0 0x55cb0b204a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbcf400640 1 -- 172.21.15.90:0/2880313719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f52f0 msgr2=0x55cb0b204a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbcf400640 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f52f0 0x55cb0b204a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbcf400640 1 -- 172.21.15.90:0/2880313719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f3120 msgr2=0x55cb0b20a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbcf400640 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f3120 0x55cb0b20a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbcf400640 1 -- 172.21.15.90:0/2880313719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb0b1ff270 con 0x55cb0b1f5c10 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbc7e00640 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f52f0 0x55cb0b204a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbcea00640 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f3120 0x55cb0b20a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbcf400640 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 0x55cb0b204f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009a10 tx=0x7fdbc8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b8abd5fb50d9e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbbcc00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbbcc00640 1 -- 172.21.15.90:0/2880313719 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc801eb80 con 0x55cb0b1f5c10 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbd8405740 1 -- 172.21.15.90:0/2880313719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc00ac3e0 con 0x55cb0b1f5c10 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.074+0000 7fdbbcc00640 1 -- 172.21.15.90:0/2880313719 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9490+0+0 (secure 0 0 0) 0x7fdbc801ed20 con 0x55cb0b1f5c10 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbbcc00640 1 -- 172.21.15.90:0/2880313719 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fdbc8004b20 con 0x55cb0b1f5c10 2024-09-10T22:15:57.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbd8405740 10 client.35810 fetch_fsmap learned FSMap version 41 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbd8405740 1 -- 172.21.15.90:0/2880313719 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55cb0b27ef30 con 0x55cb0b1f5c10 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbbcc00640 1 -- 172.21.15.90:0/2880313719 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdbc00ac3e0 con 0x55cb0b1f5c10 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbbcc00640 1 -- 172.21.15.90:0/2880313719 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 41) ==== 8410+0+0 (secure 0 0 0) 0x7fdbc8005740 con 0x55cb0b1f5c10 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbd8405740 10 client.35810 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbd8405740 10 client.35810 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbd8405740 20 client.35810 populate_metadata read hostname 'smithi090' 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55cb0b201ba0 0x55cb0b281820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbd8405740 4 client.35810 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}] 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.075+0000 7fdbd8405740 1 -- 172.21.15.90:0/2880313719 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.4, "await_for": 45.0}) -- 0x55cb0b1686a0 con 0x55cb0b201ba0 2024-09-10T22:15:57.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.076+0000 7fdbcea00640 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55cb0b201ba0 0x55cb0b281820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:15:57.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.076+0000 7fdbcea00640 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55cb0b201ba0 0x55cb0b281820 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361513d06b3c268b server_cookie=73fb08197bb72315 in_seq=0 out_seq=0 2024-09-10T22:15:57.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.076+0000 7fdbbcc00640 10 client.35810 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.155+0000 7fdbbcc00640 1 -- 172.21.15.90:0/2880313719 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cb0b1686a0 con 0x55cb0b201ba0 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.155+0000 7fdbbcc00640 10 client.35810 handle_command_reply: tid=0 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.155+0000 7fdbd8405740 1 client.35810 shutdown 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbd8405740 1 -- 172.21.15.90:0/2880313719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 msgr2=0x55cb0b204f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 0x55cb0b204f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009a10 tx=0x7fdbc8004a40 comp rx=0 tx=0).stop 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbd8405740 1 -- 172.21.15.90:0/2880313719 shutdown_connections 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55cb0b201ba0 0x55cb0b281820 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55cb0b1f5c10 0x55cb0b204f70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55cb0b1f52f0 0x55cb0b204a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbd8405740 1 --2- 172.21.15.90:0/2880313719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cb0b1f3120 0x55cb0b20a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbd8405740 1 -- 172.21.15.90:0/2880313719 >> 172.21.15.90:0/2880313719 conn(0x55cb0b1fe4c0 msgr2=0x55cb0b1fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:57.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbbcc00640 0 client.35810 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:15:57.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.156+0000 7fdbd8405740 1 -- 172.21.15.90:0/2880313719 shutdown_connections 2024-09-10T22:15:57.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.157+0000 7fdbd8405740 1 -- 172.21.15.90:0/2880313719 wait complete. 2024-09-10T22:15:57.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.157+0000 7fdbd8405740 20 client.35810 trim_cache size 0 max 16384 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 41, 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 38, 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "33c49c9b": { 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 38, 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:15:57.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 145.4, 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:15:57.161 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:15:57.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 -- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdbb40b5500 msgr2=0x7fdbb40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:57.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdbb40b5500 0x7fdbb40b79c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8011670 tx=0x7fdbc8009500 comp rx=0 tx=0).stop 2024-09-10T22:15:57.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 -- 172.21.15.90:0/400544060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd0101700 msgr2=0x7fdbd019b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:15:57.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd0101700 0x7fdbd019b1b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000e9f0 tx=0x7fdbc000edf0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 -- 172.21.15.90:0/400544060 shutdown_connections 2024-09-10T22:15:57.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fdbb40b5500 0x7fdbb40b79c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fdbd010f2b0 0x7fdbd01a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbd0102020 0x7fdbd019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 --2- 172.21.15.90:0/400544060 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fdbd0101700 0x7fdbd019b1b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:15:57.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 -- 172.21.15.90:0/400544060 >> 172.21.15.90:0/400544060 conn(0x7fdbd00fd5b0 msgr2=0x7fdbd01106a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:15:57.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 -- 172.21.15.90:0/400544060 shutdown_connections 2024-09-10T22:15:57.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:15:57.158+0000 7fdbcfe00640 1 -- 172.21.15.90:0/400544060 wait complete. 2024-09-10T22:15:57.505 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.11 MB/sec execute 529 sec latency 284.406 ms 2024-09-10T22:15:58.149 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '33c49c9b', quiesce took 4.5 seconds. Will release after: 20.9 2024-09-10T22:15:58.505 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.11 MB/sec execute 530 sec latency 1284.580 ms 2024-09-10T22:15:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:15:58 smithi154 ceph-mon[34495]: pgmap v792: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 107 KiB/s wr, 17 op/s 2024-09-10T22:15:59.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:15:58 smithi090 ceph-mon[30350]: pgmap v792: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 107 KiB/s wr, 17 op/s 2024-09-10T22:15:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:15:58 smithi171 ceph-mon[34138]: pgmap v792: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 107 KiB/s wr, 17 op/s 2024-09-10T22:15:59.505 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.10 MB/sec execute 531 sec latency 2284.648 ms 2024-09-10T22:16:00.505 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.10 MB/sec execute 532 sec latency 3284.734 ms 2024-09-10T22:16:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:00 smithi154 ceph-mon[34495]: pgmap v793: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 485 KiB/s wr, 47 op/s 2024-09-10T22:16:01.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:00 smithi171 ceph-mon[34138]: pgmap v793: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 485 KiB/s wr, 47 op/s 2024-09-10T22:16:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:00 smithi090 ceph-mon[30350]: pgmap v793: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 485 KiB/s wr, 47 op/s 2024-09-10T22:16:01.505 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.10 MB/sec execute 533 sec latency 4284.809 ms 2024-09-10T22:16:02.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:01 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:16:02.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:01 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:16:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:01 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:16:02.505 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.10 MB/sec execute 534 sec latency 5284.906 ms 2024-09-10T22:16:03.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:02 smithi154 ceph-mon[34495]: pgmap v794: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 488 KiB/s wr, 44 op/s 2024-09-10T22:16:03.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:16:03.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:16:03.170 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:16:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:02 smithi171 ceph-mon[34138]: pgmap v794: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 488 KiB/s wr, 44 op/s 2024-09-10T22:16:03.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:16:03.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:16:03.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:16:03.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:02 smithi090 ceph-mon[30350]: pgmap v794: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 488 KiB/s wr, 44 op/s 2024-09-10T22:16:03.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:16:03.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:16:03.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:16:03.505 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.10 MB/sec execute 535 sec latency 6285.019 ms 2024-09-10T22:16:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:03 smithi154 ceph-mon[34495]: pgmap v795: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 531 KiB/s wr, 62 op/s 2024-09-10T22:16:04.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:03 smithi171 ceph-mon[34138]: pgmap v795: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 531 KiB/s wr, 62 op/s 2024-09-10T22:16:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:03 smithi090 ceph-mon[30350]: pgmap v795: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 531 KiB/s wr, 62 op/s 2024-09-10T22:16:04.506 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.09 MB/sec execute 536 sec latency 7285.105 ms 2024-09-10T22:16:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:04 smithi171 ceph-mon[34138]: osdmap e103: 12 total, 12 up, 12 in 2024-09-10T22:16:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:04 smithi090 ceph-mon[30350]: osdmap e103: 12 total, 12 up, 12 in 2024-09-10T22:16:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:04 smithi154 ceph-mon[34495]: osdmap e103: 12 total, 12 up, 12 in 2024-09-10T22:16:05.508 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.09 MB/sec execute 537 sec latency 8285.232 ms 2024-09-10T22:16:06.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:05 smithi171 ceph-mon[34138]: pgmap v797: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 648 KiB/s wr, 66 op/s 2024-09-10T22:16:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:05 smithi090 ceph-mon[30350]: pgmap v797: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 648 KiB/s wr, 66 op/s 2024-09-10T22:16:06.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:05 smithi154 ceph-mon[34495]: pgmap v797: 129 pgs: 129 active+clean; 182 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 648 KiB/s wr, 66 op/s 2024-09-10T22:16:06.506 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.09 MB/sec execute 538 sec latency 9285.354 ms 2024-09-10T22:16:07.506 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.09 MB/sec execute 539 sec latency 10285.460 ms 2024-09-10T22:16:08.506 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.09 MB/sec execute 540 sec latency 11285.576 ms 2024-09-10T22:16:09.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:08 smithi154 ceph-mon[34495]: pgmap v798: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 627 KiB/s wr, 64 op/s 2024-09-10T22:16:09.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:08 smithi171 ceph-mon[34138]: pgmap v798: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 627 KiB/s wr, 64 op/s 2024-09-10T22:16:09.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:08 smithi090 ceph-mon[30350]: pgmap v798: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 627 KiB/s wr, 64 op/s 2024-09-10T22:16:09.506 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.08 MB/sec execute 541 sec latency 12285.710 ms 2024-09-10T22:16:10.506 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.08 MB/sec execute 542 sec latency 13285.809 ms 2024-09-10T22:16:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:10 smithi090 ceph-mon[30350]: pgmap v799: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 191 KiB/s wr, 35 op/s 2024-09-10T22:16:11.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:10 smithi090 ceph-mon[30350]: osdmap e104: 12 total, 12 up, 12 in 2024-09-10T22:16:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:10 smithi154 ceph-mon[34495]: pgmap v799: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 191 KiB/s wr, 35 op/s 2024-09-10T22:16:11.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:10 smithi154 ceph-mon[34495]: osdmap e104: 12 total, 12 up, 12 in 2024-09-10T22:16:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:10 smithi171 ceph-mon[34138]: pgmap v799: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 191 KiB/s wr, 35 op/s 2024-09-10T22:16:11.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:10 smithi171 ceph-mon[34138]: osdmap e104: 12 total, 12 up, 12 in 2024-09-10T22:16:11.506 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.08 MB/sec execute 543 sec latency 14285.874 ms 2024-09-10T22:16:12.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.08 MB/sec execute 544 sec latency 15286.051 ms 2024-09-10T22:16:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:12 smithi154 ceph-mon[34495]: pgmap v801: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 161 KiB/s wr, 15 op/s 2024-09-10T22:16:13.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:12 smithi171 ceph-mon[34138]: pgmap v801: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 161 KiB/s wr, 15 op/s 2024-09-10T22:16:13.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:12 smithi090 ceph-mon[30350]: pgmap v801: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 161 KiB/s wr, 15 op/s 2024-09-10T22:16:13.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.08 MB/sec execute 545 sec latency 16286.200 ms 2024-09-10T22:16:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:13 smithi171 ceph-mon[34138]: pgmap v802: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 130 KiB/s wr, 13 op/s 2024-09-10T22:16:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:13 smithi090 ceph-mon[30350]: pgmap v802: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 130 KiB/s wr, 13 op/s 2024-09-10T22:16:14.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:13 smithi154 ceph-mon[34495]: pgmap v802: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 130 KiB/s wr, 13 op/s 2024-09-10T22:16:14.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.07 MB/sec execute 546 sec latency 17286.271 ms 2024-09-10T22:16:15.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.07 MB/sec execute 547 sec latency 18286.411 ms 2024-09-10T22:16:16.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.07 MB/sec execute 548 sec latency 19286.552 ms 2024-09-10T22:16:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:16 smithi154 ceph-mon[34495]: pgmap v803: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 KiB/s wr, 11 op/s 2024-09-10T22:16:17.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:16 smithi171 ceph-mon[34138]: pgmap v803: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 KiB/s wr, 11 op/s 2024-09-10T22:16:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:16 smithi090 ceph-mon[30350]: pgmap v803: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 56 KiB/s wr, 11 op/s 2024-09-10T22:16:17.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.07 MB/sec execute 549 sec latency 20286.678 ms 2024-09-10T22:16:18.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.07 MB/sec execute 550 sec latency 21286.786 ms 2024-09-10T22:16:19.050 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:16:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:18 smithi154 ceph-mon[34495]: pgmap v804: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-10T22:16:19.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:18 smithi171 ceph-mon[34138]: pgmap v804: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-10T22:16:19.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:18 smithi090 ceph-mon[30350]: pgmap v804: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-09-10T22:16:19.433 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:16:19.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.06 MB/sec execute 551 sec latency 22286.937 ms 2024-09-10T22:16:20.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.063+0000 7f1cf1000640 1 -- 172.21.15.90:0/1186574792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec105ef0 msgr2=0x7f1cec10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:20.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.063+0000 7f1cf1000640 1 --2- 172.21.15.90:0/1186574792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec105ef0 0x7f1cec10f930 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009ca0 tx=0x7f1ce402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.064+0000 7f1cf1000640 1 -- 172.21.15.90:0/1186574792 shutdown_connections 2024-09-10T22:16:20.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.064+0000 7f1cf1000640 1 --2- 172.21.15.90:0/1186574792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec105ef0 0x7f1cec10f930 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.064+0000 7f1cf1000640 1 --2- 172.21.15.90:0/1186574792 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec105570 0x7f1cec1059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.064+0000 7f1cf1000640 1 --2- 172.21.15.90:0/1186574792 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cec101220 0x7f1cec101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.064+0000 7f1cf1000640 1 -- 172.21.15.90:0/1186574792 >> 172.21.15.90:0/1186574792 conn(0x7f1cec078100 msgr2=0x7f1cec0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:20.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.064+0000 7f1cf1000640 1 -- 172.21.15.90:0/1186574792 shutdown_connections 2024-09-10T22:16:20.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.064+0000 7f1cf1000640 1 -- 172.21.15.90:0/1186574792 wait complete. 2024-09-10T22:16:20.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.065+0000 7f1cf1000640 1 Processor -- start 2024-09-10T22:16:20.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.065+0000 7f1cf1000640 1 -- start start 2024-09-10T22:16:20.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1cf1000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec101220 0x7f1cec1021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:20.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1cf1000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cec105570 0x7f1cec102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1cf1000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec105ef0 0x7f1cec102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1cf1000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec112cb0 con 0x7f1cec101220 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1cf1000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec112b30 con 0x7f1cec105570 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1cf1000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec112e30 con 0x7f1cec105ef0 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1ce9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec101220 0x7f1cec1021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1ce9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec101220 0x7f1cec1021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33172/0 (socket says 172.21.15.90:33172) 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1ce9e00640 1 -- 172.21.15.90:0/2613952825 learned_addr learned my addr 172.21.15.90:0/2613952825 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.066+0000 7f1ce9400640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cec105570 0x7f1cec102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1cea800640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec105ef0 0x7f1cec102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1ce9e00640 1 -- 172.21.15.90:0/2613952825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec105ef0 msgr2=0x7f1cec102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1ce9e00640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec105ef0 0x7f1cec102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1ce9e00640 1 -- 172.21.15.90:0/2613952825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cec105570 msgr2=0x7f1cec102730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:20.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1ce9e00640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cec105570 0x7f1cec102730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1ce9e00640 1 -- 172.21.15.90:0/2613952825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec1a7620 con 0x7f1cec101220 2024-09-10T22:16:20.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1ce9e00640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec101220 0x7f1cec1021f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00d950 tx=0x7f1cdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:20.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1ce2a00640 1 -- 172.21.15.90:0/2613952825 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc017030 con 0x7f1cec101220 2024-09-10T22:16:20.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1ce2a00640 1 -- 172.21.15.90:0/2613952825 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1cdc004020 con 0x7f1cec101220 2024-09-10T22:16:20.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1ce2a00640 1 -- 172.21.15.90:0/2613952825 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0055d0 con 0x7f1cec101220 2024-09-10T22:16:20.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.067+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec1a7910 con 0x7f1cec101220 2024-09-10T22:16:20.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.068+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1a7ea0 con 0x7f1cec101220 2024-09-10T22:16:20.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.069+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec101600 con 0x7f1cec101220 2024-09-10T22:16:20.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.071+0000 7f1ce2a00640 1 -- 172.21.15.90:0/2613952825 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f1cdc009a30 con 0x7f1cec101220 2024-09-10T22:16:20.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.072+0000 7f1ce2a00640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1cd00b54d0 0x7f1cd00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:20.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.072+0000 7f1ce2a00640 1 -- 172.21.15.90:0/2613952825 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9490+0+0 (secure 0 0 0) 0x7f1cdc0e5db0 con 0x7f1cec101220 2024-09-10T22:16:20.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.072+0000 7f1ce9400640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1cd00b54d0 0x7f1cd00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:20.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.073+0000 7f1ce9400640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1cd00b54d0 0x7f1cd00b7990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1cec103ce0 tx=0x7f1cd800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:20.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.076+0000 7f1ce2a00640 1 -- 172.21.15.90:0/2613952825 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1cdc0ac500 con 0x7f1cec101220 2024-09-10T22:16:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:19 smithi090 ceph-mon[30350]: pgmap v805: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-09-10T22:16:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:19 smithi171 ceph-mon[34138]: pgmap v805: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-09-10T22:16:20.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.298+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1cec1034f0 con 0x7f1cec101220 2024-09-10T22:16:20.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.298+0000 7f1ce2a00640 1 -- 172.21.15.90:0/2613952825 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+12003 (secure 0 0 0) 0x7f1cdc0abe00 con 0x7f1cec101220 2024-09-10T22:16:20.300 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:16:20.301 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":41,"btime":"2024-09-10T22:15:01:892115+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:15:00.940095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:16:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 41 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1cd00b54d0 msgr2=0x7f1cd00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1cd00b54d0 0x7f1cd00b7990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f1cec103ce0 tx=0x7f1cd800b390 comp rx=0 tx=0).stop 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec101220 msgr2=0x7f1cec1021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec101220 0x7f1cec1021f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00d950 tx=0x7f1cdc00de10 comp rx=0 tx=0).stop 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 shutdown_connections 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f1cd00b54d0 0x7f1cd00b7990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f1cec105ef0 0x7f1cec102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f1cec105570 0x7f1cec102730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 --2- 172.21.15.90:0/2613952825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cec101220 0x7f1cec1021f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 >> 172.21.15.90:0/2613952825 conn(0x7f1cec078100 msgr2=0x7f1cec0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 shutdown_connections 2024-09-10T22:16:20.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:20.301+0000 7f1cf1000640 1 -- 172.21.15.90:0/2613952825 wait complete. 2024-09-10T22:16:20.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:19 smithi154 ceph-mon[34495]: pgmap v805: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 10 op/s 2024-09-10T22:16:20.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.06 MB/sec execute 552 sec latency 23287.053 ms 2024-09-10T22:16:21.244 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:20 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2613952825' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:16:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:20 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2613952825' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:16:21.283 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 33c49c9b --release --await' 2024-09-10T22:16:21.284 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 33c49c9b --release --await 2024-09-10T22:16:21.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:20 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2613952825' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:16:21.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138114 1.06 MB/sec execute 553 sec latency 24287.114 ms 2024-09-10T22:16:21.651 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:16:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:21 smithi090 ceph-mon[30350]: pgmap v806: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T22:16:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:21 smithi171 ceph-mon[34138]: pgmap v806: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T22:16:22.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.310+0000 7fa814c00640 1 -- 172.21.15.90:0/3674133449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 msgr2=0x7fa81010cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:22.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.310+0000 7fa814c00640 1 --2- 172.21.15.90:0/3674133449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 0x7fa81010cad0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa7f8009900 tx=0x7fa7f802a410 comp rx=0 tx=0).stop 2024-09-10T22:16:22.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.310+0000 7fa814c00640 1 -- 172.21.15.90:0/3674133449 shutdown_connections 2024-09-10T22:16:22.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.310+0000 7fa814c00640 1 --2- 172.21.15.90:0/3674133449 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa81010d120 0x7fa81010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.310+0000 7fa814c00640 1 --2- 172.21.15.90:0/3674133449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 0x7fa81010cad0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.310+0000 7fa814c00640 1 --2- 172.21.15.90:0/3674133449 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa810103cc0 0x7fa8101040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.310+0000 7fa814c00640 1 -- 172.21.15.90:0/3674133449 >> 172.21.15.90:0/3674133449 conn(0x7fa8100fd700 msgr2=0x7fa8100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:22.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.310+0000 7fa814c00640 1 -- 172.21.15.90:0/3674133449 shutdown_connections 2024-09-10T22:16:22.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.310+0000 7fa814c00640 1 -- 172.21.15.90:0/3674133449 wait complete. 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.311+0000 7fa814c00640 1 Processor -- start 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa814c00640 1 -- start start 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa814c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa810103cc0 0x7fa81019b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa814c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 0x7fa81019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa814c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa81010d120 0x7fa8101a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa814c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa810112cd0 con 0x7fa8101045e0 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa814c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa810112b50 con 0x7fa81010d120 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa814c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa810112e50 con 0x7fa810103cc0 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa807e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 0x7fa81019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa807e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 0x7fa81019ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33194/0 (socket says 172.21.15.90:33194) 2024-09-10T22:16:22.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa807e00640 1 -- 172.21.15.90:0/3117995546 learned_addr learned my addr 172.21.15.90:0/3117995546 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:16:22.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.312+0000 7fa80de00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa810103cc0 0x7fa81019b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa80e800640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa81010d120 0x7fa8101a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa807e00640 1 -- 172.21.15.90:0/3117995546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa810103cc0 msgr2=0x7fa81019b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:22.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa807e00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa810103cc0 0x7fa81019b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa807e00640 1 -- 172.21.15.90:0/3117995546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa81010d120 msgr2=0x7fa8101a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:22.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa807e00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa81010d120 0x7fa8101a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa807e00640 1 -- 172.21.15.90:0/3117995546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8101a1570 con 0x7fa8101045e0 2024-09-10T22:16:22.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa807e00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 0x7fa81019ba00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa7f802a930 tx=0x7fa7f8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:22.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa806a00640 1 -- 172.21.15.90:0/3117995546 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f803f070 con 0x7fa8101045e0 2024-09-10T22:16:22.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa806a00640 1 -- 172.21.15.90:0/3117995546 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7f803baf0 con 0x7fa8101045e0 2024-09-10T22:16:22.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.313+0000 7fa814c00640 1 -- 172.21.15.90:0/3117995546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8101a1800 con 0x7fa8101045e0 2024-09-10T22:16:22.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.314+0000 7fa806a00640 1 -- 172.21.15.90:0/3117995546 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8035030 con 0x7fa8101045e0 2024-09-10T22:16:22.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.314+0000 7fa814c00640 1 -- 172.21.15.90:0/3117995546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8101a7f20 con 0x7fa8101045e0 2024-09-10T22:16:22.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.317+0000 7fa81785a740 1 Processor -- start 2024-09-10T22:16:22.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.317+0000 7fa806a00640 1 -- 172.21.15.90:0/3117995546 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa7f803b660 con 0x7fa8101045e0 2024-09-10T22:16:22.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.317+0000 7fa81785a740 1 -- start start 2024-09-10T22:16:22.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa81785a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564996611ff0 0x5649966123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa81785a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa81785a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56499661e8a0 0x564996620c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa81785a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564996589290 con 0x56499661e8a0 2024-09-10T22:16:22.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa81785a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5649965786c0 con 0x564996612910 2024-09-10T22:16:22.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa81785a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56499657c4d0 con 0x564996611ff0 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa80de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa807e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56499661e8a0 0x564996620c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa80de00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:48350/0 (socket says 172.21.15.90:48350) 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa80e800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564996611ff0 0x5649966123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.318+0000 7fa80de00640 1 -- 172.21.15.90:0/526580741 learned_addr learned my addr 172.21.15.90:0/526580741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa80de00640 1 -- 172.21.15.90:0/526580741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564996611ff0 msgr2=0x5649966123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa80de00640 1 --2- 172.21.15.90:0/526580741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564996611ff0 0x5649966123d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa80de00640 1 -- 172.21.15.90:0/526580741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56499661e8a0 msgr2=0x564996620c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa80de00640 1 --2- 172.21.15.90:0/526580741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56499661e8a0 0x564996620c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa80de00640 1 -- 172.21.15.90:0/526580741 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564996505460 con 0x564996612910 2024-09-10T22:16:22.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa80de00640 1 --2- 172.21.15.90:0/526580741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661e360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa800009a70 tx=0x7fa800009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce3f0c2de1787358 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:22.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa806a00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa7f40b54d0 0x7fa7f40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa7ff000640 1 -- 172.21.15.90:0/526580741 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800017070 con 0x564996612910 2024-09-10T22:16:22.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa7ff000640 1 -- 172.21.15.90:0/526580741 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa800013470 con 0x564996612910 2024-09-10T22:16:22.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa7ff000640 1 -- 172.21.15.90:0/526580741 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800012630 con 0x564996612910 2024-09-10T22:16:22.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.319+0000 7fa806a00640 1 -- 172.21.15.90:0/3117995546 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9490+0+0 (secure 0 0 0) 0x7fa7f8106d40 con 0x7fa8101045e0 2024-09-10T22:16:22.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa80e800640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa7f40b54d0 0x7fa7f40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa81785a740 1 -- 172.21.15.90:0/526580741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 msgr2=0x56499661e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa81785a740 1 --2- 172.21.15.90:0/526580741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661e360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa800009a70 tx=0x7fa800009f30 comp rx=0 tx=0).stop 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa81785a740 1 -- 172.21.15.90:0/526580741 shutdown_connections 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa81785a740 1 --2- 172.21.15.90:0/526580741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56499661e8a0 0x564996620c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa81785a740 1 --2- 172.21.15.90:0/526580741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661e360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa81785a740 1 --2- 172.21.15.90:0/526580741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564996611ff0 0x5649966123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa81785a740 1 -- 172.21.15.90:0/526580741 >> 172.21.15.90:0/526580741 conn(0x56499660fe20 msgr2=0x564996610fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa80e800640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa7f40b54d0 0x7fa7f40b7990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa808005fd0 tx=0x7fa808009860 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa81785a740 1 -- 172.21.15.90:0/526580741 shutdown_connections 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.320+0000 7fa81785a740 1 -- 172.21.15.90:0/526580741 wait complete. 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.321+0000 7fa81785a740 1 -- start start 2024-09-10T22:16:22.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.321+0000 7fa81785a740 1 Processor -- start 2024-09-10T22:16:22.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa81785a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x56499661b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa81785a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa81785a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x56499661c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa81785a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5649963d9bb0 con 0x564996611ff0 2024-09-10T22:16:22.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa81785a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5649965136c0 con 0x564996612910 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa81785a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564996621550 con 0x56499661e8a0 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa81785a740 10 client.0 _ll_register_callbacks cb 0x56499652e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa80de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x56499661b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa80de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x56499661b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33216/0 (socket says 172.21.15.90:33216) 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa80de00640 1 -- 172.21.15.90:0/1427124096 learned_addr learned my addr 172.21.15.90:0/1427124096 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa807e00640 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa80e800640 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x56499661c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa80de00640 1 -- 172.21.15.90:0/1427124096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 msgr2=0x56499661c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa80de00640 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x56499661c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa80de00640 1 -- 172.21.15.90:0/1427124096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 msgr2=0x56499661bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:16:22.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa80de00640 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.322+0000 7fa80de00640 1 -- 172.21.15.90:0/1427124096 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649962d6b00 con 0x564996611ff0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.323+0000 7fa80de00640 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x56499661b840 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa800004700 tx=0x7fa800013850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bffcbf8fd53cc89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.323+0000 7fa7f3400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.323+0000 7fa7f3400640 1 -- 172.21.15.90:0/1427124096 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800017050 con 0x564996611ff0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.323+0000 7fa81785a740 1 -- 172.21.15.90:0/1427124096 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa800013470 con 0x564996611ff0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.323+0000 7fa7f3400640 1 -- 172.21.15.90:0/1427124096 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9490+0+0 (secure 0 0 0) 0x7fa800027480 con 0x564996611ff0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.323+0000 7fa7f3400640 1 -- 172.21.15.90:0/1427124096 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa800030050 con 0x564996611ff0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.323+0000 7fa81785a740 10 client.16194 fetch_fsmap learned FSMap version 41 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.323+0000 7fa81785a740 1 -- 172.21.15.90:0/1427124096 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56499634d960 con 0x564996611ff0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.324+0000 7fa7f3400640 1 -- 172.21.15.90:0/1427124096 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa800013470 con 0x564996611ff0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.324+0000 7fa7f3400640 1 -- 172.21.15.90:0/1427124096 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8410+0+0 (secure 0 0 0) 0x7fa80001c650 con 0x564996611ff0 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.324+0000 7fa81785a740 10 client.16194 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T22:16:22.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.324+0000 7fa81785a740 10 client.16194 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:16:22.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.324+0000 7fa81785a740 20 client.16194 populate_metadata read hostname 'smithi090' 2024-09-10T22:16:22.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.324+0000 7fa81785a740 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56499662e340 0x564996630730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.324+0000 7fa81785a740 4 client.16194 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:16:22.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.324+0000 7fa81785a740 1 -- 172.21.15.90:0/1427124096 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56499658b1e0 con 0x56499662e340 2024-09-10T22:16:22.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.325+0000 7fa807e00640 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56499662e340 0x564996630730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.325+0000 7fa807e00640 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56499662e340 0x564996630730 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8729f3a609d0b62c server_cookie=9ce36350358f533e in_seq=0 out_seq=0 2024-09-10T22:16:22.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.325+0000 7fa7f3400640 10 client.16194 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:16:22.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.326+0000 7fa7f3400640 1 -- 172.21.15.90:0/1427124096 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56499658b1e0 con 0x56499662e340 2024-09-10T22:16:22.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.326+0000 7fa7f3400640 10 client.16194 handle_command_reply: tid=0 2024-09-10T22:16:22.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.326+0000 7fa81785a740 1 client.16194 shutdown 2024-09-10T22:16:22.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 -- 172.21.15.90:0/1427124096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 msgr2=0x56499661b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:22.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x56499661b840 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fa800004700 tx=0x7fa800013850 comp rx=0 tx=0).stop 2024-09-10T22:16:22.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 -- 172.21.15.90:0/1427124096 shutdown_connections 2024-09-10T22:16:22.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56499662e340 0x564996630730 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x56499661c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x56499661bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 --2- 172.21.15.90:0/1427124096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x56499661b840 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa7f3400640 0 client.16194 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:16:22.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 -- 172.21.15.90:0/1427124096 >> 172.21.15.90:0/1427124096 conn(0x56499660fe20 msgr2=0x5649966159c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:22.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 -- 172.21.15.90:0/1427124096 shutdown_connections 2024-09-10T22:16:22.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 1 -- 172.21.15.90:0/1427124096 wait complete. 2024-09-10T22:16:22.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.327+0000 7fa81785a740 20 client.16194 trim_cache size 0 max 16384 2024-09-10T22:16:22.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.345+0000 7fa81785a740 1 Processor -- start 2024-09-10T22:16:22.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.345+0000 7fa81785a740 1 -- start start 2024-09-10T22:16:22.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.345+0000 7fa81785a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996611ff0 0x5649966160f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.345+0000 7fa81785a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996612910 0x5649966114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.345+0000 7fa81785a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x56499660eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.345+0000 7fa81785a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b20b0 con 0x564996612910 2024-09-10T22:16:22.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.345+0000 7fa81785a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b3dd0 con 0x564996611ff0 2024-09-10T22:16:22.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.345+0000 7fa81785a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b2780 con 0x56499661e8a0 2024-09-10T22:16:22.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.345+0000 7fa807e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996612910 0x5649966114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa807e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996612910 0x5649966114e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33224/0 (socket says 172.21.15.90:33224) 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa807e00640 1 -- 172.21.15.90:0/1320208971 learned_addr learned my addr 172.21.15.90:0/1320208971 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa80de00640 1 --2- 172.21.15.90:0/1320208971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996611ff0 0x5649966160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa80e800640 1 --2- 172.21.15.90:0/1320208971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x56499660eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa807e00640 1 -- 172.21.15.90:0/1320208971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 msgr2=0x56499660eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa807e00640 1 --2- 172.21.15.90:0/1320208971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x56499660eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa807e00640 1 -- 172.21.15.90:0/1320208971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996611ff0 msgr2=0x5649966160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa807e00640 1 --2- 172.21.15.90:0/1320208971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996611ff0 0x5649966160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa807e00640 1 -- 172.21.15.90:0/1320208971 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649966108a0 con 0x564996612910 2024-09-10T22:16:22.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa807e00640 1 --2- 172.21.15.90:0/1320208971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996612910 0x5649966114e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80cca90 tx=0x7fa7f80b4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd61db3e5196cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:22.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa7fe600640 1 -- 172.21.15.90:0/1320208971 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8049070 con 0x564996612910 2024-09-10T22:16:22.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.346+0000 7fa7fe600640 1 -- 172.21.15.90:0/1320208971 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7f80cc8a0 con 0x564996612910 2024-09-10T22:16:22.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa7fe600640 1 -- 172.21.15.90:0/1320208971 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f80ccd90 con 0x564996612910 2024-09-10T22:16:22.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa81785a740 1 -- 172.21.15.90:0/1320208971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996612910 msgr2=0x5649966114e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:22.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa81785a740 1 --2- 172.21.15.90:0/1320208971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996612910 0x5649966114e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fa7f80cca90 tx=0x7fa7f80b4040 comp rx=0 tx=0).stop 2024-09-10T22:16:22.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa81785a740 1 -- 172.21.15.90:0/1320208971 shutdown_connections 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa81785a740 1 --2- 172.21.15.90:0/1320208971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x56499660eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa81785a740 1 --2- 172.21.15.90:0/1320208971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996612910 0x5649966114e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa81785a740 1 --2- 172.21.15.90:0/1320208971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996611ff0 0x5649966160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa81785a740 1 -- 172.21.15.90:0/1320208971 >> 172.21.15.90:0/1320208971 conn(0x56499660fe20 msgr2=0x56499660e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa81785a740 1 -- 172.21.15.90:0/1320208971 shutdown_connections 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.347+0000 7fa81785a740 1 -- 172.21.15.90:0/1320208971 wait complete. 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa81785a740 1 -- start start 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa81785a740 1 Processor -- start 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa81785a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x564996626bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa81785a740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x564996627140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa81785a740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x564996623920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa81785a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b3640 con 0x564996611ff0 2024-09-10T22:16:22.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa81785a740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b2eb0 con 0x564996612910 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa81785a740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa8101b00b0 con 0x56499661e8a0 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa81785a740 10 client.0 _ll_register_callbacks cb 0x7fa7f810a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa80de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x564996626bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.348+0000 7fa80de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x564996626bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33240/0 (socket says 172.21.15.90:33240) 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa80de00640 1 -- 172.21.15.90:0/1353800140 learned_addr learned my addr 172.21.15.90:0/1353800140 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa807e00640 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x564996627140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa80e800640 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x564996623920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa80de00640 1 -- 172.21.15.90:0/1353800140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 msgr2=0x564996623920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa80de00640 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x564996623920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa80de00640 1 -- 172.21.15.90:0/1353800140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 msgr2=0x564996627140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa80de00640 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x564996627140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa80de00640 1 -- 172.21.15.90:0/1353800140 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564996624150 con 0x564996611ff0 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa80de00640 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x564996626bc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa800013ec0 tx=0x7fa800006ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19057ff28c106eea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:16:22.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa7f3400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa7f3400640 1 -- 172.21.15.90:0/1353800140 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800017040 con 0x564996611ff0 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.349+0000 7fa7f3400640 1 -- 172.21.15.90:0/1353800140 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9490+0+0 (secure 0 0 0) 0x7fa80001c450 con 0x564996611ff0 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa81785a740 1 -- 172.21.15.90:0/1353800140 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f80cc8a0 con 0x564996611ff0 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa7f3400640 1 -- 172.21.15.90:0/1353800140 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa800013470 con 0x564996611ff0 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa81785a740 10 client.16200 fetch_fsmap learned FSMap version 41 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa81785a740 1 -- 172.21.15.90:0/1353800140 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56499669bc40 con 0x564996611ff0 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa7f3400640 1 -- 172.21.15.90:0/1353800140 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa7f80cc8a0 con 0x564996611ff0 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa7f3400640 1 -- 172.21.15.90:0/1353800140 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 41) ==== 8410+0+0 (secure 0 0 0) 0x7fa800027b10 con 0x564996611ff0 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa81785a740 10 client.16200 fetch_fsmap finished waiting for FSMap version 41 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa81785a740 10 client.16200 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa81785a740 20 client.16200 populate_metadata read hostname 'smithi090' 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa81785a740 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56499669c450 0x56499669e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa81785a740 4 client.16200 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "33c49c9b", "release": true, "await": true}] 2024-09-10T22:16:22.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.350+0000 7fa81785a740 1 -- 172.21.15.90:0/1353800140 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "33c49c9b", "release": true, "await": true}) -- 0x564996586c90 con 0x56499669c450 2024-09-10T22:16:22.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.351+0000 7fa807e00640 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56499669c450 0x56499669e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:16:22.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.351+0000 7fa807e00640 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56499669c450 0x56499669e870 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca95f5514bebeae server_cookie=7b134da3f45123e5 in_seq=0 out_seq=0 2024-09-10T22:16:22.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.351+0000 7fa7f3400640 10 client.16200 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:16:22.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa7f3400640 1 -- 172.21.15.90:0/1353800140 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564996586c90 con 0x56499669c450 2024-09-10T22:16:22.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa7f3400640 10 client.16200 handle_command_reply: tid=0 2024-09-10T22:16:22.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 client.16200 shutdown 2024-09-10T22:16:22.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 -- 172.21.15.90:0/1353800140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 msgr2=0x564996626bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:22.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x564996626bc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa800013ec0 tx=0x7fa800006ed0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 -- 172.21.15.90:0/1353800140 shutdown_connections 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56499669c450 0x56499669e870 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56499661e8a0 0x564996623920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564996612910 0x564996627140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 --2- 172.21.15.90:0/1353800140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564996611ff0 0x564996626bc0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 -- 172.21.15.90:0/1353800140 >> 172.21.15.90:0/1353800140 conn(0x56499660fe20 msgr2=0x56499660edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa7f3400640 0 client.16200 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 -- 172.21.15.90:0/1353800140 shutdown_connections 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.360+0000 7fa81785a740 1 -- 172.21.15.90:0/1353800140 wait complete. 2024-09-10T22:16:22.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.361+0000 7fa81785a740 20 client.16200 trim_cache size 0 max 16384 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 41, 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 40, 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "33c49c9b": { 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 40, 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:16:22.364 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 145.4, 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:16:22.365 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:16:22.366 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:16:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.361+0000 7fa814c00640 1 -- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa7f40b54d0 msgr2=0x7fa7f40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.361+0000 7fa814c00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa7f40b54d0 0x7fa7f40b7990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa808005fd0 tx=0x7fa808009860 comp rx=0 tx=0).stop 2024-09-10T22:16:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.361+0000 7fa814c00640 1 -- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 msgr2=0x7fa81019ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:16:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.361+0000 7fa814c00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 0x7fa81019ba00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fa7f802a930 tx=0x7fa7f8003b90 comp rx=0 tx=0).stop 2024-09-10T22:16:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.362+0000 7fa814c00640 1 -- 172.21.15.90:0/3117995546 shutdown_connections 2024-09-10T22:16:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.362+0000 7fa814c00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa7f40b54d0 0x7fa7f40b7990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.362+0000 7fa814c00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa81010d120 0x7fa8101a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.362+0000 7fa814c00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8101045e0 0x7fa81019ba00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.362+0000 7fa814c00640 1 --2- 172.21.15.90:0/3117995546 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa810103cc0 0x7fa81019b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:16:22.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.362+0000 7fa814c00640 1 -- 172.21.15.90:0/3117995546 >> 172.21.15.90:0/3117995546 conn(0x7fa8100fd700 msgr2=0x7fa8100ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:16:22.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.362+0000 7fa814c00640 1 -- 172.21.15.90:0/3117995546 shutdown_connections 2024-09-10T22:16:22.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:16:22.362+0000 7fa814c00640 1 -- 172.21.15.90:0/3117995546 wait complete. 2024-09-10T22:16:22.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:21 smithi154 ceph-mon[34495]: pgmap v806: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 5.7 KiB/s wr, 9 op/s 2024-09-10T22:16:22.507 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138130 1.06 MB/sec execute 554 sec latency 25135.115 ms 2024-09-10T22:16:23.265 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '33c49c9b', total seconds elapsed: 29.6 2024-09-10T22:16:23.508 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138437 1.06 MB/sec execute 555 sec latency 23.341 ms 2024-09-10T22:16:24.508 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138437 1.06 MB/sec execute 556 sec latency 1023.430 ms 2024-09-10T22:16:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:24 smithi154 ceph-mon[34495]: pgmap v807: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-09-10T22:16:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:24 smithi171 ceph-mon[34138]: pgmap v807: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-09-10T22:16:25.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:24 smithi090 ceph-mon[30350]: pgmap v807: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 13 KiB/s wr, 9 op/s 2024-09-10T22:16:25.508 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138437 1.06 MB/sec execute 557 sec latency 2023.499 ms 2024-09-10T22:16:26.508 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138437 1.06 MB/sec execute 558 sec latency 3023.624 ms 2024-09-10T22:16:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:26 smithi154 ceph-mon[34495]: pgmap v808: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-10T22:16:27.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:26 smithi171 ceph-mon[34138]: pgmap v808: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-10T22:16:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:26 smithi090 ceph-mon[30350]: pgmap v808: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 54 KiB/s wr, 18 op/s 2024-09-10T22:16:27.508 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138437 1.06 MB/sec execute 559 sec latency 4023.764 ms 2024-09-10T22:16:28.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:27 smithi171 ceph-mon[34138]: pgmap v809: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-09-10T22:16:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:27 smithi090 ceph-mon[30350]: pgmap v809: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-09-10T22:16:28.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:27 smithi154 ceph-mon[34495]: pgmap v809: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 55 KiB/s wr, 14 op/s 2024-09-10T22:16:28.508 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138504 1.05 MB/sec execute 560 sec latency 4146.009 ms 2024-09-10T22:16:29.508 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138504 1.05 MB/sec execute 561 sec latency 1550.352 ms 2024-09-10T22:16:30.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138504 1.05 MB/sec execute 562 sec latency 2550.522 ms 2024-09-10T22:16:31.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:30 smithi154 ceph-mon[34495]: pgmap v810: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 88 KiB/s wr, 24 op/s 2024-09-10T22:16:31.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:30 smithi171 ceph-mon[34138]: pgmap v810: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 88 KiB/s wr, 24 op/s 2024-09-10T22:16:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:30 smithi090 ceph-mon[30350]: pgmap v810: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 88 KiB/s wr, 24 op/s 2024-09-10T22:16:31.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138504 1.05 MB/sec execute 563 sec latency 3550.631 ms 2024-09-10T22:16:32.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138504 1.05 MB/sec execute 564 sec latency 4550.759 ms 2024-09-10T22:16:33.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:32 smithi154 ceph-mon[34495]: pgmap v811: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 83 KiB/s wr, 21 op/s 2024-09-10T22:16:33.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:32 smithi171 ceph-mon[34138]: pgmap v811: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 83 KiB/s wr, 21 op/s 2024-09-10T22:16:33.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:32 smithi090 ceph-mon[30350]: pgmap v811: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 83 KiB/s wr, 21 op/s 2024-09-10T22:16:33.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138679 1.05 MB/sec execute 565 sec latency 4671.651 ms 2024-09-10T22:16:34.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:33 smithi171 ceph-mon[34138]: pgmap v812: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 112 KiB/s wr, 23 op/s 2024-09-10T22:16:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:33 smithi090 ceph-mon[30350]: pgmap v812: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 112 KiB/s wr, 23 op/s 2024-09-10T22:16:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:33 smithi154 ceph-mon[34495]: pgmap v812: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 112 KiB/s wr, 23 op/s 2024-09-10T22:16:34.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138850 1.04 MB/sec execute 566 sec latency 19.948 ms 2024-09-10T22:16:35.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 139119 1.04 MB/sec execute 567 sec latency 21.633 ms 2024-09-10T22:16:36.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 139464 1.04 MB/sec execute 568 sec latency 21.987 ms 2024-09-10T22:16:37.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:36 smithi154 ceph-mon[34495]: pgmap v813: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 320 KiB/s wr, 43 op/s 2024-09-10T22:16:37.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:36 smithi171 ceph-mon[34138]: pgmap v813: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 320 KiB/s wr, 43 op/s 2024-09-10T22:16:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:36 smithi090 ceph-mon[30350]: pgmap v813: 129 pgs: 129 active+clean; 183 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 320 KiB/s wr, 43 op/s 2024-09-10T22:16:37.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 139839 1.05 MB/sec execute 569 sec latency 25.376 ms 2024-09-10T22:16:38.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 140181 1.05 MB/sec execute 570 sec latency 31.497 ms 2024-09-10T22:16:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:38 smithi154 ceph-mon[34495]: pgmap v814: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 371 KiB/s wr, 40 op/s 2024-09-10T22:16:39.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:38 smithi090 ceph-mon[30350]: pgmap v814: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 371 KiB/s wr, 40 op/s 2024-09-10T22:16:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:38 smithi171 ceph-mon[34138]: pgmap v814: 129 pgs: 129 active+clean; 184 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 371 KiB/s wr, 40 op/s 2024-09-10T22:16:39.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 140551 1.05 MB/sec execute 571 sec latency 27.548 ms 2024-09-10T22:16:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:39 smithi171 ceph-mon[34138]: pgmap v815: 129 pgs: 129 active+clean; 185 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 617 KiB/s wr, 71 op/s 2024-09-10T22:16:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:39 smithi090 ceph-mon[30350]: pgmap v815: 129 pgs: 129 active+clean; 185 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 617 KiB/s wr, 71 op/s 2024-09-10T22:16:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:39 smithi154 ceph-mon[34495]: pgmap v815: 129 pgs: 129 active+clean; 185 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 617 KiB/s wr, 71 op/s 2024-09-10T22:16:40.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 140861 1.05 MB/sec execute 572 sec latency 18.053 ms 2024-09-10T22:16:41.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 141344 1.05 MB/sec execute 573 sec latency 21.953 ms 2024-09-10T22:16:42.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:41 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:16:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:41 smithi171 ceph-mon[34138]: pgmap v816: 129 pgs: 129 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 668 KiB/s wr, 67 op/s 2024-09-10T22:16:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:41 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:16:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:41 smithi090 ceph-mon[30350]: pgmap v816: 129 pgs: 129 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 668 KiB/s wr, 67 op/s 2024-09-10T22:16:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:41 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:16:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:41 smithi154 ceph-mon[34495]: pgmap v816: 129 pgs: 129 active+clean; 186 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 668 KiB/s wr, 67 op/s 2024-09-10T22:16:42.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 141692 1.06 MB/sec execute 574 sec latency 20.758 ms 2024-09-10T22:16:43.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 141983 1.06 MB/sec execute 575 sec latency 234.037 ms 2024-09-10T22:16:44.509 INFO:tasks.workunit.client.0.smithi090.stdout: 1 141983 1.06 MB/sec execute 576 sec latency 1234.172 ms 2024-09-10T22:16:45.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:44 smithi154 ceph-mon[34495]: pgmap v817: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 914 KiB/s wr, 77 op/s 2024-09-10T22:16:45.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:44 smithi171 ceph-mon[34138]: pgmap v817: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 914 KiB/s wr, 77 op/s 2024-09-10T22:16:45.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:44 smithi090 ceph-mon[30350]: pgmap v817: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 914 KiB/s wr, 77 op/s 2024-09-10T22:16:45.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 141983 1.06 MB/sec execute 577 sec latency 2234.301 ms 2024-09-10T22:16:46.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 141983 1.05 MB/sec execute 578 sec latency 3234.357 ms 2024-09-10T22:16:47.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:46 smithi154 ceph-mon[34495]: pgmap v818: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-09-10T22:16:47.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:46 smithi171 ceph-mon[34138]: pgmap v818: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-09-10T22:16:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:46 smithi090 ceph-mon[30350]: pgmap v818: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.1 MiB/s wr, 98 op/s 2024-09-10T22:16:47.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 141983 1.05 MB/sec execute 579 sec latency 4234.514 ms 2024-09-10T22:16:48.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:47 smithi171 ceph-mon[34138]: pgmap v819: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 963 KiB/s wr, 79 op/s 2024-09-10T22:16:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:47 smithi090 ceph-mon[30350]: pgmap v819: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 963 KiB/s wr, 79 op/s 2024-09-10T22:16:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:47 smithi154 ceph-mon[34495]: pgmap v819: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 963 KiB/s wr, 79 op/s 2024-09-10T22:16:48.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 142050 1.05 MB/sec execute 580 sec latency 4354.398 ms 2024-09-10T22:16:49.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 142050 1.05 MB/sec execute 581 sec latency 1543.735 ms 2024-09-10T22:16:50.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 142050 1.05 MB/sec execute 582 sec latency 2543.856 ms 2024-09-10T22:16:51.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:50 smithi154 ceph-mon[34495]: pgmap v820: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 879 KiB/s wr, 80 op/s 2024-09-10T22:16:51.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:50 smithi171 ceph-mon[34138]: pgmap v820: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 879 KiB/s wr, 80 op/s 2024-09-10T22:16:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:50 smithi090 ceph-mon[30350]: pgmap v820: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 879 KiB/s wr, 80 op/s 2024-09-10T22:16:51.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 142050 1.04 MB/sec execute 583 sec latency 3543.981 ms 2024-09-10T22:16:52.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:51 smithi171 ceph-mon[34138]: pgmap v821: 129 pgs: 129 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 666 KiB/s wr, 53 op/s 2024-09-10T22:16:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:51 smithi090 ceph-mon[30350]: pgmap v821: 129 pgs: 129 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 666 KiB/s wr, 53 op/s 2024-09-10T22:16:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:51 smithi154 ceph-mon[34495]: pgmap v821: 129 pgs: 129 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 666 KiB/s wr, 53 op/s 2024-09-10T22:16:52.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 142050 1.04 MB/sec execute 584 sec latency 4544.066 ms 2024-09-10T22:16:53.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 142222 1.04 MB/sec execute 585 sec latency 4663.368 ms 2024-09-10T22:16:54.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 142386 1.04 MB/sec execute 586 sec latency 20.265 ms 2024-09-10T22:16:55.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:54 smithi154 ceph-mon[34495]: pgmap v822: 129 pgs: 129 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 592 KiB/s wr, 50 op/s 2024-09-10T22:16:55.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:54 smithi171 ceph-mon[34138]: pgmap v822: 129 pgs: 129 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 592 KiB/s wr, 50 op/s 2024-09-10T22:16:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:54 smithi090 ceph-mon[30350]: pgmap v822: 129 pgs: 129 active+clean; 191 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 592 KiB/s wr, 50 op/s 2024-09-10T22:16:55.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 142633 1.04 MB/sec execute 587 sec latency 23.998 ms 2024-09-10T22:16:56.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:55 smithi171 ceph-mon[34138]: pgmap v823: 129 pgs: 129 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 557 KiB/s wr, 59 op/s 2024-09-10T22:16:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:55 smithi090 ceph-mon[30350]: pgmap v823: 129 pgs: 129 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 557 KiB/s wr, 59 op/s 2024-09-10T22:16:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:55 smithi154 ceph-mon[34495]: pgmap v823: 129 pgs: 129 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 557 KiB/s wr, 59 op/s 2024-09-10T22:16:56.510 INFO:tasks.workunit.client.0.smithi090.stdout: 1 142982 1.04 MB/sec execute 588 sec latency 21.462 ms 2024-09-10T22:16:57.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 143364 1.05 MB/sec execute 589 sec latency 25.171 ms 2024-09-10T22:16:58.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 143698 1.04 MB/sec execute 590 sec latency 27.966 ms 2024-09-10T22:16:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:58 smithi154 ceph-mon[34495]: pgmap v824: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 412 KiB/s wr, 46 op/s 2024-09-10T22:16:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:58 smithi171 ceph-mon[34138]: pgmap v824: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 412 KiB/s wr, 46 op/s 2024-09-10T22:16:59.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:58 smithi090 ceph-mon[30350]: pgmap v824: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 412 KiB/s wr, 46 op/s 2024-09-10T22:16:59.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 144092 1.04 MB/sec execute 591 sec latency 24.595 ms 2024-09-10T22:17:00.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:16:59 smithi171 ceph-mon[34138]: pgmap v825: 129 pgs: 129 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 659 KiB/s wr, 72 op/s 2024-09-10T22:17:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:16:59 smithi090 ceph-mon[30350]: pgmap v825: 129 pgs: 129 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 659 KiB/s wr, 72 op/s 2024-09-10T22:17:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:16:59 smithi154 ceph-mon[34495]: pgmap v825: 129 pgs: 129 active+clean; 193 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 659 KiB/s wr, 72 op/s 2024-09-10T22:17:00.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 144408 1.04 MB/sec execute 592 sec latency 17.719 ms 2024-09-10T22:17:01.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 144850 1.05 MB/sec execute 593 sec latency 153.277 ms 2024-09-10T22:17:02.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 145229 1.05 MB/sec execute 594 sec latency 19.494 ms 2024-09-10T22:17:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:02 smithi154 ceph-mon[34495]: pgmap v826: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 652 KiB/s wr, 66 op/s 2024-09-10T22:17:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:02 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:17:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:02 smithi171 ceph-mon[34138]: pgmap v826: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 652 KiB/s wr, 66 op/s 2024-09-10T22:17:03.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:02 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:17:03.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:02 smithi090 ceph-mon[30350]: pgmap v826: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 652 KiB/s wr, 66 op/s 2024-09-10T22:17:03.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:02 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:17:03.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 145526 1.06 MB/sec execute 595 sec latency 168.680 ms 2024-09-10T22:17:04.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:03 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:04.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:03 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:17:04.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:03 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:17:04.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:03 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:17:04.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:03 smithi171 ceph-mon[34138]: pgmap v827: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 878 KiB/s wr, 90 op/s 2024-09-10T22:17:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:03 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:04.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:03 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:17:04.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:03 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:17:04.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:03 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:17:04.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:03 smithi090 ceph-mon[30350]: pgmap v827: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 878 KiB/s wr, 90 op/s 2024-09-10T22:17:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:03 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:03 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:17:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:03 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:17:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:03 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:17:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:03 smithi154 ceph-mon[34495]: pgmap v827: 129 pgs: 129 active+clean; 195 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 878 KiB/s wr, 90 op/s 2024-09-10T22:17:04.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 145526 1.05 MB/sec execute 596 sec latency 1168.779 ms 2024-09-10T22:17:05.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 145526 1.05 MB/sec execute 597 sec latency 2168.919 ms 2024-09-10T22:17:05.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:05 smithi090 ceph-mon[30350]: osdmap e105: 12 total, 12 up, 12 in 2024-09-10T22:17:05.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:05 smithi154 ceph-mon[34495]: osdmap e105: 12 total, 12 up, 12 in 2024-09-10T22:17:05.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:05 smithi171 ceph-mon[34138]: osdmap e105: 12 total, 12 up, 12 in 2024-09-10T22:17:06.511 INFO:tasks.workunit.client.0.smithi090.stdout: 1 145526 1.05 MB/sec execute 598 sec latency 3169.061 ms 2024-09-10T22:17:06.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:06 smithi090 ceph-mon[30350]: pgmap v829: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-09-10T22:17:06.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:06 smithi154 ceph-mon[34495]: pgmap v829: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-09-10T22:17:06.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:06 smithi171 ceph-mon[34138]: pgmap v829: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-09-10T22:17:07.512 INFO:tasks.workunit.client.0.smithi090.stdout: 1 145526 1.05 MB/sec execute 599 sec latency 4169.145 ms 2024-09-10T22:17:08.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:07 smithi171 ceph-mon[34138]: pgmap v830: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-09-10T22:17:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:07 smithi090 ceph-mon[30350]: pgmap v830: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-09-10T22:17:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:07 smithi154 ceph-mon[34495]: pgmap v830: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 197 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-09-10T22:17:08.512 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 600 sec 2024-09-10T22:17:09.512 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 601 sec 2024-09-10T22:17:09.747 INFO:tasks.workunit.client.0.smithi090.stdout: 0 cleanup 601 sec 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: Operation Count AvgLat MaxLat 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: ---------------------------------------- 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: NTCreateX 20052 7.172 25016.910 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: Close 14760 0.072 1.267 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: Rename 854 108.942 33074.313 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: Unlink 4027 11.444 20754.019 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: Qpathinfo 18237 4.591 1920.783 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: Qfileinfo 3191 0.157 0.665 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: Qfsinfo 3341 0.669 4.392 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: Sfileinfo 1632 18.566 26922.085 2024-09-10T22:17:09.748 INFO:tasks.workunit.client.0.smithi090.stdout: Find 7049 15.095 547.113 2024-09-10T22:17:09.749 INFO:tasks.workunit.client.0.smithi090.stdout: WriteX 9943 8.752 30300.101 2024-09-10T22:17:09.749 INFO:tasks.workunit.client.0.smithi090.stdout: ReadX 31663 0.006 3.582 2024-09-10T22:17:09.749 INFO:tasks.workunit.client.0.smithi090.stdout: LockX 66 0.936 1.432 2024-09-10T22:17:09.749 INFO:tasks.workunit.client.0.smithi090.stdout: UnlockX 66 0.835 1.333 2024-09-10T22:17:09.749 INFO:tasks.workunit.client.0.smithi090.stdout: Flush 1395 3.586 12.617 2024-09-10T22:17:09.749 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-09-10T22:17:09.749 INFO:tasks.workunit.client.0.smithi090.stdout:Throughput 1.04872 MB/sec 1 clients 1 procs max_latency=33074.323 ms 2024-09-10T22:17:09.749 INFO:tasks.workunit.client.0.smithi090.stderr:+ dbench 10 2024-09-10T22:17:09.750 INFO:tasks.workunit.client.0.smithi090.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T22:17:09.750 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-09-10T22:17:09.750 INFO:tasks.workunit.client.0.smithi090.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T22:17:09.751 INFO:tasks.workunit.client.0.smithi090.stdout:9 of 10 processes prepared for launch 0 sec 2024-09-10T22:17:09.852 INFO:tasks.workunit.client.0.smithi090.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-10T22:17:09.852 INFO:tasks.workunit.client.0.smithi090.stdout:releasing clients 2024-09-10T22:17:10.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:09 smithi171 ceph-mon[34138]: pgmap v831: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-09-10T22:17:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:09 smithi090 ceph-mon[30350]: pgmap v831: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-09-10T22:17:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:09 smithi154 ceph-mon[34495]: pgmap v831: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-09-10T22:17:10.852 INFO:tasks.workunit.client.0.smithi090.stdout: 10 204 80.34 MB/sec warmup 1 sec latency 99.457 ms 2024-09-10T22:17:11.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 359 64.33 MB/sec warmup 2 sec latency 93.727 ms 2024-09-10T22:17:11.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:11 smithi171 ceph-mon[34138]: osdmap e106: 12 total, 12 up, 12 in 2024-09-10T22:17:12.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:11 smithi090 ceph-mon[30350]: osdmap e106: 12 total, 12 up, 12 in 2024-09-10T22:17:12.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:11 smithi154 ceph-mon[34495]: osdmap e106: 12 total, 12 up, 12 in 2024-09-10T22:17:12.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 520 60.24 MB/sec warmup 3 sec latency 94.177 ms 2024-09-10T22:17:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:12 smithi171 ceph-mon[34138]: pgmap v833: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-09-10T22:17:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:12 smithi090 ceph-mon[30350]: pgmap v833: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-09-10T22:17:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:12 smithi154 ceph-mon[34495]: pgmap v833: 129 pgs: 129 active+clean; 202 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.3 MiB/s wr, 80 op/s 2024-09-10T22:17:13.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 682 52.92 MB/sec warmup 4 sec latency 183.101 ms 2024-09-10T22:17:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:13 smithi171 ceph-mon[34138]: pgmap v834: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 13 MiB/s wr, 122 op/s 2024-09-10T22:17:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:13 smithi090 ceph-mon[30350]: pgmap v834: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 13 MiB/s wr, 122 op/s 2024-09-10T22:17:14.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:13 smithi154 ceph-mon[34495]: pgmap v834: 129 pgs: 129 active+clean; 314 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 13 MiB/s wr, 122 op/s 2024-09-10T22:17:14.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 834 43.04 MB/sec warmup 5 sec latency 86.234 ms 2024-09-10T22:17:15.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 967 36.55 MB/sec warmup 6 sec latency 58.266 ms 2024-09-10T22:17:16.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 1110 32.80 MB/sec warmup 7 sec latency 82.601 ms 2024-09-10T22:17:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:16 smithi171 ceph-mon[34138]: pgmap v835: 129 pgs: 129 active+clean; 391 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 20 MiB/s wr, 152 op/s 2024-09-10T22:17:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:16 smithi090 ceph-mon[30350]: pgmap v835: 129 pgs: 129 active+clean; 391 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 20 MiB/s wr, 152 op/s 2024-09-10T22:17:17.119 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:16 smithi154 ceph-mon[34495]: pgmap v835: 129 pgs: 129 active+clean; 391 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 20 MiB/s wr, 152 op/s 2024-09-10T22:17:17.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 1204 28.81 MB/sec warmup 8 sec latency 546.011 ms 2024-09-10T22:17:18.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 1367 28.56 MB/sec warmup 9 sec latency 1225.639 ms 2024-09-10T22:17:18.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:18 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:18.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:18 smithi171 ceph-mon[34138]: pgmap v836: 129 pgs: 129 active+clean; 398 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 21 MiB/s wr, 172 op/s 2024-09-10T22:17:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:18 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:18 smithi090 ceph-mon[30350]: pgmap v836: 129 pgs: 129 active+clean; 398 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 21 MiB/s wr, 172 op/s 2024-09-10T22:17:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:18 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:18 smithi154 ceph-mon[34495]: pgmap v836: 129 pgs: 129 active+clean; 398 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 21 MiB/s wr, 172 op/s 2024-09-10T22:17:19.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 1491 25.78 MB/sec warmup 10 sec latency 71.723 ms 2024-09-10T22:17:19.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:19 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:19 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:19 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:20.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 1622 23.79 MB/sec warmup 11 sec latency 68.894 ms 2024-09-10T22:17:20.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:20 smithi171 ceph-mon[34138]: pgmap v837: 129 pgs: 129 active+clean; 410 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 23 MiB/s wr, 229 op/s 2024-09-10T22:17:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:20 smithi090 ceph-mon[30350]: pgmap v837: 129 pgs: 129 active+clean; 410 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 23 MiB/s wr, 229 op/s 2024-09-10T22:17:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:20 smithi154 ceph-mon[34495]: pgmap v837: 129 pgs: 129 active+clean; 410 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 23 MiB/s wr, 229 op/s 2024-09-10T22:17:21.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 1768 21.91 MB/sec warmup 12 sec latency 131.000 ms 2024-09-10T22:17:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:21 smithi171 ceph-mon[34138]: pgmap v838: 129 pgs: 129 active+clean; 411 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 20 MiB/s wr, 221 op/s 2024-09-10T22:17:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:21 smithi090 ceph-mon[30350]: pgmap v838: 129 pgs: 129 active+clean; 411 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 20 MiB/s wr, 221 op/s 2024-09-10T22:17:22.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:21 smithi154 ceph-mon[34495]: pgmap v838: 129 pgs: 129 active+clean; 411 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 20 MiB/s wr, 221 op/s 2024-09-10T22:17:22.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 1892 20.31 MB/sec warmup 13 sec latency 130.528 ms 2024-09-10T22:17:23.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2040 18.99 MB/sec warmup 14 sec latency 70.985 ms 2024-09-10T22:17:24.853 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2197 18.02 MB/sec warmup 15 sec latency 70.432 ms 2024-09-10T22:17:24.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:24 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:24.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:24 smithi090 ceph-mon[30350]: pgmap v839: 129 pgs: 129 active+clean; 413 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 19 MiB/s wr, 250 op/s 2024-09-10T22:17:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:24 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:24.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:24 smithi154 ceph-mon[34495]: pgmap v839: 129 pgs: 129 active+clean; 413 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 19 MiB/s wr, 250 op/s 2024-09-10T22:17:24.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:24 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:24.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:24 smithi171 ceph-mon[34138]: pgmap v839: 129 pgs: 129 active+clean; 413 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 19 MiB/s wr, 250 op/s 2024-09-10T22:17:25.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2336 17.37 MB/sec warmup 16 sec latency 60.647 ms 2024-09-10T22:17:26.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2463 16.71 MB/sec warmup 17 sec latency 58.544 ms 2024-09-10T22:17:26.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:26 smithi171 ceph-mon[34138]: pgmap v840: 129 pgs: 129 active+clean; 415 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.8 MiB/s wr, 251 op/s 2024-09-10T22:17:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:26 smithi090 ceph-mon[30350]: pgmap v840: 129 pgs: 129 active+clean; 415 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.8 MiB/s wr, 251 op/s 2024-09-10T22:17:27.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:26 smithi154 ceph-mon[34495]: pgmap v840: 129 pgs: 129 active+clean; 415 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.8 MiB/s wr, 251 op/s 2024-09-10T22:17:27.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2655 16.59 MB/sec warmup 18 sec latency 64.529 ms 2024-09-10T22:17:28.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:27 smithi171 ceph-mon[34138]: pgmap v841: 129 pgs: 129 active+clean; 418 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-09-10T22:17:28.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:27 smithi090 ceph-mon[30350]: pgmap v841: 129 pgs: 129 active+clean; 418 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-09-10T22:17:28.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:27 smithi154 ceph-mon[34495]: pgmap v841: 129 pgs: 129 active+clean; 418 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 3.8 MiB/s wr, 212 op/s 2024-09-10T22:17:28.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2894 17.17 MB/sec warmup 19 sec latency 72.592 ms 2024-09-10T22:17:29.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3143 17.03 MB/sec warmup 20 sec latency 57.652 ms 2024-09-10T22:17:30.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:29 smithi171 ceph-mon[34138]: pgmap v842: 129 pgs: 129 active+clean; 431 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.7 MiB/s wr, 245 op/s 2024-09-10T22:17:30.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:29 smithi090 ceph-mon[30350]: pgmap v842: 129 pgs: 129 active+clean; 431 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.7 MiB/s wr, 245 op/s 2024-09-10T22:17:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:29 smithi154 ceph-mon[34495]: pgmap v842: 129 pgs: 129 active+clean; 431 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 4.7 MiB/s wr, 245 op/s 2024-09-10T22:17:30.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3241 16.54 MB/sec warmup 21 sec latency 67.155 ms 2024-09-10T22:17:31.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3350 16.37 MB/sec warmup 22 sec latency 61.602 ms 2024-09-10T22:17:32.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:31 smithi171 ceph-mon[34138]: pgmap v843: 129 pgs: 129 active+clean; 435 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-09-10T22:17:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:31 smithi090 ceph-mon[30350]: pgmap v843: 129 pgs: 129 active+clean; 435 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-09-10T22:17:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:31 smithi154 ceph-mon[34495]: pgmap v843: 129 pgs: 129 active+clean; 435 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-09-10T22:17:32.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3508 16.28 MB/sec warmup 23 sec latency 63.250 ms 2024-09-10T22:17:33.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3685 16.16 MB/sec warmup 24 sec latency 68.275 ms 2024-09-10T22:17:34.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:33 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:34.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:33 smithi171 ceph-mon[34138]: pgmap v844: 129 pgs: 129 active+clean; 442 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 4.3 MiB/s wr, 213 op/s 2024-09-10T22:17:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:33 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:33 smithi090 ceph-mon[30350]: pgmap v844: 129 pgs: 129 active+clean; 442 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 4.3 MiB/s wr, 213 op/s 2024-09-10T22:17:34.421 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:33 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:34.421 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:33 smithi154 ceph-mon[34495]: pgmap v844: 129 pgs: 129 active+clean; 442 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 4.3 MiB/s wr, 213 op/s 2024-09-10T22:17:34.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3797 15.76 MB/sec warmup 25 sec latency 61.415 ms 2024-09-10T22:17:35.861 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3885 15.21 MB/sec warmup 26 sec latency 56.927 ms 2024-09-10T22:17:36.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:35 smithi171 ceph-mon[34138]: pgmap v845: 129 pgs: 129 active+clean; 447 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 221 op/s 2024-09-10T22:17:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:35 smithi090 ceph-mon[30350]: pgmap v845: 129 pgs: 129 active+clean; 447 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 221 op/s 2024-09-10T22:17:36.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:35 smithi154 ceph-mon[34495]: pgmap v845: 129 pgs: 129 active+clean; 447 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 221 op/s 2024-09-10T22:17:36.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3989 14.78 MB/sec warmup 27 sec latency 72.810 ms 2024-09-10T22:17:37.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4068 14.27 MB/sec warmup 28 sec latency 270.463 ms 2024-09-10T22:17:38.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4160 13.82 MB/sec warmup 29 sec latency 313.664 ms 2024-09-10T22:17:39.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:38 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:39.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:38 smithi090 ceph-mon[30350]: pgmap v846: 129 pgs: 129 active+clean; 449 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.2 MiB/s wr, 190 op/s 2024-09-10T22:17:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:38 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:38 smithi154 ceph-mon[34495]: pgmap v846: 129 pgs: 129 active+clean; 449 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.2 MiB/s wr, 190 op/s 2024-09-10T22:17:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:38 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:17:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:38 smithi171 ceph-mon[34138]: pgmap v846: 129 pgs: 129 active+clean; 449 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.2 MiB/s wr, 190 op/s 2024-09-10T22:17:39.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4301 13.46 MB/sec warmup 30 sec latency 565.555 ms 2024-09-10T22:17:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:39 smithi171 ceph-mon[34138]: pgmap v847: 129 pgs: 129 active+clean; 453 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 236 op/s 2024-09-10T22:17:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:39 smithi090 ceph-mon[30350]: pgmap v847: 129 pgs: 129 active+clean; 453 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 236 op/s 2024-09-10T22:17:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:39 smithi154 ceph-mon[34495]: pgmap v847: 129 pgs: 129 active+clean; 453 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 236 op/s 2024-09-10T22:17:40.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4434 13.24 MB/sec warmup 31 sec latency 83.652 ms 2024-09-10T22:17:41.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4560 12.98 MB/sec warmup 32 sec latency 85.161 ms 2024-09-10T22:17:42.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:41 smithi171 ceph-mon[34138]: pgmap v848: 129 pgs: 129 active+clean; 451 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 187 op/s 2024-09-10T22:17:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:41 smithi090 ceph-mon[30350]: pgmap v848: 129 pgs: 129 active+clean; 451 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 187 op/s 2024-09-10T22:17:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:41 smithi154 ceph-mon[34495]: pgmap v848: 129 pgs: 129 active+clean; 451 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 187 op/s 2024-09-10T22:17:42.854 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4680 12.71 MB/sec warmup 33 sec latency 122.618 ms 2024-09-10T22:17:43.855 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4775 12.61 MB/sec warmup 34 sec latency 429.674 ms 2024-09-10T22:17:44.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:43 smithi171 ceph-mon[34138]: pgmap v849: 129 pgs: 129 active+clean; 441 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-09-10T22:17:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:43 smithi090 ceph-mon[30350]: pgmap v849: 129 pgs: 129 active+clean; 441 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-09-10T22:17:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:43 smithi154 ceph-mon[34495]: pgmap v849: 129 pgs: 129 active+clean; 441 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 223 op/s 2024-09-10T22:17:44.855 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4940 12.76 MB/sec warmup 35 sec latency 80.793 ms 2024-09-10T22:17:45.855 INFO:tasks.workunit.client.0.smithi090.stdout: 10 5067 12.43 MB/sec warmup 36 sec latency 73.970 ms 2024-09-10T22:17:46.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:45 smithi171 ceph-mon[34138]: pgmap v850: 129 pgs: 129 active+clean; 437 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 262 op/s 2024-09-10T22:17:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:45 smithi090 ceph-mon[30350]: pgmap v850: 129 pgs: 129 active+clean; 437 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 262 op/s 2024-09-10T22:17:46.671 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:45 smithi154 ceph-mon[34495]: pgmap v850: 129 pgs: 129 active+clean; 437 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 262 op/s 2024-09-10T22:17:46.855 INFO:tasks.workunit.client.0.smithi090.stdout: 10 5209 12.13 MB/sec warmup 37 sec latency 62.311 ms 2024-09-10T22:17:47.855 INFO:tasks.workunit.client.0.smithi090.stdout: 10 5356 11.85 MB/sec warmup 38 sec latency 88.752 ms 2024-09-10T22:17:48.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:47 smithi171 ceph-mon[34138]: pgmap v851: 129 pgs: 129 active+clean; 437 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-09-10T22:17:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:47 smithi090 ceph-mon[30350]: pgmap v851: 129 pgs: 129 active+clean; 437 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-09-10T22:17:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:47 smithi154 ceph-mon[34495]: pgmap v851: 129 pgs: 129 active+clean; 437 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-09-10T22:17:48.855 INFO:tasks.workunit.client.0.smithi090.stdout: 10 5441 11.58 MB/sec warmup 39 sec latency 436.889 ms 2024-09-10T22:17:49.855 INFO:tasks.workunit.client.0.smithi090.stdout: 10 5597 11.37 MB/sec warmup 40 sec latency 60.884 ms 2024-09-10T22:17:50.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:49 smithi171 ceph-mon[34138]: pgmap v852: 129 pgs: 129 active+clean; 439 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-09-10T22:17:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:49 smithi090 ceph-mon[30350]: pgmap v852: 129 pgs: 129 active+clean; 439 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-09-10T22:17:50.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:49 smithi154 ceph-mon[34495]: pgmap v852: 129 pgs: 129 active+clean; 439 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-09-10T22:17:50.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 5753 11.28 MB/sec warmup 41 sec latency 65.069 ms 2024-09-10T22:17:51.855 INFO:tasks.workunit.client.0.smithi090.stdout: 10 5865 11.14 MB/sec warmup 42 sec latency 65.970 ms 2024-09-10T22:17:52.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:51 smithi171 ceph-mon[34138]: pgmap v853: 129 pgs: 129 active+clean; 439 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.2 MiB/s wr, 224 op/s 2024-09-10T22:17:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:51 smithi090 ceph-mon[30350]: pgmap v853: 129 pgs: 129 active+clean; 439 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.2 MiB/s wr, 224 op/s 2024-09-10T22:17:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:51 smithi154 ceph-mon[34495]: pgmap v853: 129 pgs: 129 active+clean; 439 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.2 MiB/s wr, 224 op/s 2024-09-10T22:17:52.855 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6027 11.23 MB/sec warmup 43 sec latency 59.861 ms 2024-09-10T22:17:53.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6152 11.22 MB/sec warmup 44 sec latency 250.825 ms 2024-09-10T22:17:54.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:53 smithi171 ceph-mon[34138]: pgmap v854: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 3.3 MiB/s wr, 252 op/s 2024-09-10T22:17:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:53 smithi090 ceph-mon[30350]: pgmap v854: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 3.3 MiB/s wr, 252 op/s 2024-09-10T22:17:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:53 smithi154 ceph-mon[34495]: pgmap v854: 129 pgs: 129 active+clean; 445 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 3.3 MiB/s wr, 252 op/s 2024-09-10T22:17:54.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6302 11.34 MB/sec warmup 45 sec latency 639.137 ms 2024-09-10T22:17:55.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6511 11.38 MB/sec warmup 46 sec latency 65.948 ms 2024-09-10T22:17:56.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:55 smithi171 ceph-mon[34138]: pgmap v855: 129 pgs: 129 active+clean; 454 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-09-10T22:17:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:55 smithi090 ceph-mon[30350]: pgmap v855: 129 pgs: 129 active+clean; 454 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-09-10T22:17:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:55 smithi154 ceph-mon[34495]: pgmap v855: 129 pgs: 129 active+clean; 454 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 256 op/s 2024-09-10T22:17:56.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6709 11.40 MB/sec warmup 47 sec latency 68.576 ms 2024-09-10T22:17:57.609 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T22:17:57.609 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T22:17:57.610 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:17:57.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6765 11.19 MB/sec warmup 48 sec latency 73.183 ms 2024-09-10T22:17:57.975 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:57 smithi090 ceph-mon[30350]: pgmap v856: 129 pgs: 129 active+clean; 460 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 3.4 MiB/s wr, 201 op/s 2024-09-10T22:17:58.183 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:17:58.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:57 smithi171 ceph-mon[34138]: pgmap v856: 129 pgs: 129 active+clean; 460 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 3.4 MiB/s wr, 201 op/s 2024-09-10T22:17:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:57 smithi154 ceph-mon[34495]: pgmap v856: 129 pgs: 129 active+clean; 460 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 3.4 MiB/s wr, 201 op/s 2024-09-10T22:17:58.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6883 11.20 MB/sec warmup 49 sec latency 72.978 ms 2024-09-10T22:17:59.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6954 11.12 MB/sec warmup 50 sec latency 595.374 ms 2024-09-10T22:18:00.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:17:59 smithi171 ceph-mon[34138]: pgmap v857: 129 pgs: 129 active+clean; 465 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 4.1 MiB/s wr, 229 op/s 2024-09-10T22:18:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:17:59 smithi090 ceph-mon[30350]: pgmap v857: 129 pgs: 129 active+clean; 465 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 4.1 MiB/s wr, 229 op/s 2024-09-10T22:18:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:17:59 smithi154 ceph-mon[34495]: pgmap v857: 129 pgs: 129 active+clean; 465 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 4.1 MiB/s wr, 229 op/s 2024-09-10T22:18:00.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7004 11.05 MB/sec warmup 51 sec latency 835.220 ms 2024-09-10T22:18:01.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7004 10.84 MB/sec warmup 52 sec latency 1835.354 ms 2024-09-10T22:18:02.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:01 smithi171 ceph-mon[34138]: pgmap v858: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.8 MiB/s wr, 167 op/s 2024-09-10T22:18:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:01 smithi090 ceph-mon[30350]: pgmap v858: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.8 MiB/s wr, 167 op/s 2024-09-10T22:18:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:01 smithi154 ceph-mon[34495]: pgmap v858: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 3.8 MiB/s wr, 167 op/s 2024-09-10T22:18:02.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7004 10.63 MB/sec warmup 53 sec latency 2835.548 ms 2024-09-10T22:18:03.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:03 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:18:03.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7023 10.46 MB/sec warmup 54 sec latency 3604.090 ms 2024-09-10T22:18:03.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:03 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:18:03.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:03 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:18:04.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:04 smithi090 ceph-mon[30350]: pgmap v859: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 4.2 MiB/s wr, 178 op/s 2024-09-10T22:18:04.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7023 10.27 MB/sec warmup 55 sec latency 1223.091 ms 2024-09-10T22:18:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:04 smithi154 ceph-mon[34495]: pgmap v859: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 4.2 MiB/s wr, 178 op/s 2024-09-10T22:18:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:04 smithi171 ceph-mon[34138]: pgmap v859: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 4.2 MiB/s wr, 178 op/s 2024-09-10T22:18:05.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.164+0000 7f3c94800640 1 -- 172.21.15.90:0/1152977213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c102dc0 msgr2=0x7f3c8c1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.165+0000 7f3c94800640 1 --2- 172.21.15.90:0/1152977213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c102dc0 0x7f3c8c1031a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009900 tx=0x7f3c8002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.165+0000 7f3c94800640 1 -- 172.21.15.90:0/1152977213 shutdown_connections 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.165+0000 7f3c94800640 1 --2- 172.21.15.90:0/1152977213 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8c109150 0x7f3c8c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.165+0000 7f3c94800640 1 --2- 172.21.15.90:0/1152977213 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c8c1087d0 0x7f3c8c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.165+0000 7f3c94800640 1 --2- 172.21.15.90:0/1152977213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c102dc0 0x7f3c8c1031a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.165+0000 7f3c94800640 1 -- 172.21.15.90:0/1152977213 >> 172.21.15.90:0/1152977213 conn(0x7f3c8c0fe830 msgr2=0x7f3c8c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.165+0000 7f3c94800640 1 -- 172.21.15.90:0/1152977213 shutdown_connections 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.165+0000 7f3c94800640 1 -- 172.21.15.90:0/1152977213 wait complete. 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.166+0000 7f3c94800640 1 Processor -- start 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.166+0000 7f3c94800640 1 -- start start 2024-09-10T22:18:05.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.166+0000 7f3c94800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8c102dc0 0x7f3c8c10eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.166+0000 7f3c94800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c1087d0 0x7f3c8c10f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.166+0000 7f3c94800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c8c109150 0x7f3c8c1a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.166+0000 7f3c94800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c112d10 con 0x7f3c8c1087d0 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.166+0000 7f3c94800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c112b90 con 0x7f3c8c102dc0 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.166+0000 7f3c94800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c112e90 con 0x7f3c8c109150 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c1087d0 0x7f3c8c10f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c1087d0 0x7f3c8c10f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43716/0 (socket says 172.21.15.90:43716) 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91400640 1 -- 172.21.15.90:0/2053731451 learned_addr learned my addr 172.21.15.90:0/2053731451 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91e00640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8c102dc0 0x7f3c8c10eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:05.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c92800640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c8c109150 0x7f3c8c1a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91400640 1 -- 172.21.15.90:0/2053731451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c8c109150 msgr2=0x7f3c8c1a1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91400640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c8c109150 0x7f3c8c1a1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91400640 1 -- 172.21.15.90:0/2053731451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8c102dc0 msgr2=0x7f3c8c10eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91400640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8c102dc0 0x7f3c8c10eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91400640 1 -- 172.21.15.90:0/2053731451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c1a2660 con 0x7f3c8c1087d0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.167+0000 7f3c91400640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c1087d0 0x7f3c8c10f030 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e9f0 tx=0x7f3c7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.168+0000 7f3c86a00640 1 -- 172.21.15.90:0/2053731451 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c00cd80 con 0x7f3c8c1087d0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.168+0000 7f3c86a00640 1 -- 172.21.15.90:0/2053731451 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3c7c0108c0 con 0x7f3c8c1087d0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.168+0000 7f3c86a00640 1 -- 172.21.15.90:0/2053731451 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c0187c0 con 0x7f3c8c1087d0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.168+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8c1a2950 con 0x7f3c8c1087d0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.168+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8c1ac2c0 con 0x7f3c8c1087d0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.169+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8c1031a0 con 0x7f3c8c1087d0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.171+0000 7f3c86a00640 1 -- 172.21.15.90:0/2053731451 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3c7c010430 con 0x7f3c8c1087d0 2024-09-10T22:18:05.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.173+0000 7f3c86a00640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c740b5450 0x7f3c740b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.173+0000 7f3c86a00640 1 -- 172.21.15.90:0/2053731451 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9490+0+0 (secure 0 0 0) 0x7f3c7c0e6790 con 0x7f3c8c1087d0 2024-09-10T22:18:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.173+0000 7f3c91e00640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c740b5450 0x7f3c740b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.174+0000 7f3c91e00640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c740b5450 0x7f3c740b7910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3c80006010 tx=0x7f3c80005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:05.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.177+0000 7f3c86a00640 1 -- 172.21.15.90:0/2053731451 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3c7c0ac890 con 0x7f3c8c1087d0 2024-09-10T22:18:05.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.409+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c8c1ac630 con 0x7f3c8c1087d0 2024-09-10T22:18:05.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.410+0000 7f3c86a00640 1 -- 172.21.15.90:0/2053731451 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+12013 (secure 0 0 0) 0x7f3c7c0b12c0 con 0x7f3c8c1087d0 2024-09-10T22:18:05.412 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:18:05.413 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":48,"btime":"2024-09-10T22:17:37:801370+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:17:37.751219+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:18:05.413 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 48 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c740b5450 msgr2=0x7f3c740b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c740b5450 0x7f3c740b7910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3c80006010 tx=0x7f3c80005e80 comp rx=0 tx=0).stop 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c1087d0 msgr2=0x7f3c8c10f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c1087d0 0x7f3c8c10f030 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e9f0 tx=0x7f3c7c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 shutdown_connections 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c740b5450 0x7f3c740b7910 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3c8c109150 0x7f3c8c1a1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c8c1087d0 0x7f3c8c10f030 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 --2- 172.21.15.90:0/2053731451 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3c8c102dc0 0x7f3c8c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:05.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 >> 172.21.15.90:0/2053731451 conn(0x7f3c8c0fe830 msgr2=0x7f3c8c0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:05.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 shutdown_connections 2024-09-10T22:18:05.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:05.413+0000 7f3c94800640 1 -- 172.21.15.90:0/2053731451 wait complete. 2024-09-10T22:18:05.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:18:05.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:18:05.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:18:05.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:05 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2053731451' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:05.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7023 10.09 MB/sec warmup 56 sec latency 2223.139 ms 2024-09-10T22:18:05.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:18:05.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:18:05.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:18:05.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:05 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2053731451' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:05.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:18:05.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:18:05.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:18:05.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:05 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2053731451' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:06.788 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T22:18:06.788 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T22:18:06.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7023 9.91 MB/sec warmup 57 sec latency 3223.178 ms 2024-09-10T22:18:06.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:06 smithi154 ceph-mon[34495]: pgmap v860: 129 pgs: 129 active+clean; 467 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 3.2 MiB/s wr, 164 op/s 2024-09-10T22:18:06.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:06 smithi171 ceph-mon[34138]: pgmap v860: 129 pgs: 129 active+clean; 467 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 3.2 MiB/s wr, 164 op/s 2024-09-10T22:18:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:06 smithi090 ceph-mon[30350]: pgmap v860: 129 pgs: 129 active+clean; 467 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 3.2 MiB/s wr, 164 op/s 2024-09-10T22:18:07.159 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:18:07.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.779+0000 7ff52c000640 1 -- 172.21.15.90:0/599143769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5241087d0 msgr2=0x7ff524108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:07.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.779+0000 7ff52c000640 1 --2- 172.21.15.90:0/599143769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5241087d0 0x7ff524108c10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff510009900 tx=0x7ff51002a410 comp rx=0 tx=0).stop 2024-09-10T22:18:07.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.780+0000 7ff52c000640 1 -- 172.21.15.90:0/599143769 shutdown_connections 2024-09-10T22:18:07.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.780+0000 7ff52c000640 1 --2- 172.21.15.90:0/599143769 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff524109150 0x7ff52410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.780+0000 7ff52c000640 1 --2- 172.21.15.90:0/599143769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5241087d0 0x7ff524108c10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.780+0000 7ff52c000640 1 --2- 172.21.15.90:0/599143769 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff524102dc0 0x7ff5241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.780+0000 7ff52c000640 1 -- 172.21.15.90:0/599143769 >> 172.21.15.90:0/599143769 conn(0x7ff5240fe830 msgr2=0x7ff524100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:07.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.780+0000 7ff52c000640 1 -- 172.21.15.90:0/599143769 shutdown_connections 2024-09-10T22:18:07.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.780+0000 7ff52c000640 1 -- 172.21.15.90:0/599143769 wait complete. 2024-09-10T22:18:07.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.781+0000 7ff52c000640 1 Processor -- start 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.781+0000 7ff52c000640 1 -- start start 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.781+0000 7ff52c000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff524102dc0 0x7ff52419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.781+0000 7ff52c000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5241087d0 0x7ff52419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.781+0000 7ff52c000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff524109150 0x7ff5241a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.781+0000 7ff52c000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff524112d10 con 0x7ff524109150 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.781+0000 7ff52c000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff524112b90 con 0x7ff524102dc0 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.781+0000 7ff52c000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff524112e90 con 0x7ff5241087d0 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff529600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff524102dc0 0x7ff52419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff52a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff524109150 0x7ff5241a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff529600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff524102dc0 0x7ff52419b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:59862/0 (socket says 172.21.15.90:59862) 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff529600640 1 -- 172.21.15.90:0/2228909898 learned_addr learned my addr 172.21.15.90:0/2228909898 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:07.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff528c00640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5241087d0 0x7ff52419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff52a000640 1 -- 172.21.15.90:0/2228909898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5241087d0 msgr2=0x7ff52419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:07.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff52a000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5241087d0 0x7ff52419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff52a000640 1 -- 172.21.15.90:0/2228909898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff524102dc0 msgr2=0x7ff52419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:07.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff52a000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff524102dc0 0x7ff52419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.782+0000 7ff52a000640 1 -- 172.21.15.90:0/2228909898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5241a1490 con 0x7ff524109150 2024-09-10T22:18:07.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.783+0000 7ff52a000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff524109150 0x7ff5241a0cc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00e940 tx=0x7ff51c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:07.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.783+0000 7ff522000640 1 -- 172.21.15.90:0/2228909898 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c004d20 con 0x7ff524109150 2024-09-10T22:18:07.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.783+0000 7ff522000640 1 -- 172.21.15.90:0/2228909898 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff51c005020 con 0x7ff524109150 2024-09-10T22:18:07.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.783+0000 7ff52c000640 1 -- 172.21.15.90:0/2228909898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5241a1780 con 0x7ff524109150 2024-09-10T22:18:07.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.783+0000 7ff522000640 1 -- 172.21.15.90:0/2228909898 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0116d0 con 0x7ff524109150 2024-09-10T22:18:07.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.783+0000 7ff52c000640 1 -- 172.21.15.90:0/2228909898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5241a7e40 con 0x7ff524109150 2024-09-10T22:18:07.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.786+0000 7ff52ea8b740 1 Processor -- start 2024-09-10T22:18:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.788+0000 7ff522000640 1 -- 172.21.15.90:0/2228909898 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff51c00f040 con 0x7ff524109150 2024-09-10T22:18:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.788+0000 7ff52ea8b740 1 -- start start 2024-09-10T22:18:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d97998c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d97998cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 0x55d97999a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d979901a90 con 0x55d97998c6e0 2024-09-10T22:18:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d9798f0ec0 con 0x55d97998dc40 2024-09-10T22:18:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d9798f4cd0 con 0x55d979998510 2024-09-10T22:18:07.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff528c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d97998c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff529600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d97998cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff528c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d97998c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:59866/0 (socket says 172.21.15.90:59866) 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff528c00640 1 -- 172.21.15.90:0/2049483087 learned_addr learned my addr 172.21.15.90:0/2049483087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.789+0000 7ff528c00640 1 --2- 172.21.15.90:0/2049483087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 0x55d97999a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff529600640 1 -- 172.21.15.90:0/2049483087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 msgr2=0x55d97999a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff529600640 1 --2- 172.21.15.90:0/2049483087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 0x55d97999a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff529600640 1 -- 172.21.15.90:0/2049483087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 msgr2=0x55d97998c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff529600640 1 --2- 172.21.15.90:0/2049483087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d97998c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff529600640 1 -- 172.21.15.90:0/2049483087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9798969d0 con 0x55d97998c6e0 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff528c00640 1 --2- 172.21.15.90:0/2049483087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d97998c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:18:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff522000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b54d0 0x7ff50c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff529600640 1 --2- 172.21.15.90:0/2049483087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d97998cb00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff5180058c0 tx=0x7ff5180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce3ee1a3cbd321b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff522000640 1 -- 172.21.15.90:0/2228909898 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9490+0+0 (secure 0 0 0) 0x7ff51c0e6110 con 0x7ff524109150 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.790+0000 7ff528c00640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b54d0 0x7ff50c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.791+0000 7ff516600640 1 -- 172.21.15.90:0/2049483087 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518004230 con 0x55d97998c6e0 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.791+0000 7ff516600640 1 -- 172.21.15.90:0/2049483087 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff51800be90 con 0x55d97998c6e0 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.791+0000 7ff516600640 1 -- 172.21.15.90:0/2049483087 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5180116a0 con 0x55d97998c6e0 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2049483087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 msgr2=0x55d97998cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2049483087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d97998cb00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff5180058c0 tx=0x7ff5180107a0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff528c00640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b54d0 0x7ff50c0b7990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff5100096c0 tx=0x7ff510005b50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2049483087 shutdown_connections 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2049483087 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 0x55d97999a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2049483087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d97998cb00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2049483087 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d97998c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2049483087 >> 172.21.15.90:0/2049483087 conn(0x55d9799566f0 msgr2=0x55d979989fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2049483087 shutdown_connections 2024-09-10T22:18:07.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2049483087 wait complete. 2024-09-10T22:18:07.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.792+0000 7ff52ea8b740 1 -- start start 2024-09-10T22:18:07.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.793+0000 7ff52ea8b740 1 Processor -- start 2024-09-10T22:18:07.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.793+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 0x55d979995500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.793+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979995a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.793+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979995f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.793+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d9797527b0 con 0x55d979998510 2024-09-10T22:18:07.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.793+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d979895650 con 0x55d97998dc40 2024-09-10T22:18:07.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.793+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d97999b330 con 0x55d97998c6e0 2024-09-10T22:18:07.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.793+0000 7ff52ea8b740 10 client.0 _ll_register_callbacks cb 0x55d9795fc0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff52a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979995f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff528c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979995a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff528c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979995a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:41828/0 (socket says 172.21.15.90:41828) 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff528c00640 1 -- 172.21.15.90:0/1567347587 learned_addr learned my addr 172.21.15.90:0/1567347587 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff529600640 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 0x55d979995500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff528c00640 1 -- 172.21.15.90:0/1567347587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 msgr2=0x55d979995500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff528c00640 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 0x55d979995500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff528c00640 1 -- 172.21.15.90:0/1567347587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 msgr2=0x55d979995f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff528c00640 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979995f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff528c00640 1 -- 172.21.15.90:0/1567347587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97965eba0 con 0x55d97998dc40 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff528c00640 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979995a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff510009230 tx=0x7ff51002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7820d95008b48497 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:07.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.794+0000 7ff50aa00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:18:07.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.795+0000 7ff50aa00640 1 -- 172.21.15.90:0/1567347587 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510035040 con 0x55d97998dc40 2024-09-10T22:18:07.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.795+0000 7ff52ea8b740 1 -- 172.21.15.90:0/1567347587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51800be90 con 0x55d97998dc40 2024-09-10T22:18:07.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.795+0000 7ff50aa00640 1 -- 172.21.15.90:0/1567347587 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9490+0+0 (secure 0 0 0) 0x7ff5100035f0 con 0x55d97998dc40 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.795+0000 7ff50aa00640 1 -- 172.21.15.90:0/1567347587 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff510003830 con 0x55d97998dc40 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.795+0000 7ff52ea8b740 10 client.35852 fetch_fsmap learned FSMap version 48 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.795+0000 7ff52ea8b740 1 -- 172.21.15.90:0/1567347587 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d9799a6f40 con 0x55d97998dc40 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.796+0000 7ff50aa00640 1 -- 172.21.15.90:0/1567347587 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff51800be90 con 0x55d97998dc40 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.796+0000 7ff50aa00640 1 -- 172.21.15.90:0/1567347587 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 48) ==== 8434+0+0 (secure 0 0 0) 0x7ff51003a040 con 0x55d97998dc40 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.796+0000 7ff52ea8b740 10 client.35852 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.796+0000 7ff52ea8b740 10 client.35852 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.796+0000 7ff52ea8b740 20 client.35852 populate_metadata read hostname 'smithi090' 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.796+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d9799a8450 0x55d9799aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.796+0000 7ff52ea8b740 4 client.35852 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.796+0000 7ff52ea8b740 1 -- 172.21.15.90:0/1567347587 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9799039e0 con 0x55d9799a8450 2024-09-10T22:18:07.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.797+0000 7ff529600640 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d9799a8450 0x55d9799aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.797+0000 7ff529600640 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d9799a8450 0x55d9799aa8e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feeafa6dee16856a server_cookie=9a8d4626f7bea659 in_seq=0 out_seq=0 2024-09-10T22:18:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.797+0000 7ff50aa00640 10 client.35852 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.798+0000 7ff50aa00640 1 -- 172.21.15.90:0/1567347587 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9799039e0 con 0x55d9799a8450 2024-09-10T22:18:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.798+0000 7ff50aa00640 10 client.35852 handle_command_reply: tid=0 2024-09-10T22:18:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.798+0000 7ff52ea8b740 1 client.35852 shutdown 2024-09-10T22:18:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.798+0000 7ff52ea8b740 1 -- 172.21.15.90:0/1567347587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 msgr2=0x55d979995a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:07.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.798+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979995a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff510009230 tx=0x7ff51002a410 comp rx=0 tx=0).stop 2024-09-10T22:18:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff52ea8b740 1 -- 172.21.15.90:0/1567347587 shutdown_connections 2024-09-10T22:18:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d9799a8450 0x55d9799aa8e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979995f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979995a40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff50aa00640 0 client.35852 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:07.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/1567347587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 0x55d979995500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff52ea8b740 1 -- 172.21.15.90:0/1567347587 >> 172.21.15.90:0/1567347587 conn(0x55d9799566f0 msgr2=0x55d97998fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:07.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff52ea8b740 1 -- 172.21.15.90:0/1567347587 shutdown_connections 2024-09-10T22:18:07.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff52ea8b740 1 -- 172.21.15.90:0/1567347587 wait complete. 2024-09-10T22:18:07.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.799+0000 7ff52ea8b740 20 client.35852 trim_cache size 0 max 16384 2024-09-10T22:18:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.817+0000 7ff52ea8b740 1 Processor -- start 2024-09-10T22:18:07.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.817+0000 7ff52ea8b740 1 -- start start 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 0x55d97998a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979992730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979992c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b20d0 con 0x55d979998510 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b3e50 con 0x55d97998dc40 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b2800 con 0x55d97998c6e0 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979992c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979992c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54838/0 (socket says 172.21.15.90:54838) 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52a000640 1 -- 172.21.15.90:0/2003840489 learned_addr learned my addr 172.21.15.90:0/2003840489 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff528c00640 1 --2- 172.21.15.90:0/2003840489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979992730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff529600640 1 --2- 172.21.15.90:0/2003840489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 0x55d97998a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52a000640 1 -- 172.21.15.90:0/2003840489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 msgr2=0x55d97998a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52a000640 1 --2- 172.21.15.90:0/2003840489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 0x55d97998a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52a000640 1 -- 172.21.15.90:0/2003840489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 msgr2=0x55d979992730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.818+0000 7ff52a000640 1 --2- 172.21.15.90:0/2003840489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979992730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.819+0000 7ff52a000640 1 -- 172.21.15.90:0/2003840489 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d979993a00 con 0x55d979998510 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.819+0000 7ff52a000640 1 --2- 172.21.15.90:0/2003840489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979992c70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0a8790 tx=0x7ff51c0ab9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa599cb5b91c418b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.819+0000 7ff515c00640 1 -- 172.21.15.90:0/2003840489 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c01f040 con 0x55d979998510 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.819+0000 7ff515c00640 1 -- 172.21.15.90:0/2003840489 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff51c0abcd0 con 0x55d979998510 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.819+0000 7ff515c00640 1 -- 172.21.15.90:0/2003840489 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c0e3710 con 0x55d979998510 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.820+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2003840489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 msgr2=0x55d979992c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.820+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2003840489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979992c70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff51c0a8790 tx=0x7ff51c0ab9f0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.820+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2003840489 shutdown_connections 2024-09-10T22:18:07.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.820+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2003840489 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d979998510 0x55d979992c70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.820+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2003840489 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d979992730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.820+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2003840489 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d97998c6e0 0x55d97998a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.820+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2003840489 >> 172.21.15.90:0/2003840489 conn(0x55d9799566f0 msgr2=0x55d979989980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:07.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.820+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2003840489 shutdown_connections 2024-09-10T22:18:07.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.820+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2003840489 wait complete. 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff52ea8b740 1 -- start start 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff52ea8b740 1 Processor -- start 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d9799a0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d9799a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff52ea8b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 0x55d979991780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b36c0 con 0x55d97998c6e0 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b2f90 con 0x55d97998dc40 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff52ea8b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff5241b00d0 con 0x55d979998510 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff52ea8b740 10 client.0 _ll_register_callbacks cb 0x7ff5100055f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:18:07.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.821+0000 7ff529600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d9799a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff529600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d9799a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54846/0 (socket says 172.21.15.90:54846) 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff529600640 1 -- 172.21.15.90:0/2209890996 learned_addr learned my addr 172.21.15.90:0/2209890996 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff528c00640 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d9799a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff52a000640 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 0x55d979991780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff529600640 1 -- 172.21.15.90:0/2209890996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 msgr2=0x55d979991780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff529600640 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 0x55d979991780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff529600640 1 -- 172.21.15.90:0/2209890996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 msgr2=0x55d9799a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff529600640 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d9799a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff529600640 1 -- 172.21.15.90:0/2209890996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d979991fb0 con 0x55d97998c6e0 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff529600640 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d9799a0910 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff518010ca0 tx=0x7ff51800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796012bc3f5da4bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.822+0000 7ff50aa00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.823+0000 7ff50aa00640 1 -- 172.21.15.90:0/2209890996 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518005480 con 0x55d97998c6e0 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.823+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2209890996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51c0abcd0 con 0x55d97998c6e0 2024-09-10T22:18:07.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.823+0000 7ff50aa00640 1 -- 172.21.15.90:0/2209890996 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9490+0+0 (secure 0 0 0) 0x7ff5180035f0 con 0x55d97998c6e0 2024-09-10T22:18:07.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.823+0000 7ff50aa00640 1 -- 172.21.15.90:0/2209890996 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff518038050 con 0x55d97998c6e0 2024-09-10T22:18:07.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.823+0000 7ff52ea8b740 10 client.16224 fetch_fsmap learned FSMap version 48 2024-09-10T22:18:07.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.823+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2209890996 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d979993290 con 0x55d97998c6e0 2024-09-10T22:18:07.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.823+0000 7ff50aa00640 1 -- 172.21.15.90:0/2209890996 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff51c0abcd0 con 0x55d97998c6e0 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.823+0000 7ff50aa00640 1 -- 172.21.15.90:0/2209890996 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 8434+0+0 (secure 0 0 0) 0x7ff5180115d0 con 0x55d97998c6e0 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.824+0000 7ff52ea8b740 10 client.16224 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.824+0000 7ff52ea8b740 10 client.16224 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.824+0000 7ff52ea8b740 20 client.16224 populate_metadata read hostname 'smithi090' 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.824+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d979a16510 0x55d979a18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.824+0000 7ff52ea8b740 4 client.16224 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.824+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2209890996 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x55d9798ff490 con 0x55d979a16510 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.824+0000 7ff528c00640 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d979a16510 0x55d979a18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.825+0000 7ff528c00640 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d979a16510 0x55d979a18900 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7950c83f67d7396 server_cookie=ba0e435014943129 in_seq=0 out_seq=0 2024-09-10T22:18:07.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:07.825+0000 7ff50aa00640 10 client.16224 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:07.856 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7023 9.74 MB/sec warmup 58 sec latency 4223.219 ms 2024-09-10T22:18:08.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff50aa00640 1 -- 172.21.15.90:0/2209890996 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d9798ff490 con 0x55d979a16510 2024-09-10T22:18:08.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff50aa00640 10 client.16224 handle_command_reply: tid=0 2024-09-10T22:18:08.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff52ea8b740 1 client.16224 shutdown 2024-09-10T22:18:08.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2209890996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 msgr2=0x55d9799a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:08.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d9799a0910 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff518010ca0 tx=0x7ff51800b730 comp rx=0 tx=0).stop 2024-09-10T22:18:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2209890996 shutdown_connections 2024-09-10T22:18:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d979a16510 0x55d979a18900 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d979998510 0x55d979991780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:08.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d97998dc40 0x55d9799a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:08.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff52ea8b740 1 --2- 172.21.15.90:0/2209890996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d97998c6e0 0x55d9799a0910 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:08.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2209890996 >> 172.21.15.90:0/2209890996 conn(0x55d9799566f0 msgr2=0x55d9799a3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:08.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.106+0000 7ff50aa00640 0 client.16224 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:08.110 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:18:08.110 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 48, 2024-09-10T22:18:08.110 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 42, 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "455cd15e": { 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 42, 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 142.8, 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:18:08.111 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:08.112 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.107+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2209890996 shutdown_connections 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.107+0000 7ff52ea8b740 1 -- 172.21.15.90:0/2209890996 wait complete. 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.107+0000 7ff52ea8b740 20 client.16224 trim_cache size 0 max 16384 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 -- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b54d0 msgr2=0x7ff50c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b54d0 0x7ff50c0b7990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff5100096c0 tx=0x7ff510005b50 comp rx=0 tx=0).stop 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 -- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff524109150 msgr2=0x7ff5241a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff524109150 0x7ff5241a0cc0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00e940 tx=0x7ff51c00ee00 comp rx=0 tx=0).stop 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 -- 172.21.15.90:0/2228909898 shutdown_connections 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff50c0b54d0 0x7ff50c0b7990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff524109150 0x7ff5241a0cc0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:08.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff5241087d0 0x7ff52419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:08.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 --2- 172.21.15.90:0/2228909898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff524102dc0 0x7ff52419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:08.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 -- 172.21.15.90:0/2228909898 >> 172.21.15.90:0/2228909898 conn(0x7ff5240fe830 msgr2=0x7ff524104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:08.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.108+0000 7ff52c000640 1 -- 172.21.15.90:0/2228909898 shutdown_connections 2024-09-10T22:18:08.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:08.109+0000 7ff52c000640 1 -- 172.21.15.90:0/2228909898 wait complete. 2024-09-10T22:18:08.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:07 smithi171 ceph-mon[34138]: pgmap v861: 129 pgs: 129 active+clean; 468 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-09-10T22:18:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:07 smithi090 ceph-mon[30350]: pgmap v861: 129 pgs: 129 active+clean; 468 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-09-10T22:18:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:07 smithi154 ceph-mon[34495]: pgmap v861: 129 pgs: 129 active+clean; 468 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 2.2 MiB/s wr, 105 op/s 2024-09-10T22:18:08.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 9.62 MB/sec warmup 59 sec latency 5158.221 ms 2024-09-10T22:18:09.609 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '455cd15e', quiesce took 12.0 seconds. Will release after: 10.75 2024-09-10T22:18:09.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 9.46 MB/sec warmup 60 sec latency 6158.266 ms 2024-09-10T22:18:10.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:09 smithi171 ceph-mon[34138]: osdmap e107: 12 total, 12 up, 12 in 2024-09-10T22:18:10.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:09 smithi171 ceph-mon[34138]: pgmap v863: 129 pgs: 129 active+clean; 467 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 824 KiB/s wr, 74 op/s 2024-09-10T22:18:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:09 smithi090 ceph-mon[30350]: osdmap e107: 12 total, 12 up, 12 in 2024-09-10T22:18:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:09 smithi090 ceph-mon[30350]: pgmap v863: 129 pgs: 129 active+clean; 467 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 824 KiB/s wr, 74 op/s 2024-09-10T22:18:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:09 smithi154 ceph-mon[34495]: osdmap e107: 12 total, 12 up, 12 in 2024-09-10T22:18:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:09 smithi154 ceph-mon[34495]: pgmap v863: 129 pgs: 129 active+clean; 467 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 824 KiB/s wr, 74 op/s 2024-09-10T22:18:10.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 9.31 MB/sec warmup 61 sec latency 7158.317 ms 2024-09-10T22:18:11.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 9.16 MB/sec warmup 62 sec latency 8158.391 ms 2024-09-10T22:18:12.251 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:11 smithi171 ceph-mon[34138]: pgmap v864: 129 pgs: 129 active+clean; 467 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 724 KiB/s wr, 72 op/s 2024-09-10T22:18:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:11 smithi090 ceph-mon[30350]: pgmap v864: 129 pgs: 129 active+clean; 467 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 724 KiB/s wr, 72 op/s 2024-09-10T22:18:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:11 smithi154 ceph-mon[34495]: pgmap v864: 129 pgs: 129 active+clean; 467 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 724 KiB/s wr, 72 op/s 2024-09-10T22:18:12.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 9.01 MB/sec warmup 63 sec latency 9158.532 ms 2024-09-10T22:18:13.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 8.87 MB/sec warmup 64 sec latency 10158.676 ms 2024-09-10T22:18:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:13 smithi171 ceph-mon[34138]: pgmap v865: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 341 KiB/s wr, 58 op/s 2024-09-10T22:18:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:13 smithi090 ceph-mon[30350]: pgmap v865: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 341 KiB/s wr, 58 op/s 2024-09-10T22:18:14.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:13 smithi154 ceph-mon[34495]: pgmap v865: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 341 KiB/s wr, 58 op/s 2024-09-10T22:18:14.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 8.74 MB/sec warmup 65 sec latency 11158.779 ms 2024-09-10T22:18:15.857 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 8.60 MB/sec warmup 66 sec latency 12158.922 ms 2024-09-10T22:18:16.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 8.48 MB/sec warmup 67 sec latency 13159.065 ms 2024-09-10T22:18:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:16 smithi171 ceph-mon[34138]: osdmap e108: 12 total, 12 up, 12 in 2024-09-10T22:18:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:16 smithi171 ceph-mon[34138]: pgmap v867: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 272 KiB/s wr, 51 op/s 2024-09-10T22:18:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:16 smithi090 ceph-mon[30350]: osdmap e108: 12 total, 12 up, 12 in 2024-09-10T22:18:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:16 smithi090 ceph-mon[30350]: pgmap v867: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 272 KiB/s wr, 51 op/s 2024-09-10T22:18:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:16 smithi154 ceph-mon[34495]: osdmap e108: 12 total, 12 up, 12 in 2024-09-10T22:18:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:16 smithi154 ceph-mon[34495]: pgmap v867: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 272 KiB/s wr, 51 op/s 2024-09-10T22:18:17.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 8.35 MB/sec warmup 68 sec latency 14159.200 ms 2024-09-10T22:18:18.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:17 smithi171 ceph-mon[34138]: pgmap v868: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 198 KiB/s wr, 26 op/s 2024-09-10T22:18:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:17 smithi090 ceph-mon[30350]: pgmap v868: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 198 KiB/s wr, 26 op/s 2024-09-10T22:18:18.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:17 smithi154 ceph-mon[34495]: pgmap v868: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 198 KiB/s wr, 26 op/s 2024-09-10T22:18:18.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 8.23 MB/sec warmup 69 sec latency 15159.275 ms 2024-09-10T22:18:19.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 8.11 MB/sec warmup 70 sec latency 16159.419 ms 2024-09-10T22:18:20.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:19 smithi171 ceph-mon[34138]: pgmap v869: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 138 KiB/s wr, 17 op/s 2024-09-10T22:18:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:19 smithi090 ceph-mon[30350]: pgmap v869: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 138 KiB/s wr, 17 op/s 2024-09-10T22:18:20.361 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:18:20.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:19 smithi154 ceph-mon[34495]: pgmap v869: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 138 KiB/s wr, 17 op/s 2024-09-10T22:18:20.747 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:18:20.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 8.00 MB/sec warmup 71 sec latency 17159.542 ms 2024-09-10T22:18:21.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.331+0000 7f6f25a00640 1 -- 172.21.15.90:0/3112434148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010a820 msgr2=0x7f6f2010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:21.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.331+0000 7f6f25a00640 1 --2- 172.21.15.90:0/3112434148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010a820 0x7f6f2010ac60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009900 tx=0x7f6f0c02a410 comp rx=0 tx=0).stop 2024-09-10T22:18:21.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.331+0000 7f6f25a00640 1 -- 172.21.15.90:0/3112434148 shutdown_connections 2024-09-10T22:18:21.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.331+0000 7f6f25a00640 1 --2- 172.21.15.90:0/3112434148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f2010b1a0 0x7f6f201119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:21.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.331+0000 7f6f25a00640 1 --2- 172.21.15.90:0/3112434148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010a820 0x7f6f2010ac60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.331+0000 7f6f25a00640 1 --2- 172.21.15.90:0/3112434148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f20104e10 0x7f6f201051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.331+0000 7f6f25a00640 1 -- 172.21.15.90:0/3112434148 >> 172.21.15.90:0/3112434148 conn(0x7f6f20100840 msgr2=0x7f6f20102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.331+0000 7f6f25a00640 1 -- 172.21.15.90:0/3112434148 shutdown_connections 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.331+0000 7f6f25a00640 1 -- 172.21.15.90:0/3112434148 wait complete. 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.332+0000 7f6f25a00640 1 Processor -- start 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.332+0000 7f6f25a00640 1 -- start start 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.332+0000 7f6f25a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f20104e10 0x7f6f2019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.332+0000 7f6f25a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f2010a820 0x7f6f2019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.332+0000 7f6f25a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010b1a0 0x7f6f201a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.332+0000 7f6f25a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20114d60 con 0x7f6f2010b1a0 2024-09-10T22:18:21.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.332+0000 7f6f25a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20114be0 con 0x7f6f20104e10 2024-09-10T22:18:21.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.332+0000 7f6f25a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20114ee0 con 0x7f6f2010a820 2024-09-10T22:18:21.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f25000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010b1a0 0x7f6f201a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:21.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f25000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010b1a0 0x7f6f201a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35688/0 (socket says 172.21.15.90:35688) 2024-09-10T22:18:21.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f25000640 1 -- 172.21.15.90:0/4077438345 learned_addr learned my addr 172.21.15.90:0/4077438345 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:21.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f1fe00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f20104e10 0x7f6f2019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:21.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f1f400640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f2010a820 0x7f6f2019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:21.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f25000640 1 -- 172.21.15.90:0/4077438345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f2010a820 msgr2=0x7f6f2019b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f25000640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f2010a820 0x7f6f2019b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f25000640 1 -- 172.21.15.90:0/4077438345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f20104e10 msgr2=0x7f6f2019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f25000640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f20104e10 0x7f6f2019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f25000640 1 -- 172.21.15.90:0/4077438345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f201a1260 con 0x7f6f2010b1a0 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.333+0000 7f6f1f400640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f2010a820 0x7f6f2019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.334+0000 7f6f25000640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010b1a0 0x7f6f201a0a90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800d950 tx=0x7f6f1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.334+0000 7f6f1cc00640 1 -- 172.21.15.90:0/4077438345 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18017030 con 0x7f6f2010b1a0 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.334+0000 7f6f1cc00640 1 -- 172.21.15.90:0/4077438345 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f18004020 con 0x7f6f2010b1a0 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.334+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f201a1550 con 0x7f6f2010b1a0 2024-09-10T22:18:21.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.334+0000 7f6f1cc00640 1 -- 172.21.15.90:0/4077438345 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18003ab0 con 0x7f6f2010b1a0 2024-09-10T22:18:21.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.334+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f201a7c10 con 0x7f6f2010b1a0 2024-09-10T22:18:21.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.335+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f201051f0 con 0x7f6f2010b1a0 2024-09-10T22:18:21.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.337+0000 7f6f1cc00640 1 -- 172.21.15.90:0/4077438345 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6f18003c50 con 0x7f6f2010b1a0 2024-09-10T22:18:21.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.338+0000 7f6f1cc00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f040b5580 0x7f6f040b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:21.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.339+0000 7f6f1cc00640 1 -- 172.21.15.90:0/4077438345 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7f6f180e5e70 con 0x7f6f2010b1a0 2024-09-10T22:18:21.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.339+0000 7f6f1fe00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f040b5580 0x7f6f040b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:21.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.339+0000 7f6f1fe00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f040b5580 0x7f6f040b7a40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f10005fd0 tx=0x7f6f10009680 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:21.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.343+0000 7f6f1cc00640 1 -- 172.21.15.90:0/4077438345 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6f18021070 con 0x7f6f2010b1a0 2024-09-10T22:18:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.563+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f2002e5c0 con 0x7f6f2010b1a0 2024-09-10T22:18:21.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.564+0000 7f6f1cc00640 1 -- 172.21.15.90:0/4077438345 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+12013 (secure 0 0 0) 0x7f6f180ac5c0 con 0x7f6f2010b1a0 2024-09-10T22:18:21.566 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:18:21.566 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":48,"btime":"2024-09-10T22:17:37:801370+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:17:37.751219+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:18:21.566 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 48 2024-09-10T22:18:21.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f040b5580 msgr2=0x7f6f040b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:21.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f040b5580 0x7f6f040b7a40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f10005fd0 tx=0x7f6f10009680 comp rx=0 tx=0).stop 2024-09-10T22:18:21.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010b1a0 msgr2=0x7f6f201a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:21.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010b1a0 0x7f6f201a0a90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6f1800d950 tx=0x7f6f1800de10 comp rx=0 tx=0).stop 2024-09-10T22:18:21.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 shutdown_connections 2024-09-10T22:18:21.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f040b5580 0x7f6f040b7a40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:21.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f2010b1a0 0x7f6f201a0a90 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:21.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f2010a820 0x7f6f2019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:21.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 --2- 172.21.15.90:0/4077438345 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f20104e10 0x7f6f2019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:21.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.566+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 >> 172.21.15.90:0/4077438345 conn(0x7f6f20100840 msgr2=0x7f6f20102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:21.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.567+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 shutdown_connections 2024-09-10T22:18:21.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:21.567+0000 7f6f25a00640 1 -- 172.21.15.90:0/4077438345 wait complete. 2024-09-10T22:18:21.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:21 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4077438345' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:21.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 7.89 MB/sec warmup 72 sec latency 18159.627 ms 2024-09-10T22:18:21.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:21 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4077438345' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:21.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:21 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4077438345' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:22.345 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 455cd15e --release --await' 2024-09-10T22:18:22.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 455cd15e --release --await 2024-09-10T22:18:22.719 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:18:22.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7037 7.78 MB/sec warmup 73 sec latency 19159.671 ms 2024-09-10T22:18:22.920 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:22 smithi154 ceph-mon[34495]: pgmap v870: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 138 KiB/s wr, 16 op/s 2024-09-10T22:18:22.945 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:22 smithi090 ceph-mon[30350]: pgmap v870: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 138 KiB/s wr, 16 op/s 2024-09-10T22:18:22.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:22 smithi171 ceph-mon[34138]: pgmap v870: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 138 KiB/s wr, 16 op/s 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.423+0000 7f796dc00640 1 -- 172.21.15.90:0/3263066826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f796810f220 msgr2=0x7f7968111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.423+0000 7f796dc00640 1 --2- 172.21.15.90:0/3263066826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f796810f220 0x7f7968111610 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7960009ca0 tx=0x7f796002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.423+0000 7f796dc00640 1 -- 172.21.15.90:0/3263066826 shutdown_connections 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.423+0000 7f796dc00640 1 --2- 172.21.15.90:0/3263066826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f796810f220 0x7f7968111610 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.423+0000 7f796dc00640 1 --2- 172.21.15.90:0/3263066826 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7968075de0 0x7f796810ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.423+0000 7f796dc00640 1 --2- 172.21.15.90:0/3263066826 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79680754c0 0x7f79680758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.423+0000 7f796dc00640 1 -- 172.21.15.90:0/3263066826 >> 172.21.15.90:0/3263066826 conn(0x7f79680fe660 msgr2=0x7f7968100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.423+0000 7f796dc00640 1 -- 172.21.15.90:0/3263066826 shutdown_connections 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.423+0000 7f796dc00640 1 -- 172.21.15.90:0/3263066826 wait complete. 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.424+0000 7f796dc00640 1 Processor -- start 2024-09-10T22:18:23.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.424+0000 7f796dc00640 1 -- start start 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f796dc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79680754c0 0x7f796819b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f796dc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7968075de0 0x7f796819b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f796dc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f796810f220 0x7f79681a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f796dc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7968114d00 con 0x7f7968075de0 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f796dc00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7968114b80 con 0x7f79680754c0 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f796dc00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7968114e80 con 0x7f796810f220 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f7967400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7968075de0 0x7f796819b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f7967400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7968075de0 0x7f796819b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35700/0 (socket says 172.21.15.90:35700) 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f7967400640 1 -- 172.21.15.90:0/818691936 learned_addr learned my addr 172.21.15.90:0/818691936 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:23.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f7967e00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79680754c0 0x7f796819b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f796d200640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f796810f220 0x7f79681a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f7967400640 1 -- 172.21.15.90:0/818691936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f796810f220 msgr2=0x7f79681a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.425+0000 7f7967400640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f796810f220 0x7f79681a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.426+0000 7f7967400640 1 -- 172.21.15.90:0/818691936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79680754c0 msgr2=0x7f796819b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.426+0000 7f7967400640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79680754c0 0x7f796819b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.426+0000 7f7967400640 1 -- 172.21.15.90:0/818691936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79681a1280 con 0x7f7968075de0 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.426+0000 7f7967400640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7968075de0 0x7f796819b710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f795400d950 tx=0x7f795400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.426+0000 7f7964c00640 1 -- 172.21.15.90:0/818691936 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954017030 con 0x7f7968075de0 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.426+0000 7f7964c00640 1 -- 172.21.15.90:0/818691936 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7954004020 con 0x7f7968075de0 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.426+0000 7f7964c00640 1 -- 172.21.15.90:0/818691936 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540055d0 con 0x7f7968075de0 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.426+0000 7f796dc00640 1 -- 172.21.15.90:0/818691936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79681a1570 con 0x7f7968075de0 2024-09-10T22:18:23.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.426+0000 7f796dc00640 1 -- 172.21.15.90:0/818691936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79681a7e00 con 0x7f7968075de0 2024-09-10T22:18:23.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.430+0000 7f7971e33740 1 Processor -- start 2024-09-10T22:18:23.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.430+0000 7f7971e33740 1 -- start start 2024-09-10T22:18:23.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f7971e33740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf0395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f7971e33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf045540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f7971e33740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf045a80 0x561baf047e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f7971e33740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561baefb0290 con 0x561baf0391d0 2024-09-10T22:18:23.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f7971e33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561baef9f6c0 con 0x561baf039af0 2024-09-10T22:18:23.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f7971e33740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561baefa34d0 con 0x561baf045a80 2024-09-10T22:18:23.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f7964c00640 1 -- 172.21.15.90:0/818691936 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7954009a30 con 0x7f7968075de0 2024-09-10T22:18:23.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f796d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf0395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f796d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf0395b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35712/0 (socket says 172.21.15.90:35712) 2024-09-10T22:18:23.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f796d200640 1 -- 172.21.15.90:0/3798044502 learned_addr learned my addr 172.21.15.90:0/3798044502 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:23.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.431+0000 7f7967e00640 1 --2- 172.21.15.90:0/3798044502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf045540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f796d200640 1 -- 172.21.15.90:0/3798044502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf045a80 msgr2=0x561baf047e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:23.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f796d200640 1 --2- 172.21.15.90:0/3798044502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf045a80 0x561baf047e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f796d200640 1 -- 172.21.15.90:0/3798044502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 msgr2=0x561baf045540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:23.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f796d200640 1 --2- 172.21.15.90:0/3798044502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf045540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f796d200640 1 -- 172.21.15.90:0/3798044502 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561baef2c460 con 0x561baf0391d0 2024-09-10T22:18:23.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f7967e00640 1 --2- 172.21.15.90:0/3798044502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf045540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:18:23.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f796d200640 1 --2- 172.21.15.90:0/3798044502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf0395b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7960035000 tx=0x7f7960031450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e15d7a97fadcca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:23.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f795d600640 1 -- 172.21.15.90:0/3798044502 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960031980 con 0x561baf0391d0 2024-09-10T22:18:23.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f795d600640 1 -- 172.21.15.90:0/3798044502 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7960040070 con 0x561baf0391d0 2024-09-10T22:18:23.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.432+0000 7f795d600640 1 -- 172.21.15.90:0/3798044502 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79600043c0 con 0x561baf0391d0 2024-09-10T22:18:23.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.433+0000 7f7964c00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f794c0b5580 0x7f794c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.433+0000 7f7964c00640 1 -- 172.21.15.90:0/818691936 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7f79540e5d20 con 0x7f7968075de0 2024-09-10T22:18:23.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.433+0000 7f7967e00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f794c0b5580 0x7f794c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.433+0000 7f7971e33740 1 -- 172.21.15.90:0/3798044502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 msgr2=0x561baf0395b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:23.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.433+0000 7f7971e33740 1 --2- 172.21.15.90:0/3798044502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf0395b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7960035000 tx=0x7f7960031450 comp rx=0 tx=0).stop 2024-09-10T22:18:23.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7971e33740 1 -- 172.21.15.90:0/3798044502 shutdown_connections 2024-09-10T22:18:23.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7971e33740 1 --2- 172.21.15.90:0/3798044502 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf045a80 0x561baf047e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7971e33740 1 --2- 172.21.15.90:0/3798044502 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf045540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7971e33740 1 --2- 172.21.15.90:0/3798044502 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf0395b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7971e33740 1 -- 172.21.15.90:0/3798044502 >> 172.21.15.90:0/3798044502 conn(0x561baf037000 msgr2=0x561baf0381a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:23.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7971e33740 1 -- 172.21.15.90:0/3798044502 shutdown_connections 2024-09-10T22:18:23.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7971e33740 1 -- 172.21.15.90:0/3798044502 wait complete. 2024-09-10T22:18:23.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7967e00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f794c0b5580 0x7f794c0b7a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x561baf03c350 tx=0x7f7958005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:23.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7971e33740 1 -- start start 2024-09-10T22:18:23.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.434+0000 7f7971e33740 1 Processor -- start 2024-09-10T22:18:23.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f7971e33740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf0391d0 0x561baf042aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f7971e33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf042fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f7971e33740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf045a80 0x561baf043520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f7971e33740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561baee00bb0 con 0x561baf045a80 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f7971e33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561baef3a6c0 con 0x561baf039af0 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f7971e33740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561baf0486d0 con 0x561baf0391d0 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f7971e33740 10 client.0 _ll_register_callbacks cb 0x561baef55490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f796d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf045a80 0x561baf043520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f796d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf045a80 0x561baf043520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35716/0 (socket says 172.21.15.90:35716) 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f796d200640 1 -- 172.21.15.90:0/1240216170 learned_addr learned my addr 172.21.15.90:0/1240216170 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f7967e00640 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf0391d0 0x561baf042aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.435+0000 7f7967400640 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf042fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f796d200640 1 -- 172.21.15.90:0/1240216170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf0391d0 msgr2=0x561baf042aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f796d200640 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf0391d0 0x561baf042aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f796d200640 1 -- 172.21.15.90:0/1240216170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 msgr2=0x561baf042fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:23.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f796d200640 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf042fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f796d200640 1 -- 172.21.15.90:0/1240216170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561baecfdb00 con 0x561baf045a80 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f796d200640 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf045a80 0x561baf043520 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7960006010 tx=0x7f7960005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd8b13a4354ed26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f7952000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f7952000640 1 -- 172.21.15.90:0/1240216170 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960031d90 con 0x561baf045a80 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f7952000640 1 -- 172.21.15.90:0/1240216170 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7f7960004140 con 0x561baf045a80 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.436+0000 7f7971e33740 1 -- 172.21.15.90:0/1240216170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7960040070 con 0x561baf045a80 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7952000640 1 -- 172.21.15.90:0/1240216170 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7960004340 con 0x561baf045a80 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7971e33740 10 client.16242 fetch_fsmap learned FSMap version 48 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7971e33740 1 -- 172.21.15.90:0/1240216170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561baed74960 con 0x561baf045a80 2024-09-10T22:18:23.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7952000640 1 -- 172.21.15.90:0/1240216170 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7960040070 con 0x561baf045a80 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7952000640 1 -- 172.21.15.90:0/1240216170 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 8434+0+0 (secure 0 0 0) 0x7f7960058030 con 0x561baf045a80 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7971e33740 10 client.16242 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7971e33740 10 client.16242 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7971e33740 20 client.16242 populate_metadata read hostname 'smithi090' 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7971e33740 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561baf0554c0 0x561baf0578b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7971e33740 4 client.16242 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.437+0000 7f7971e33740 1 -- 172.21.15.90:0/1240216170 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561baefb21e0 con 0x561baf0554c0 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.438+0000 7f7967e00640 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561baf0554c0 0x561baf0578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.438+0000 7f7967e00640 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561baf0554c0 0x561baf0578b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6eeeea28530dae5 server_cookie=38f530556e9c76f0 in_seq=0 out_seq=0 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.438+0000 7f7952000640 10 client.16242 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.439+0000 7f7952000640 1 -- 172.21.15.90:0/1240216170 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561baefb21e0 con 0x561baf0554c0 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.439+0000 7f7952000640 10 client.16242 handle_command_reply: tid=0 2024-09-10T22:18:23.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.439+0000 7f7971e33740 1 client.16242 shutdown 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 -- 172.21.15.90:0/1240216170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf045a80 msgr2=0x561baf043520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf045a80 0x561baf043520 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7960006010 tx=0x7f7960005c60 comp rx=0 tx=0).stop 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 -- 172.21.15.90:0/1240216170 shutdown_connections 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561baf0554c0 0x561baf0578b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf045a80 0x561baf043520 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf042fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 --2- 172.21.15.90:0/1240216170 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf0391d0 0x561baf042aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 -- 172.21.15.90:0/1240216170 >> 172.21.15.90:0/1240216170 conn(0x561baf037000 msgr2=0x561baf03cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7952000640 0 client.16242 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 -- 172.21.15.90:0/1240216170 shutdown_connections 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 1 -- 172.21.15.90:0/1240216170 wait complete. 2024-09-10T22:18:23.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.440+0000 7f7971e33740 20 client.16242 trim_cache size 0 max 16384 2024-09-10T22:18:23.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.457+0000 7f7971e33740 1 Processor -- start 2024-09-10T22:18:23.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7971e33740 1 -- start start 2024-09-10T22:18:23.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7971e33740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf03d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7971e33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf038580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7971e33740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf045a80 0x561baf03fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7971e33740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f79681b1ea0 con 0x561baf0391d0 2024-09-10T22:18:23.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7971e33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f79681b3c20 con 0x561baf039af0 2024-09-10T22:18:23.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7971e33740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f79681b25d0 con 0x561baf045a80 2024-09-10T22:18:23.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7967e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf03d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7967e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf03d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35728/0 (socket says 172.21.15.90:35728) 2024-09-10T22:18:23.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7967e00640 1 -- 172.21.15.90:0/2508876981 learned_addr learned my addr 172.21.15.90:0/2508876981 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:23.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f7967400640 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf038580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.458+0000 7f796d200640 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf045a80 0x561baf03fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f7967e00640 1 -- 172.21.15.90:0/2508876981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf045a80 msgr2=0x561baf03fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f7967e00640 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf045a80 0x561baf03fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f7967e00640 1 -- 172.21.15.90:0/2508876981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 msgr2=0x561baf038580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f7967e00640 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf038580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f7967e00640 1 -- 172.21.15.90:0/2508876981 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561baf037a80 con 0x561baf0391d0 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f7967400640 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf038580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f7967e00640 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf03d390 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7958009a90 tx=0x7f7958019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54586fcd22c7955f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f795cc00640 1 -- 172.21.15.90:0/2508876981 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795801dc70 con 0x561baf0391d0 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f795cc00640 1 -- 172.21.15.90:0/2508876981 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f795801de10 con 0x561baf0391d0 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.459+0000 7f795cc00640 1 -- 172.21.15.90:0/2508876981 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958030750 con 0x561baf0391d0 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 -- 172.21.15.90:0/2508876981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 msgr2=0x561baf03d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:23.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf03d390 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7958009a90 tx=0x7f7958019000 comp rx=0 tx=0).stop 2024-09-10T22:18:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 -- 172.21.15.90:0/2508876981 shutdown_connections 2024-09-10T22:18:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf045a80 0x561baf03fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf039af0 0x561baf038580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 --2- 172.21.15.90:0/2508876981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf03d390 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 -- 172.21.15.90:0/2508876981 >> 172.21.15.90:0/2508876981 conn(0x561baf037000 msgr2=0x561baf0448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 -- 172.21.15.90:0/2508876981 shutdown_connections 2024-09-10T22:18:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 -- 172.21.15.90:0/2508876981 wait complete. 2024-09-10T22:18:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.460+0000 7f7971e33740 1 -- start start 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7971e33740 1 Processor -- start 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7971e33740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf058e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7971e33740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf039af0 0x561baf059410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7971e33740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf045a80 0x561baf0bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7971e33740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f79681b3490 con 0x561baf0391d0 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7971e33740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f79681b2d60 con 0x561baf045a80 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7971e33740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f79681afea0 con 0x561baf039af0 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7971e33740 10 client.0 _ll_register_callbacks cb 0x7f7960009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7967e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf058e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7967e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf058e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35742/0 (socket says 172.21.15.90:35742) 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.461+0000 7f7967e00640 1 -- 172.21.15.90:0/2715597683 learned_addr learned my addr 172.21.15.90:0/2715597683 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7967400640 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf039af0 0x561baf059410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7967e00640 1 -- 172.21.15.90:0/2715597683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf039af0 msgr2=0x561baf059410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7967e00640 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf039af0 0x561baf059410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7967e00640 1 -- 172.21.15.90:0/2715597683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf045a80 msgr2=0x561baf0bb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7967e00640 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf045a80 0x561baf0bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7967e00640 1 -- 172.21.15.90:0/2715597683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561baf0bb740 con 0x561baf0391d0 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7967400640 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf039af0 0x561baf059410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7967e00640 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf058e70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f795801efd0 tx=0x7f7958012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899f7c7864cdfd0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7952000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7952000640 1 -- 172.21.15.90:0/2715597683 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958040060 con 0x561baf0391d0 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7952000640 1 -- 172.21.15.90:0/2715597683 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7f7958005040 con 0x561baf0391d0 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.462+0000 7f7971e33740 1 -- 172.21.15.90:0/2715597683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f795801de10 con 0x561baf0391d0 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7952000640 1 -- 172.21.15.90:0/2715597683 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7958047050 con 0x561baf0391d0 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7971e33740 10 client.16248 fetch_fsmap learned FSMap version 48 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7971e33740 1 -- 172.21.15.90:0/2715597683 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x561baf0c2be0 con 0x561baf0391d0 2024-09-10T22:18:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7952000640 1 -- 172.21.15.90:0/2715597683 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f795801de10 con 0x561baf0391d0 2024-09-10T22:18:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7952000640 1 -- 172.21.15.90:0/2715597683 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 8434+0+0 (secure 0 0 0) 0x7f795802a450 con 0x561baf0391d0 2024-09-10T22:18:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7971e33740 10 client.16248 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T22:18:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7971e33740 10 client.16248 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:18:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7971e33740 20 client.16248 populate_metadata read hostname 'smithi090' 2024-09-10T22:18:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7971e33740 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561baf0c33c0 0x561baf0c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7971e33740 4 client.16248 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "455cd15e", "release": true, "await": true}] 2024-09-10T22:18:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.463+0000 7f7971e33740 1 -- 172.21.15.90:0/2715597683 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "455cd15e", "release": true, "await": true}) -- 0x561baefadc90 con 0x561baf0c33c0 2024-09-10T22:18:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.464+0000 7f7967400640 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561baf0c33c0 0x561baf0c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:23.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.464+0000 7f7967400640 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561baf0c33c0 0x561baf0c57e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418208c6fa15a851 server_cookie=2d2f3707b8cf7050 in_seq=0 out_seq=0 2024-09-10T22:18:23.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.464+0000 7f7952000640 10 client.16248 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:23.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7952000640 1 -- 172.21.15.90:0/2715597683 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561baefadc90 con 0x561baf0c33c0 2024-09-10T22:18:23.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7952000640 10 client.16248 handle_command_reply: tid=0 2024-09-10T22:18:23.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7971e33740 1 client.16248 shutdown 2024-09-10T22:18:23.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7971e33740 1 -- 172.21.15.90:0/2715597683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 msgr2=0x561baf058e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:23.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7971e33740 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf058e70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f795801efd0 tx=0x7f7958012f00 comp rx=0 tx=0).stop 2024-09-10T22:18:23.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7971e33740 1 -- 172.21.15.90:0/2715597683 shutdown_connections 2024-09-10T22:18:23.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7971e33740 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561baf0c33c0 0x561baf0c57e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7971e33740 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561baf045a80 0x561baf0bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7971e33740 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561baf039af0 0x561baf059410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7971e33740 1 --2- 172.21.15.90:0/2715597683 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561baf0391d0 0x561baf058e70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7971e33740 1 -- 172.21.15.90:0/2715597683 >> 172.21.15.90:0/2715597683 conn(0x561baf037000 msgr2=0x561baf0509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:23.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.508+0000 7f7952000640 0 client.16248 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:23.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.509+0000 7f7971e33740 1 -- 172.21.15.90:0/2715597683 shutdown_connections 2024-09-10T22:18:23.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.509+0000 7f7971e33740 1 -- 172.21.15.90:0/2715597683 wait complete. 2024-09-10T22:18:23.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.509+0000 7f7971e33740 20 client.16248 trim_cache size 0 max 16384 2024-09-10T22:18:23.512 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:18:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 48, 2024-09-10T22:18:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:18:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 44, 2024-09-10T22:18:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:18:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: "455cd15e": { 2024-09-10T22:18:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 44, 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 142.8, 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:23.513 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:23.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:23.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:23.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:23.514 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:18:23.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 -- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f794c0b5580 msgr2=0x7f794c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:23.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f794c0b5580 0x7f794c0b7a40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x561baf03c350 tx=0x7f7958005f90 comp rx=0 tx=0).stop 2024-09-10T22:18:23.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 -- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7968075de0 msgr2=0x7f796819b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:23.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7968075de0 0x7f796819b710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f795400d950 tx=0x7f795400de10 comp rx=0 tx=0).stop 2024-09-10T22:18:23.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 -- 172.21.15.90:0/818691936 shutdown_connections 2024-09-10T22:18:23.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f794c0b5580 0x7f794c0b7a40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f796810f220 0x7f79681a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7968075de0 0x7f796819b710 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 --2- 172.21.15.90:0/818691936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f79680754c0 0x7f796819b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:23.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 -- 172.21.15.90:0/818691936 >> 172.21.15.90:0/818691936 conn(0x7f79680fe660 msgr2=0x7f7968111370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:23.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.510+0000 7f796dc00640 1 -- 172.21.15.90:0/818691936 shutdown_connections 2024-09-10T22:18:23.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:23.511+0000 7f796dc00640 1 -- 172.21.15.90:0/818691936 wait complete. 2024-09-10T22:18:23.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7081 7.69 MB/sec warmup 74 sec latency 19813.044 ms 2024-09-10T22:18:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:23 smithi171 ceph-mon[34138]: pgmap v871: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-10T22:18:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:23 smithi090 ceph-mon[30350]: pgmap v871: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-10T22:18:24.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:23 smithi154 ceph-mon[34495]: pgmap v871: 129 pgs: 129 active+clean; 466 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 14 op/s 2024-09-10T22:18:24.480 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '455cd15e', total seconds elapsed: 26.9 2024-09-10T22:18:24.480 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T22:18:24.480 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:18:24.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7188 7.67 MB/sec warmup 75 sec latency 1345.647 ms 2024-09-10T22:18:25.049 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:18:25.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.787+0000 7f4e20e00640 1 -- 172.21.15.90:0/2864485747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1c104430 msgr2=0x7f4e1c10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:25.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.787+0000 7f4e20e00640 1 --2- 172.21.15.90:0/2864485747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1c104430 0x7f4e1c10c920 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009900 tx=0x7f4e0802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:18:25.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.787+0000 7f4e20e00640 1 -- 172.21.15.90:0/2864485747 shutdown_connections 2024-09-10T22:18:25.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.787+0000 7f4e20e00640 1 --2- 172.21.15.90:0/2864485747 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e1c10cf70 0x7f4e1c10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:25.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.787+0000 7f4e20e00640 1 --2- 172.21.15.90:0/2864485747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1c104430 0x7f4e1c10c920 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:25.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.787+0000 7f4e20e00640 1 --2- 172.21.15.90:0/2864485747 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e1c103b10 0x7f4e1c103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:25.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.787+0000 7f4e20e00640 1 -- 172.21.15.90:0/2864485747 >> 172.21.15.90:0/2864485747 conn(0x7f4e1c0fd5b0 msgr2=0x7f4e1c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:25.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.789+0000 7f4e20e00640 1 -- 172.21.15.90:0/2864485747 shutdown_connections 2024-09-10T22:18:25.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.789+0000 7f4e20e00640 1 -- 172.21.15.90:0/2864485747 wait complete. 2024-09-10T22:18:25.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.790+0000 7f4e20e00640 1 Processor -- start 2024-09-10T22:18:25.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.790+0000 7f4e20e00640 1 -- start start 2024-09-10T22:18:25.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.791+0000 7f4e20e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e1c103b10 0x7f4e1c111300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:25.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.791+0000 7f4e20e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1c104430 0x7f4e1c111840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:25.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.791+0000 7f4e20e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e1c10cf70 0x7f4e1c111d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:25.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.791+0000 7f4e20e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c102fa0 con 0x7f4e1c104430 2024-09-10T22:18:25.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.791+0000 7f4e20e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c102e20 con 0x7f4e1c103b10 2024-09-10T22:18:25.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.791+0000 7f4e20e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c103120 con 0x7f4e1c10cf70 2024-09-10T22:18:25.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.792+0000 7f4e1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e1c103b10 0x7f4e1c111300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:25.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.792+0000 7f4e1b400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e1c103b10 0x7f4e1c111300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:58034/0 (socket says 172.21.15.90:58034) 2024-09-10T22:18:25.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.792+0000 7f4e1b400640 1 -- 172.21.15.90:0/1112824882 learned_addr learned my addr 172.21.15.90:0/1112824882 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:25.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.792+0000 7f4e1be00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e1c10cf70 0x7f4e1c111d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:25.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.792+0000 7f4e1b400640 1 -- 172.21.15.90:0/1112824882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e1c10cf70 msgr2=0x7f4e1c111d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:25.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.792+0000 7f4e1b400640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e1c10cf70 0x7f4e1c111d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:25.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.792+0000 7f4e1b400640 1 -- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1c104430 msgr2=0x7f4e1c111840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T22:18:25.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.792+0000 7f4e1aa00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1c104430 0x7f4e1c111840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:25.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.793+0000 7f4e1b400640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1c104430 0x7f4e1c111840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:25.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.793+0000 7f4e1b400640 1 -- 172.21.15.90:0/1112824882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c1ab810 con 0x7f4e1c103b10 2024-09-10T22:18:25.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.793+0000 7f4e1be00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e1c10cf70 0x7f4e1c111d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:18:25.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.793+0000 7f4e1b400640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e1c103b10 0x7f4e1c111300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00e9f0 tx=0x7f4e0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:25.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.793+0000 7f4e13e00640 1 -- 172.21.15.90:0/1112824882 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00cd80 con 0x7f4e1c103b10 2024-09-10T22:18:25.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.794+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1c1abb00 con 0x7f4e1c103b10 2024-09-10T22:18:25.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.794+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1c1ac090 con 0x7f4e1c103b10 2024-09-10T22:18:25.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.794+0000 7f4e13e00640 1 -- 172.21.15.90:0/1112824882 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e0c0108c0 con 0x7f4e1c103b10 2024-09-10T22:18:25.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.794+0000 7f4e13e00640 1 -- 172.21.15.90:0/1112824882 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c0187c0 con 0x7f4e1c103b10 2024-09-10T22:18:25.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.796+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dfc0052c0 con 0x7f4e1c103b10 2024-09-10T22:18:25.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.800+0000 7f4e13e00640 1 -- 172.21.15.90:0/1112824882 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4e0c010430 con 0x7f4e1c103b10 2024-09-10T22:18:25.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.801+0000 7f4e13e00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e000b5500 0x7f4e000b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:25.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.801+0000 7f4e13e00640 1 -- 172.21.15.90:0/1112824882 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7f4e0c0e6660 con 0x7f4e1c103b10 2024-09-10T22:18:25.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.802+0000 7f4e1aa00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e000b5500 0x7f4e000b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:25.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.803+0000 7f4e1aa00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e000b5500 0x7f4e000b79c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4e08000c00 tx=0x7f4e08005d50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:25.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:25.807+0000 7f4e13e00640 1 -- 172.21.15.90:0/1112824882 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4e0c0ace10 con 0x7f4e1c103b10 2024-09-10T22:18:25.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7283 7.63 MB/sec warmup 76 sec latency 2345.798 ms 2024-09-10T22:18:26.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:25 smithi090 ceph-mon[30350]: pgmap v872: 129 pgs: 129 active+clean; 470 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 350 KiB/s wr, 26 op/s 2024-09-10T22:18:26.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.194+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4dfc005880 con 0x7f4e1c103b10 2024-09-10T22:18:26.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.195+0000 7f4e13e00640 1 -- 172.21.15.90:0/1112824882 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+12013 (secure 0 0 0) 0x7f4e0c0ac710 con 0x7f4e1c103b10 2024-09-10T22:18:26.198 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:18:26.198 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":48,"btime":"2024-09-10T22:17:37:801370+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:17:37.751219+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:18:26.198 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 48 2024-09-10T22:18:26.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.199+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e000b5500 msgr2=0x7f4e000b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:26.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.199+0000 7f4e20e00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e000b5500 0x7f4e000b79c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4e08000c00 tx=0x7f4e08005d50 comp rx=0 tx=0).stop 2024-09-10T22:18:26.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.199+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e1c103b10 msgr2=0x7f4e1c111300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:26.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.199+0000 7f4e20e00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e1c103b10 0x7f4e1c111300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00e9f0 tx=0x7f4e0c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:18:26.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.200+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 shutdown_connections 2024-09-10T22:18:26.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.200+0000 7f4e20e00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4e000b5500 0x7f4e000b79c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:26.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.200+0000 7f4e20e00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4e1c10cf70 0x7f4e1c111d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:26.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.200+0000 7f4e20e00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4e1c104430 0x7f4e1c111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:26.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.200+0000 7f4e20e00640 1 --2- 172.21.15.90:0/1112824882 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4e1c103b10 0x7f4e1c111300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:26.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.200+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 >> 172.21.15.90:0/1112824882 conn(0x7f4e1c0fd5b0 msgr2=0x7f4e1c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:26.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.200+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 shutdown_connections 2024-09-10T22:18:26.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:26.201+0000 7f4e20e00640 1 -- 172.21.15.90:0/1112824882 wait complete. 2024-09-10T22:18:26.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:25 smithi171 ceph-mon[34138]: pgmap v872: 129 pgs: 129 active+clean; 470 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 350 KiB/s wr, 26 op/s 2024-09-10T22:18:26.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:25 smithi154 ceph-mon[34495]: pgmap v872: 129 pgs: 129 active+clean; 470 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 350 KiB/s wr, 26 op/s 2024-09-10T22:18:26.858 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7393 7.56 MB/sec warmup 77 sec latency 3345.872 ms 2024-09-10T22:18:26.981 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:26 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1112824882' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:27.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:26 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1112824882' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:27.280 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T22:18:27.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T22:18:27.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:26 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1112824882' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:27.859 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7499 7.50 MB/sec warmup 78 sec latency 4274.181 ms 2024-09-10T22:18:27.888 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:18:28.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:27 smithi171 ceph-mon[34138]: pgmap v873: 129 pgs: 129 active+clean; 473 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 670 KiB/s wr, 48 op/s 2024-09-10T22:18:28.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:27 smithi090 ceph-mon[30350]: pgmap v873: 129 pgs: 129 active+clean; 473 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 670 KiB/s wr, 48 op/s 2024-09-10T22:18:28.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:27 smithi154 ceph-mon[34495]: pgmap v873: 129 pgs: 129 active+clean; 473 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 670 KiB/s wr, 48 op/s 2024-09-10T22:18:28.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.666+0000 7fe107200640 1 -- 172.21.15.90:0/1323947453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe108102020 msgr2=0x7fe10810ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:28.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.666+0000 7fe107200640 1 --2- 172.21.15.90:0/1323947453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe108102020 0x7fe10810ed70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009860 tx=0x7fe0f402a3d0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.667+0000 7fe107200640 1 -- 172.21.15.90:0/1323947453 shutdown_connections 2024-09-10T22:18:28.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.667+0000 7fe107200640 1 --2- 172.21.15.90:0/1323947453 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe10810f2b0 0x7fe1081116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.667+0000 7fe107200640 1 --2- 172.21.15.90:0/1323947453 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe108102020 0x7fe10810ed70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.667+0000 7fe107200640 1 --2- 172.21.15.90:0/1323947453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe108101700 0x7fe108101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.667+0000 7fe107200640 1 -- 172.21.15.90:0/1323947453 >> 172.21.15.90:0/1323947453 conn(0x7fe1080fd5b0 msgr2=0x7fe1080ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:28.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.668+0000 7fe107200640 1 -- 172.21.15.90:0/1323947453 shutdown_connections 2024-09-10T22:18:28.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.668+0000 7fe107200640 1 -- 172.21.15.90:0/1323947453 wait complete. 2024-09-10T22:18:28.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.669+0000 7fe107200640 1 Processor -- start 2024-09-10T22:18:28.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.669+0000 7fe107200640 1 -- start start 2024-09-10T22:18:28.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe107200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe108101700 0x7fe10810ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe107200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe108102020 0x7fe10810bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe107200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10810f2b0 0x7fe10810c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe107200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe108114cf0 con 0x7fe108102020 2024-09-10T22:18:28.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe107200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe108114b70 con 0x7fe108101700 2024-09-10T22:18:28.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe107200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe108114e70 con 0x7fe10810f2b0 2024-09-10T22:18:28.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe106800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10810f2b0 0x7fe10810c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe106800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10810f2b0 0x7fe10810c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:48380/0 (socket says 172.21.15.90:48380) 2024-09-10T22:18:28.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe106800640 1 -- 172.21.15.90:0/3593593902 learned_addr learned my addr 172.21.15.90:0/3593593902 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:28.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe105e00640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe108101700 0x7fe10810ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.670+0000 7fe105400640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe108102020 0x7fe10810bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.671+0000 7fe106800640 1 -- 172.21.15.90:0/3593593902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe108101700 msgr2=0x7fe10810ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:28.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.671+0000 7fe106800640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe108101700 0x7fe10810ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.671+0000 7fe106800640 1 -- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe108102020 msgr2=0x7fe10810bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:28.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.671+0000 7fe106800640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe108102020 0x7fe10810bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.671+0000 7fe106800640 1 -- 172.21.15.90:0/3593593902 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10810cae0 con 0x7fe10810f2b0 2024-09-10T22:18:28.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.671+0000 7fe105400640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe108102020 0x7fe10810bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:18:28.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.671+0000 7fe106800640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10810f2b0 0x7fe10810c220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe10000e9d0 tx=0x7fe10000ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:28.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.671+0000 7fe0fea00640 1 -- 172.21.15.90:0/3593593902 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10000cd80 con 0x7fe10810f2b0 2024-09-10T22:18:28.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.672+0000 7fe0fea00640 1 -- 172.21.15.90:0/3593593902 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe100005020 con 0x7fe10810f2b0 2024-09-10T22:18:28.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.672+0000 7fe0fea00640 1 -- 172.21.15.90:0/3593593902 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1000116d0 con 0x7fe10810f2b0 2024-09-10T22:18:28.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.672+0000 7fe107200640 1 -- 172.21.15.90:0/3593593902 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1081abab0 con 0x7fe10810f2b0 2024-09-10T22:18:28.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.672+0000 7fe107200640 1 -- 172.21.15.90:0/3593593902 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1081abf70 con 0x7fe10810f2b0 2024-09-10T22:18:28.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.677+0000 7fe10f434740 1 Processor -- start 2024-09-10T22:18:28.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.677+0000 7fe0fea00640 1 -- 172.21.15.90:0/3593593902 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe10000f040 con 0x7fe10810f2b0 2024-09-10T22:18:28.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.677+0000 7fe10f434740 1 -- start start 2024-09-10T22:18:28.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.678+0000 7fe10f434740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9b4e10 0x564f4b9b3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.678+0000 7fe10f434740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b38b0 0x564f4b9b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.678+0000 7fe10f434740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 0x564f4b9c1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.678+0000 7fe10f434740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564f4b928a90 con 0x564f4b9bf6e0 2024-09-10T22:18:28.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.678+0000 7fe10f434740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564f4b917ec0 con 0x564f4b9b4e10 2024-09-10T22:18:28.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.678+0000 7fe10f434740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564f4b91bcd0 con 0x564f4b9b38b0 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.679+0000 7fe105400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9b4e10 0x564f4b9b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.679+0000 7fe105400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9b4e10 0x564f4b9b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:47152/0 (socket says 172.21.15.90:47152) 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.679+0000 7fe105e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b38b0 0x564f4b9b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe105e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b38b0 0x564f4b9b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:48390/0 (socket says 172.21.15.90:48390) 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe105e00640 1 -- 172.21.15.90:0/1742080369 learned_addr learned my addr 172.21.15.90:0/1742080369 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe0fea00640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b5600 0x7fe0ec0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe105e00640 1 -- 172.21.15.90:0/1742080369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9b4e10 msgr2=0x564f4b9b3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe105e00640 1 --2- 172.21.15.90:0/1742080369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9b4e10 0x564f4b9b3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe105e00640 1 -- 172.21.15.90:0/1742080369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 msgr2=0x564f4b9c1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:28.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe105e00640 1 --2- 172.21.15.90:0/1742080369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 0x564f4b9c1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe105e00640 1 -- 172.21.15.90:0/1742080369 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4b8bd9d0 con 0x564f4b9b38b0 2024-09-10T22:18:28.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe0fea00640 1 -- 172.21.15.90:0/3593593902 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7fe1000e6240 con 0x7fe10810f2b0 2024-09-10T22:18:28.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.680+0000 7fe105400640 1 --2- 172.21.15.90:0/1742080369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9b4e10 0x564f4b9b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:18:28.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.681+0000 7fe106800640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b5600 0x7fe0ec0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.681+0000 7fe105e00640 1 --2- 172.21.15.90:0/1742080369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b38b0 0x564f4b9b3cd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009cc0 tx=0x7fe0f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48dd7d2de7160aa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.682+0000 7fe0f3000640 1 -- 172.21.15.90:0/1742080369 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8005220 con 0x564f4b9b38b0 2024-09-10T22:18:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.682+0000 7fe0f3000640 1 -- 172.21.15.90:0/1742080369 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0f8004c30 con 0x564f4b9b38b0 2024-09-10T22:18:28.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.682+0000 7fe0f3000640 1 -- 172.21.15.90:0/1742080369 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8019650 con 0x564f4b9b38b0 2024-09-10T22:18:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.683+0000 7fe10f434740 1 -- 172.21.15.90:0/1742080369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b38b0 msgr2=0x564f4b9b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.683+0000 7fe10f434740 1 --2- 172.21.15.90:0/1742080369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b38b0 0x564f4b9b3cd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009cc0 tx=0x7fe0f8011810 comp rx=0 tx=0).stop 2024-09-10T22:18:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.683+0000 7fe106800640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b5600 0x7fe0ec0b7ac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe1000a9500 tx=0x7fe100005b00 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.683+0000 7fe10f434740 1 -- 172.21.15.90:0/1742080369 shutdown_connections 2024-09-10T22:18:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.683+0000 7fe10f434740 1 --2- 172.21.15.90:0/1742080369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 0x564f4b9c1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.683+0000 7fe10f434740 1 --2- 172.21.15.90:0/1742080369 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b38b0 0x564f4b9b3cd0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009cc0 tx=0x7fe0f8011810 comp rx=0 tx=0).stop 2024-09-10T22:18:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.683+0000 7fe10f434740 1 --2- 172.21.15.90:0/1742080369 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9b4e10 0x564f4b9b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.683+0000 7fe10f434740 1 -- 172.21.15.90:0/1742080369 >> 172.21.15.90:0/1742080369 conn(0x564f4b97d6f0 msgr2=0x564f4b9b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.684+0000 7fe10f434740 1 -- 172.21.15.90:0/1742080369 shutdown_connections 2024-09-10T22:18:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.684+0000 7fe10f434740 1 -- 172.21.15.90:0/1742080369 wait complete. 2024-09-10T22:18:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.684+0000 7fe10f434740 1 -- start start 2024-09-10T22:18:28.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.685+0000 7fe10f434740 1 Processor -- start 2024-09-10T22:18:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.685+0000 7fe10f434740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b4e10 0x564f4b9bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.685+0000 7fe10f434740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 0x564f4b9bcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.685+0000 7fe10f434740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bd4c0 0x564f4b9dc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.685+0000 7fe10f434740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564f4b7797b0 con 0x564f4b9bf6e0 2024-09-10T22:18:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.685+0000 7fe10f434740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564f4b8bc650 con 0x564f4b9bd4c0 2024-09-10T22:18:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.685+0000 7fe10f434740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564f4b9d3510 con 0x564f4b9b4e10 2024-09-10T22:18:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.685+0000 7fe10f434740 10 client.0 _ll_register_callbacks cb 0x564f4b6230a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:18:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.686+0000 7fe105e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 0x564f4b9bcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.686+0000 7fe105e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 0x564f4b9bcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55264/0 (socket says 172.21.15.90:55264) 2024-09-10T22:18:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.686+0000 7fe105400640 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b4e10 0x564f4b9bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.686+0000 7fe105400640 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bd4c0 0x564f4b9dc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.686+0000 7fe105e00640 1 -- 172.21.15.90:0/4191531935 learned_addr learned my addr 172.21.15.90:0/4191531935 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:28.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.687+0000 7fe105e00640 1 -- 172.21.15.90:0/4191531935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b4e10 msgr2=0x564f4b9bc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:28.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.687+0000 7fe105e00640 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b4e10 0x564f4b9bc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.687+0000 7fe105e00640 1 -- 172.21.15.90:0/4191531935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bd4c0 msgr2=0x564f4b9dc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:28.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.687+0000 7fe105e00640 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bd4c0 0x564f4b9dc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.687+0000 7fe105e00640 1 -- 172.21.15.90:0/4191531935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4b685ba0 con 0x564f4b9bf6e0 2024-09-10T22:18:28.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.687+0000 7fe105400640 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bd4c0 0x564f4b9dc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:18:28.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.688+0000 7fe105e00640 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 0x564f4b9bcee0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8000c00 tx=0x7fe0f8005010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29017a7b9c54773 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:28.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.688+0000 7fe0e7400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:18:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.689+0000 7fe0e7400640 1 -- 172.21.15.90:0/4191531935 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8029070 con 0x564f4b9bf6e0 2024-09-10T22:18:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.689+0000 7fe0e7400640 1 -- 172.21.15.90:0/4191531935 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7fe0f8023630 con 0x564f4b9bf6e0 2024-09-10T22:18:28.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.689+0000 7fe10f434740 1 -- 172.21.15.90:0/4191531935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f8004c30 con 0x564f4b9bf6e0 2024-09-10T22:18:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.690+0000 7fe0e7400640 1 -- 172.21.15.90:0/4191531935 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe0f8023830 con 0x564f4b9bf6e0 2024-09-10T22:18:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.690+0000 7fe10f434740 10 client.16260 fetch_fsmap learned FSMap version 48 2024-09-10T22:18:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.690+0000 7fe10f434740 1 -- 172.21.15.90:0/4191531935 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564f4b9df720 con 0x564f4b9bf6e0 2024-09-10T22:18:28.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.690+0000 7fe0e7400640 1 -- 172.21.15.90:0/4191531935 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0f8004c30 con 0x564f4b9bf6e0 2024-09-10T22:18:28.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.691+0000 7fe0e7400640 1 -- 172.21.15.90:0/4191531935 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 8434+0+0 (secure 0 0 0) 0x7fe0f8019430 con 0x564f4b9bf6e0 2024-09-10T22:18:28.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.691+0000 7fe10f434740 10 client.16260 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T22:18:28.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.691+0000 7fe10f434740 10 client.16260 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:18:28.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.691+0000 7fe10f434740 20 client.16260 populate_metadata read hostname 'smithi090' 2024-09-10T22:18:28.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.691+0000 7fe10f434740 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564f4b9e0ad0 0x564f4b9e2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.691+0000 7fe10f434740 4 client.16260 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:18:28.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.691+0000 7fe10f434740 1 -- 172.21.15.90:0/4191531935 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f4b92a9e0 con 0x564f4b9e0ad0 2024-09-10T22:18:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.692+0000 7fe105400640 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564f4b9e0ad0 0x564f4b9e2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.692+0000 7fe105400640 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564f4b9e0ad0 0x564f4b9e2ec0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=433b18622c907de3 server_cookie=2f371bfe594206cc in_seq=0 out_seq=0 2024-09-10T22:18:28.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.692+0000 7fe0e7400640 10 client.16260 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.694+0000 7fe0e7400640 1 -- 172.21.15.90:0/4191531935 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564f4b92a9e0 con 0x564f4b9e0ad0 2024-09-10T22:18:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.694+0000 7fe0e7400640 10 client.16260 handle_command_reply: tid=0 2024-09-10T22:18:28.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.694+0000 7fe10f434740 1 client.16260 shutdown 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 -- 172.21.15.90:0/4191531935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 msgr2=0x564f4b9bcee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 0x564f4b9bcee0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8000c00 tx=0x7fe0f8005010 comp rx=0 tx=0).stop 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 -- 172.21.15.90:0/4191531935 shutdown_connections 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564f4b9e0ad0 0x564f4b9e2ec0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bd4c0 0x564f4b9dc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9bf6e0 0x564f4b9bcee0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 --2- 172.21.15.90:0/4191531935 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9b4e10 0x564f4b9bc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 -- 172.21.15.90:0/4191531935 >> 172.21.15.90:0/4191531935 conn(0x564f4b97d6f0 msgr2=0x564f4b9b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe0e7400640 0 client.16260 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:28.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 -- 172.21.15.90:0/4191531935 shutdown_connections 2024-09-10T22:18:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 1 -- 172.21.15.90:0/4191531935 wait complete. 2024-09-10T22:18:28.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.695+0000 7fe10f434740 20 client.16260 trim_cache size 0 max 16384 2024-09-10T22:18:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.720+0000 7fe10f434740 1 Processor -- start 2024-09-10T22:18:28.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.720+0000 7fe10f434740 1 -- start start 2024-09-10T22:18:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.720+0000 7fe10f434740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 0x564f4b9dc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe10f434740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bd4c0 0x564f4b9b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe10f434740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bf6e0 0x564f4b9b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe10f434740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081adad0 con 0x564f4b9b4e10 2024-09-10T22:18:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe10f434740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081af850 con 0x564f4b9bf6e0 2024-09-10T22:18:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe10f434740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081ae200 con 0x564f4b9bd4c0 2024-09-10T22:18:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe105400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 0x564f4b9dc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe105e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bd4c0 0x564f4b9b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe105e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bd4c0 0x564f4b9b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:48404/0 (socket says 172.21.15.90:48404) 2024-09-10T22:18:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe105e00640 1 -- 172.21.15.90:0/4133449744 learned_addr learned my addr 172.21.15.90:0/4133449744 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe105e00640 1 -- 172.21.15.90:0/4133449744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bf6e0 msgr2=0x564f4b9b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe105e00640 1 --2- 172.21.15.90:0/4133449744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bf6e0 0x564f4b9b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.721+0000 7fe105e00640 1 -- 172.21.15.90:0/4133449744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 msgr2=0x564f4b9dc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:28.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.722+0000 7fe105e00640 1 --2- 172.21.15.90:0/4133449744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 0x564f4b9dc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.722+0000 7fe105e00640 1 -- 172.21.15.90:0/4133449744 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4b9a28d0 con 0x564f4b9bd4c0 2024-09-10T22:18:28.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.722+0000 7fe105400640 1 --2- 172.21.15.90:0/4133449744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 0x564f4b9dc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:18:28.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.722+0000 7fe105e00640 1 --2- 172.21.15.90:0/4133449744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bd4c0 0x564f4b9b0ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80199c0 tx=0x7fe0f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20657855c9e92dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:28.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.723+0000 7fe0f2600640 1 -- 172.21.15.90:0/4133449744 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f803e070 con 0x564f4b9bd4c0 2024-09-10T22:18:28.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.723+0000 7fe0f2600640 1 -- 172.21.15.90:0/4133449744 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0f8011d10 con 0x564f4b9bd4c0 2024-09-10T22:18:28.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.723+0000 7fe0f2600640 1 -- 172.21.15.90:0/4133449744 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8029050 con 0x564f4b9bd4c0 2024-09-10T22:18:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.724+0000 7fe10f434740 1 -- 172.21.15.90:0/4133449744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bd4c0 msgr2=0x564f4b9b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.724+0000 7fe10f434740 1 --2- 172.21.15.90:0/4133449744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bd4c0 0x564f4b9b0ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80199c0 tx=0x7fe0f8011810 comp rx=0 tx=0).stop 2024-09-10T22:18:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.724+0000 7fe10f434740 1 -- 172.21.15.90:0/4133449744 shutdown_connections 2024-09-10T22:18:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.724+0000 7fe10f434740 1 --2- 172.21.15.90:0/4133449744 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9bf6e0 0x564f4b9b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.724+0000 7fe10f434740 1 --2- 172.21.15.90:0/4133449744 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bd4c0 0x564f4b9b0ab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.724+0000 7fe10f434740 1 --2- 172.21.15.90:0/4133449744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 0x564f4b9dc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.724+0000 7fe10f434740 1 -- 172.21.15.90:0/4133449744 >> 172.21.15.90:0/4133449744 conn(0x564f4b97d6f0 msgr2=0x564f4b9b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.724+0000 7fe10f434740 1 -- 172.21.15.90:0/4133449744 shutdown_connections 2024-09-10T22:18:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.724+0000 7fe10f434740 1 -- 172.21.15.90:0/4133449744 wait complete. 2024-09-10T22:18:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.725+0000 7fe10f434740 1 -- start start 2024-09-10T22:18:28.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.725+0000 7fe10f434740 1 Processor -- start 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.725+0000 7fe10f434740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 0x564f4b9d87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.725+0000 7fe10f434740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bf6e0 0x564f4b9d8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.725+0000 7fe10f434740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9d9330 0x564f4b9b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.725+0000 7fe10f434740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081af0c0 con 0x564f4b9b4e10 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.725+0000 7fe10f434740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe1081ae990 con 0x564f4b9d9330 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.725+0000 7fe10f434740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe10810cfe0 con 0x564f4b9bf6e0 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.725+0000 7fe10f434740 10 client.0 _ll_register_callbacks cb 0x7fe0f8009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.726+0000 7fe105e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bf6e0 0x564f4b9d8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.726+0000 7fe105e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bf6e0 0x564f4b9d8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:48418/0 (socket says 172.21.15.90:48418) 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.726+0000 7fe105e00640 1 -- 172.21.15.90:0/1784086879 learned_addr learned my addr 172.21.15.90:0/1784086879 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.726+0000 7fe105400640 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9d9330 0x564f4b9b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.727+0000 7fe105400640 1 -- 172.21.15.90:0/1784086879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bf6e0 msgr2=0x564f4b9d8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:28.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.727+0000 7fe105400640 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bf6e0 0x564f4b9d8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.727+0000 7fe105400640 1 -- 172.21.15.90:0/1784086879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 msgr2=0x564f4b9d87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.727+0000 7fe105400640 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 0x564f4b9d87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.727+0000 7fe105400640 1 -- 172.21.15.90:0/1784086879 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4b9b9470 con 0x564f4b9d9330 2024-09-10T22:18:28.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.727+0000 7fe105e00640 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bf6e0 0x564f4b9d8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.727+0000 7fe105400640 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9d9330 0x564f4b9b8c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009830 tx=0x7fe0f403e5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e892d8d85f94336 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.727+0000 7fe0e7400640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.728+0000 7fe0e7400640 1 -- 172.21.15.90:0/1784086879 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4056060 con 0x564f4b9d9330 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.728+0000 7fe0e7400640 1 -- 172.21.15.90:0/1784086879 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7fe0f403ec10 con 0x564f4b9d9330 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.728+0000 7fe10f434740 1 -- 172.21.15.90:0/1784086879 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0f8011d10 con 0x564f4b9d9330 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.728+0000 7fe0e7400640 1 -- 172.21.15.90:0/1784086879 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fe0f403ee10 con 0x564f4b9d9330 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe10f434740 10 client.35897 fetch_fsmap learned FSMap version 48 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe10f434740 1 -- 172.21.15.90:0/1784086879 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x564f4b9ba7b0 con 0x564f4b9d9330 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe0e7400640 1 -- 172.21.15.90:0/1784086879 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe0f8011d10 con 0x564f4b9d9330 2024-09-10T22:18:28.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe0e7400640 1 -- 172.21.15.90:0/1784086879 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 48) ==== 8434+0+0 (secure 0 0 0) 0x7fe0f40514d0 con 0x564f4b9d9330 2024-09-10T22:18:28.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe10f434740 10 client.35897 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T22:18:28.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe10f434740 10 client.35897 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:18:28.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe10f434740 20 client.35897 populate_metadata read hostname 'smithi090' 2024-09-10T22:18:28.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe10f434740 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564f4ba4ec50 0x564f4ba51070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:28.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe10f434740 4 client.35897 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T22:18:28.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.729+0000 7fe10f434740 1 -- 172.21.15.90:0/1784086879 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x564f4b926490 con 0x564f4ba4ec50 2024-09-10T22:18:28.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.730+0000 7fe105e00640 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564f4ba4ec50 0x564f4ba51070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:28.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.731+0000 7fe0e7400640 10 client.35897 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:28.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:28.731+0000 7fe105e00640 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564f4ba4ec50 0x564f4ba51070 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46de9b0565adef62 server_cookie=43d7fb1d21efb992 in_seq=0 out_seq=0 2024-09-10T22:18:28.859 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7591 7.44 MB/sec warmup 79 sec latency 176.599 ms 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe0e7400640 1 -- 172.21.15.90:0/1784086879 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564f4b926490 con 0x564f4ba4ec50 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe0e7400640 10 client.35897 handle_command_reply: tid=0 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe10f434740 1 client.35897 shutdown 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe10f434740 1 -- 172.21.15.90:0/1784086879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9d9330 msgr2=0x564f4b9b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe10f434740 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9d9330 0x564f4b9b8c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009830 tx=0x7fe0f403e5e0 comp rx=0 tx=0).stop 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe10f434740 1 -- 172.21.15.90:0/1784086879 shutdown_connections 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe10f434740 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564f4ba4ec50 0x564f4ba51070 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe10f434740 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564f4b9d9330 0x564f4b9b8c40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe10f434740 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564f4b9bf6e0 0x564f4b9d8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe10f434740 1 --2- 172.21.15.90:0/1784086879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564f4b9b4e10 0x564f4b9d87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe0e7400640 0 client.35897 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.075+0000 7fe10f434740 1 -- 172.21.15.90:0/1784086879 >> 172.21.15.90:0/1784086879 conn(0x564f4b97d6f0 msgr2=0x564f4b9b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.076+0000 7fe10f434740 1 -- 172.21.15.90:0/1784086879 shutdown_connections 2024-09-10T22:18:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.076+0000 7fe10f434740 1 -- 172.21.15.90:0/1784086879 wait complete. 2024-09-10T22:18:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.076+0000 7fe10f434740 20 client.35897 trim_cache size 0 max 16384 2024-09-10T22:18:29.079 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:18:29.079 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 48, 2024-09-10T22:18:29.079 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:18:29.079 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 46, 2024-09-10T22:18:29.079 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "a2ed6f05": { 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 46, 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 142.8, 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:18:29.080 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:18:29.081 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:18:29.081 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:18:29.081 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:29.081 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:29.081 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:29.081 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:29.081 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:29.081 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:18:29.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 -- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b5600 msgr2=0x7fe0ec0b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b5600 0x7fe0ec0b7ac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe1000a9500 tx=0x7fe100005b00 comp rx=0 tx=0).stop 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 -- 172.21.15.90:0/3593593902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10810f2b0 msgr2=0x7fe10810c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10810f2b0 0x7fe10810c220 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe10000e9d0 tx=0x7fe10000ee90 comp rx=0 tx=0).stop 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 -- 172.21.15.90:0/3593593902 shutdown_connections 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe0ec0b5600 0x7fe0ec0b7ac0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe10810f2b0 0x7fe10810c220 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe108102020 0x7fe10810bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 --2- 172.21.15.90:0/3593593902 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe108101700 0x7fe10810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 -- 172.21.15.90:0/3593593902 >> 172.21.15.90:0/3593593902 conn(0x7fe1080fd5b0 msgr2=0x7fe1080ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 -- 172.21.15.90:0/3593593902 shutdown_connections 2024-09-10T22:18:29.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:29.077+0000 7fe107200640 1 -- 172.21.15.90:0/3593593902 wait complete. 2024-09-10T22:18:29.859 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 7.35 MB/sec warmup 80 sec latency 1125.266 ms 2024-09-10T22:18:30.047 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a2ed6f05', quiesce took 5.6 seconds. Will release after: 17.15 2024-09-10T22:18:30.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:29 smithi171 ceph-mon[34138]: pgmap v874: 129 pgs: 129 active+clean; 482 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-09-10T22:18:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:29 smithi090 ceph-mon[30350]: pgmap v874: 129 pgs: 129 active+clean; 482 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-09-10T22:18:30.421 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:29 smithi154 ceph-mon[34495]: pgmap v874: 129 pgs: 129 active+clean; 482 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-09-10T22:18:30.859 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 7.26 MB/sec warmup 81 sec latency 2125.359 ms 2024-09-10T22:18:31.859 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 7.17 MB/sec warmup 82 sec latency 3125.458 ms 2024-09-10T22:18:32.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:31 smithi171 ceph-mon[34138]: pgmap v875: 129 pgs: 129 active+clean; 483 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-09-10T22:18:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:31 smithi090 ceph-mon[30350]: pgmap v875: 129 pgs: 129 active+clean; 483 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-09-10T22:18:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:31 smithi154 ceph-mon[34495]: pgmap v875: 129 pgs: 129 active+clean; 483 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 1.5 MiB/s wr, 110 op/s 2024-09-10T22:18:32.859 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 7.08 MB/sec warmup 83 sec latency 4125.584 ms 2024-09-10T22:18:33.859 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 7.00 MB/sec warmup 84 sec latency 5125.683 ms 2024-09-10T22:18:34.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:33 smithi171 ceph-mon[34138]: pgmap v876: 129 pgs: 129 active+clean; 481 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-10T22:18:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:33 smithi090 ceph-mon[30350]: pgmap v876: 129 pgs: 129 active+clean; 481 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-10T22:18:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:33 smithi154 ceph-mon[34495]: pgmap v876: 129 pgs: 129 active+clean; 481 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-09-10T22:18:34.859 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.92 MB/sec warmup 85 sec latency 6125.736 ms 2024-09-10T22:18:35.860 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.84 MB/sec warmup 86 sec latency 7125.854 ms 2024-09-10T22:18:36.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:35 smithi171 ceph-mon[34138]: pgmap v877: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-10T22:18:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:35 smithi090 ceph-mon[30350]: pgmap v877: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-10T22:18:36.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:35 smithi154 ceph-mon[34495]: pgmap v877: 129 pgs: 129 active+clean; 482 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-10T22:18:36.860 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.76 MB/sec warmup 87 sec latency 8126.003 ms 2024-09-10T22:18:37.860 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.68 MB/sec warmup 88 sec latency 9126.154 ms 2024-09-10T22:18:38.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:37 smithi171 ceph-mon[34138]: pgmap v878: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-09-10T22:18:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:37 smithi090 ceph-mon[30350]: pgmap v878: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-09-10T22:18:38.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:37 smithi154 ceph-mon[34495]: pgmap v878: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.4 MiB/s wr, 120 op/s 2024-09-10T22:18:38.860 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.61 MB/sec warmup 89 sec latency 10126.257 ms 2024-09-10T22:18:39.860 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.53 MB/sec warmup 90 sec latency 11126.436 ms 2024-09-10T22:18:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:39 smithi171 ceph-mon[34138]: pgmap v879: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-10T22:18:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:39 smithi090 ceph-mon[30350]: pgmap v879: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-10T22:18:40.421 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:39 smithi154 ceph-mon[34495]: pgmap v879: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-10T22:18:40.860 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.46 MB/sec warmup 91 sec latency 12126.528 ms 2024-09-10T22:18:41.860 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.39 MB/sec warmup 92 sec latency 13126.619 ms 2024-09-10T22:18:42.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:41 smithi171 ceph-mon[34138]: pgmap v880: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 259 KiB/s wr, 42 op/s 2024-09-10T22:18:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:41 smithi090 ceph-mon[30350]: pgmap v880: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 259 KiB/s wr, 42 op/s 2024-09-10T22:18:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:41 smithi154 ceph-mon[34495]: pgmap v880: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 259 KiB/s wr, 42 op/s 2024-09-10T22:18:42.860 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.32 MB/sec warmup 93 sec latency 14126.730 ms 2024-09-10T22:18:43.861 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.25 MB/sec warmup 94 sec latency 15126.873 ms 2024-09-10T22:18:44.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:43 smithi171 ceph-mon[34138]: pgmap v881: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 206 KiB/s wr, 34 op/s 2024-09-10T22:18:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:43 smithi090 ceph-mon[30350]: pgmap v881: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 206 KiB/s wr, 34 op/s 2024-09-10T22:18:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:43 smithi154 ceph-mon[34495]: pgmap v881: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 206 KiB/s wr, 34 op/s 2024-09-10T22:18:44.861 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.19 MB/sec warmup 95 sec latency 16127.018 ms 2024-09-10T22:18:45.861 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.12 MB/sec warmup 96 sec latency 17127.155 ms 2024-09-10T22:18:46.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:45 smithi171 ceph-mon[34138]: pgmap v882: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 151 KiB/s wr, 23 op/s 2024-09-10T22:18:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:45 smithi090 ceph-mon[30350]: pgmap v882: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 151 KiB/s wr, 23 op/s 2024-09-10T22:18:46.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:45 smithi154 ceph-mon[34495]: pgmap v882: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 151 KiB/s wr, 23 op/s 2024-09-10T22:18:46.861 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.06 MB/sec warmup 97 sec latency 18127.211 ms 2024-09-10T22:18:47.199 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:18:47.583 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:18:47.861 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 6.00 MB/sec warmup 98 sec latency 19127.360 ms 2024-09-10T22:18:48.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.129+0000 7f01e3e00640 1 -- 172.21.15.90:0/2876497933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e4108fa0 msgr2=0x7f01e410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:48.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.129+0000 7f01e3e00640 1 --2- 172.21.15.90:0/2876497933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e4108fa0 0x7f01e410f7a0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009ca0 tx=0x7f01dc02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.129+0000 7f01e3e00640 1 -- 172.21.15.90:0/2876497933 shutdown_connections 2024-09-10T22:18:48.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.129+0000 7f01e3e00640 1 --2- 172.21.15.90:0/2876497933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e4108fa0 0x7f01e410f7a0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.129+0000 7f01e3e00640 1 --2- 172.21.15.90:0/2876497933 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01e4108620 0x7f01e4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.129+0000 7f01e3e00640 1 --2- 172.21.15.90:0/2876497933 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01e4102c10 0x7f01e4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.129+0000 7f01e3e00640 1 -- 172.21.15.90:0/2876497933 >> 172.21.15.90:0/2876497933 conn(0x7f01e40fe660 msgr2=0x7f01e4100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:48.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.130+0000 7f01e3e00640 1 -- 172.21.15.90:0/2876497933 shutdown_connections 2024-09-10T22:18:48.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.130+0000 7f01e3e00640 1 -- 172.21.15.90:0/2876497933 wait complete. 2024-09-10T22:18:48.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.131+0000 7f01e3e00640 1 Processor -- start 2024-09-10T22:18:48.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.131+0000 7f01e3e00640 1 -- start start 2024-09-10T22:18:48.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.131+0000 7f01e3e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01e4102c10 0x7f01e419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:48.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.131+0000 7f01e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e4108620 0x7f01e419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:48.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.131+0000 7f01e3e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01e4108fa0 0x7f01e41a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:48.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.131+0000 7f01e3e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4112b20 con 0x7f01e4108620 2024-09-10T22:18:48.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.131+0000 7f01e3e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f01e41129a0 con 0x7f01e4108fa0 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.131+0000 7f01e3e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f01e4112ca0 con 0x7f01e4102c10 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01e4102c10 0x7f01e419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e4108620 0x7f01e419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01e4102c10 0x7f01e419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:55010/0 (socket says 172.21.15.90:55010) 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2a00640 1 -- 172.21.15.90:0/410161089 learned_addr learned my addr 172.21.15.90:0/410161089 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e3400640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01e4108fa0 0x7f01e41a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2000640 1 -- 172.21.15.90:0/410161089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01e4102c10 msgr2=0x7f01e419b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2000640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01e4102c10 0x7f01e419b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2000640 1 -- 172.21.15.90:0/410161089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01e4108fa0 msgr2=0x7f01e41a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2000640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01e4108fa0 0x7f01e41a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2000640 1 -- 172.21.15.90:0/410161089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e41a1240 con 0x7f01e4108620 2024-09-10T22:18:48.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.132+0000 7f01e2000640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e4108620 0x7f01e419b6d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01d000e9f0 tx=0x7f01d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:48.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.133+0000 7f01e8a00640 1 -- 172.21.15.90:0/410161089 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d000ccc0 con 0x7f01e4108620 2024-09-10T22:18:48.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.133+0000 7f01e8a00640 1 -- 172.21.15.90:0/410161089 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f01d0005020 con 0x7f01e4108620 2024-09-10T22:18:48.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.133+0000 7f01e8a00640 1 -- 172.21.15.90:0/410161089 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d000ce60 con 0x7f01e4108620 2024-09-10T22:18:48.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.133+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e41a1530 con 0x7f01e4108620 2024-09-10T22:18:48.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.133+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e41a7bf0 con 0x7f01e4108620 2024-09-10T22:18:48.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.134+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e4102ff0 con 0x7f01e4108620 2024-09-10T22:18:48.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.137+0000 7f01e8a00640 1 -- 172.21.15.90:0/410161089 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f01d0004a60 con 0x7f01e4108620 2024-09-10T22:18:48.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.138+0000 7f01e8a00640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01c80b5580 0x7f01c80b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:48.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.138+0000 7f01e8a00640 1 -- 172.21.15.90:0/410161089 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7f01d00e6290 con 0x7f01e4108620 2024-09-10T22:18:48.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.139+0000 7f01e2a00640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01c80b5580 0x7f01c80b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:48.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.139+0000 7f01e2a00640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01c80b5580 0x7f01c80b7a40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009c90 tx=0x7f01d4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:48.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.142+0000 7f01e8a00640 1 -- 172.21.15.90:0/410161089 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f01d00ac9e0 con 0x7f01e4108620 2024-09-10T22:18:48.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:47 smithi171 ceph-mon[34138]: pgmap v883: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 111 KiB/s wr, 12 op/s 2024-09-10T22:18:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:47 smithi090 ceph-mon[30350]: pgmap v883: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 111 KiB/s wr, 12 op/s 2024-09-10T22:18:48.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.363+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01e402e5c0 con 0x7f01e4108620 2024-09-10T22:18:48.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.364+0000 7f01e8a00640 1 -- 172.21.15.90:0/410161089 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+12013 (secure 0 0 0) 0x7f01d00ac2e0 con 0x7f01e4108620 2024-09-10T22:18:48.365 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:18:48.365 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":48,"btime":"2024-09-10T22:17:37:801370+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:17:37.751219+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:18:48.365 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 48 2024-09-10T22:18:48.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01c80b5580 msgr2=0x7f01c80b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01c80b5580 0x7f01c80b7a40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009c90 tx=0x7f01d4005f90 comp rx=0 tx=0).stop 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e4108620 msgr2=0x7f01e419b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e4108620 0x7f01e419b6d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01d000e9f0 tx=0x7f01d000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 shutdown_connections 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f01c80b5580 0x7f01c80b7a40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f01e4108fa0 0x7f01e41a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01e4108620 0x7f01e419b6d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 --2- 172.21.15.90:0/410161089 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f01e4102c10 0x7f01e419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.366+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 >> 172.21.15.90:0/410161089 conn(0x7f01e40fe660 msgr2=0x7f01e4106c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.367+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 shutdown_connections 2024-09-10T22:18:48.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:48.367+0000 7f01e3e00640 1 -- 172.21.15.90:0/410161089 wait complete. 2024-09-10T22:18:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:47 smithi154 ceph-mon[34495]: pgmap v883: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 111 KiB/s wr, 12 op/s 2024-09-10T22:18:48.861 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 5.94 MB/sec warmup 99 sec latency 20127.498 ms 2024-09-10T22:18:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:48 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/410161089' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:48 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/410161089' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:49.316 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id a2ed6f05 --release --await' 2024-09-10T22:18:49.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id a2ed6f05 --release --await 2024-09-10T22:18:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:48 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/410161089' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:18:49.684 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:18:49.861 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7597 5.88 MB/sec warmup 100 sec latency 21127.571 ms 2024-09-10T22:18:50.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:49 smithi090 ceph-mon[30350]: pgmap v884: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-10T22:18:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:49 smithi171 ceph-mon[34138]: pgmap v884: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-10T22:18:50.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:49 smithi154 ceph-mon[34495]: pgmap v884: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-09-10T22:18:50.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.313+0000 7f0e11a00640 1 -- 172.21.15.90:0/2302690655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 msgr2=0x7f0e0c1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:50.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.313+0000 7f0e11a00640 1 --2- 172.21.15.90:0/2302690655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 0x7f0e0c1051d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009900 tx=0x7f0dfc02a410 comp rx=0 tx=0).stop 2024-09-10T22:18:50.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.313+0000 7f0e11a00640 1 -- 172.21.15.90:0/2302690655 shutdown_connections 2024-09-10T22:18:50.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.313+0000 7f0e11a00640 1 --2- 172.21.15.90:0/2302690655 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e0c10b180 0x7f0e0c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.313+0000 7f0e11a00640 1 --2- 172.21.15.90:0/2302690655 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e0c10a800 0x7f0e0c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.313+0000 7f0e11a00640 1 --2- 172.21.15.90:0/2302690655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 0x7f0e0c1051d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.313+0000 7f0e11a00640 1 -- 172.21.15.90:0/2302690655 >> 172.21.15.90:0/2302690655 conn(0x7f0e0c100840 msgr2=0x7f0e0c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.313+0000 7f0e11a00640 1 -- 172.21.15.90:0/2302690655 shutdown_connections 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.313+0000 7f0e11a00640 1 -- 172.21.15.90:0/2302690655 wait complete. 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.314+0000 7f0e11a00640 1 Processor -- start 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.314+0000 7f0e11a00640 1 -- start start 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.314+0000 7f0e11a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 0x7f0e0c19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.314+0000 7f0e11a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e0c10a800 0x7f0e0c19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.314+0000 7f0e11a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e0c10b180 0x7f0e0c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.314+0000 7f0e11a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c114d40 con 0x7f0e0c104df0 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.314+0000 7f0e11a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c114bc0 con 0x7f0e0c10b180 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.314+0000 7f0e11a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c114ec0 con 0x7f0e0c10a800 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 0x7f0e0c19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 0x7f0e0c19b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41640/0 (socket says 172.21.15.90:41640) 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e0be00640 1 -- 172.21.15.90:0/4241513335 learned_addr learned my addr 172.21.15.90:0/4241513335 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e11000640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e0c10b180 0x7f0e0c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e03400640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e0c10a800 0x7f0e0c19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e0be00640 1 -- 172.21.15.90:0/4241513335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e0c10a800 msgr2=0x7f0e0c19b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e0be00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e0c10a800 0x7f0e0c19b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e0be00640 1 -- 172.21.15.90:0/4241513335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e0c10b180 msgr2=0x7f0e0c1a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e0be00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e0c10b180 0x7f0e0c1a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e0be00640 1 -- 172.21.15.90:0/4241513335 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c1a11e0 con 0x7f0e0c104df0 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.315+0000 7f0e0be00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 0x7f0e0c19b130 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc02a930 tx=0x7f0dfc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.316+0000 7f0e09600640 1 -- 172.21.15.90:0/4241513335 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc03f070 con 0x7f0e0c104df0 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.316+0000 7f0e09600640 1 -- 172.21.15.90:0/4241513335 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0dfc003e40 con 0x7f0e0c104df0 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.316+0000 7f0e11a00640 1 -- 172.21.15.90:0/4241513335 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c1a1470 con 0x7f0e0c104df0 2024-09-10T22:18:50.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.316+0000 7f0e09600640 1 -- 172.21.15.90:0/4241513335 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc035030 con 0x7f0e0c104df0 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.316+0000 7f0e11a00640 1 -- 172.21.15.90:0/4241513335 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c1a7b90 con 0x7f0e0c104df0 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.319+0000 7f0e15a7c740 1 Processor -- start 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.319+0000 7f0e15a7c740 1 -- start start 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.319+0000 7f0e09600640 1 -- 172.21.15.90:0/4241513335 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0dfc031400 con 0x7f0e0c104df0 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.319+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce53521d0 0x563ce53525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.319+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce5352af0 0x563ce535e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.319+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 0x563ce5360e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.319+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563ce52c9290 con 0x563ce53521d0 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.319+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563ce52b86c0 con 0x563ce535ea80 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.319+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563ce52bc4d0 con 0x563ce5352af0 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e11000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce53521d0 0x563ce53525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e11000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce53521d0 0x563ce53525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41648/0 (socket says 172.21.15.90:41648) 2024-09-10T22:18:50.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e11000640 1 -- 172.21.15.90:0/245421190 learned_addr learned my addr 172.21.15.90:0/245421190 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e0be00640 1 --2- 172.21.15.90:0/245421190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce5352af0 0x563ce535e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e03400640 1 --2- 172.21.15.90:0/245421190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 0x563ce5360e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e11000640 1 -- 172.21.15.90:0/245421190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce5352af0 msgr2=0x563ce535e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e11000640 1 --2- 172.21.15.90:0/245421190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce5352af0 0x563ce535e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e11000640 1 -- 172.21.15.90:0/245421190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 msgr2=0x563ce5360e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e11000640 1 --2- 172.21.15.90:0/245421190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 0x563ce5360e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.320+0000 7f0e11000640 1 -- 172.21.15.90:0/245421190 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce5245460 con 0x563ce53521d0 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.321+0000 7f0e11000640 1 --2- 172.21.15.90:0/245421190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce53521d0 0x563ce53525b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0e04003a20 tx=0x7f0e04003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6ade779cbd1b0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.321+0000 7f0e01600640 1 -- 172.21.15.90:0/245421190 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0401a470 con 0x563ce53521d0 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.321+0000 7f0e01600640 1 -- 172.21.15.90:0/245421190 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e04015070 con 0x563ce53521d0 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.321+0000 7f0e01600640 1 -- 172.21.15.90:0/245421190 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04011880 con 0x563ce53521d0 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.321+0000 7f0e09600640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0df00b5580 0x7f0df00b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e0be00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0df00b5580 0x7f0df00b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e09600640 1 -- 172.21.15.90:0/4241513335 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7f0dfc106db0 con 0x7f0e0c104df0 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e15a7c740 1 -- 172.21.15.90:0/245421190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce53521d0 msgr2=0x563ce53525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/245421190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce53521d0 0x563ce53525b0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0e04003a20 tx=0x7f0e04003ee0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e15a7c740 1 -- 172.21.15.90:0/245421190 shutdown_connections 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/245421190 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 0x563ce5360e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/245421190 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce5352af0 0x563ce535e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/245421190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce53521d0 0x563ce53525b0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e15a7c740 1 -- 172.21.15.90:0/245421190 >> 172.21.15.90:0/245421190 conn(0x563ce5350000 msgr2=0x563ce53511a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e15a7c740 1 -- 172.21.15.90:0/245421190 shutdown_connections 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e15a7c740 1 -- 172.21.15.90:0/245421190 wait complete. 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.322+0000 7f0e0be00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0df00b5580 0x7f0df00b7a40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x563ce5355350 tx=0x7f0dfc10a860 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.323+0000 7f0e15a7c740 1 -- start start 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.323+0000 7f0e15a7c740 1 Processor -- start 2024-09-10T22:18:50.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.323+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 0x563ce535ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.323+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce5352af0 0x563ce535bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.323+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce535ea80 0x563ce535c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.323+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563ce5119bb0 con 0x563ce535ea80 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.323+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x563ce52536c0 con 0x563ce5352af0 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.323+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x563ce5361730 con 0x563ce53521d0 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.323+0000 7f0e15a7c740 10 client.0 _ll_register_callbacks cb 0x563ce526e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e11000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce535ea80 0x563ce535c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e03400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce5352af0 0x563ce535bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e11000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce535ea80 0x563ce535c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41664/0 (socket says 172.21.15.90:41664) 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e11000640 1 -- 172.21.15.90:0/3313547018 learned_addr learned my addr 172.21.15.90:0/3313547018 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e11000640 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 0x563ce535ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e03400640 1 -- 172.21.15.90:0/3313547018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 msgr2=0x563ce535ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e03400640 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 0x563ce535ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e03400640 1 -- 172.21.15.90:0/3313547018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce535ea80 msgr2=0x563ce535c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e03400640 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce535ea80 0x563ce535c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.324+0000 7f0e03400640 1 -- 172.21.15.90:0/3313547018 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce5016b00 con 0x563ce5352af0 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.325+0000 7f0e03400640 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce5352af0 0x563ce535bf40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c103e10 tx=0x7f0df400bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5df2dffa70fc1b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.325+0000 7f0dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.325+0000 7f0dfa000640 1 -- 172.21.15.90:0/3313547018 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4003df0 con 0x563ce5352af0 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.325+0000 7f0e15a7c740 1 -- 172.21.15.90:0/3313547018 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e04015070 con 0x563ce5352af0 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.325+0000 7f0dfa000640 1 -- 172.21.15.90:0/3313547018 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7f0df4005020 con 0x563ce5352af0 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.325+0000 7f0dfa000640 1 -- 172.21.15.90:0/3313547018 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0df4024050 con 0x563ce5352af0 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.325+0000 7f0e15a7c740 10 client.35915 fetch_fsmap learned FSMap version 48 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.325+0000 7f0e15a7c740 1 -- 172.21.15.90:0/3313547018 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563ce508d960 con 0x563ce5352af0 2024-09-10T22:18:50.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.325+0000 7f0dfa000640 1 -- 172.21.15.90:0/3313547018 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0e04015070 con 0x563ce5352af0 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.326+0000 7f0dfa000640 1 -- 172.21.15.90:0/3313547018 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 48) ==== 8434+0+0 (secure 0 0 0) 0x7f0df4004cd0 con 0x563ce5352af0 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.326+0000 7f0e15a7c740 10 client.35915 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.326+0000 7f0e15a7c740 10 client.35915 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.326+0000 7f0e15a7c740 20 client.35915 populate_metadata read hostname 'smithi090' 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.326+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x563ce5376f30 0x563ce5379350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.326+0000 7f0e15a7c740 4 client.35915 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.326+0000 7f0e15a7c740 1 -- 172.21.15.90:0/3313547018 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ce52cb1e0 con 0x563ce5376f30 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.326+0000 7f0e11000640 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x563ce5376f30 0x563ce5379350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.327+0000 7f0e11000640 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x563ce5376f30 0x563ce5379350 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645f37d73d1ad859 server_cookie=f53907fd9196d573 in_seq=0 out_seq=0 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.327+0000 7f0dfa000640 10 client.35915 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0dfa000640 1 -- 172.21.15.90:0/3313547018 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ce52cb1e0 con 0x563ce5376f30 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0dfa000640 10 client.35915 handle_command_reply: tid=0 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 client.35915 shutdown 2024-09-10T22:18:50.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 -- 172.21.15.90:0/3313547018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce5352af0 msgr2=0x563ce535bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce5352af0 0x563ce535bf40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c103e10 tx=0x7f0df400bd80 comp rx=0 tx=0).stop 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 -- 172.21.15.90:0/3313547018 shutdown_connections 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x563ce5376f30 0x563ce5379350 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce535ea80 0x563ce535c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce5352af0 0x563ce535bf40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/3313547018 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 0x563ce535ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0dfa000640 0 client.35915 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 -- 172.21.15.90:0/3313547018 >> 172.21.15.90:0/3313547018 conn(0x563ce5350000 msgr2=0x563ce5355ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 -- 172.21.15.90:0/3313547018 shutdown_connections 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 1 -- 172.21.15.90:0/3313547018 wait complete. 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.328+0000 7f0e15a7c740 20 client.35915 trim_cache size 0 max 16384 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.346+0000 7f0e15a7c740 1 Processor -- start 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.346+0000 7f0e15a7c740 1 -- start start 2024-09-10T22:18:50.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 0x563ce5372920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce5358c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 0x563ce5359150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c1ad9a0 con 0x563ce5352af0 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c1af720 con 0x563ce535ea80 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c1ae0d0 con 0x563ce53521d0 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e03400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce5358c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e03400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce5358c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41668/0 (socket says 172.21.15.90:41668) 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e03400640 1 -- 172.21.15.90:0/213543600 learned_addr learned my addr 172.21.15.90:0/213543600 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e11000640 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 0x563ce5359150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e11000640 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 0x563ce5372920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e03400640 1 -- 172.21.15.90:0/213543600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 msgr2=0x563ce5372920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e03400640 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 0x563ce5372920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e03400640 1 -- 172.21.15.90:0/213543600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 msgr2=0x563ce5359150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e03400640 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 0x563ce5359150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e03400640 1 -- 172.21.15.90:0/213543600 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce5350a80 con 0x563ce5352af0 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.347+0000 7f0e11000640 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 0x563ce5372920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.348+0000 7f0e03400640 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce5358c10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0df401b680 tx=0x7f0df4003a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ca081e108e6a11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.348+0000 7f0e00c00640 1 -- 172.21.15.90:0/213543600 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df402c070 con 0x563ce5352af0 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.348+0000 7f0e00c00640 1 -- 172.21.15.90:0/213543600 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0df4003df0 con 0x563ce5352af0 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.348+0000 7f0e00c00640 1 -- 172.21.15.90:0/213543600 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4011440 con 0x563ce5352af0 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.348+0000 7f0e15a7c740 1 -- 172.21.15.90:0/213543600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 msgr2=0x563ce5358c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce5358c10 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0df401b680 tx=0x7f0df4003a40 comp rx=0 tx=0).stop 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 -- 172.21.15.90:0/213543600 shutdown_connections 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce535ea80 0x563ce5359150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce5358c10 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/213543600 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce53521d0 0x563ce5372920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 -- 172.21.15.90:0/213543600 >> 172.21.15.90:0/213543600 conn(0x563ce5350000 msgr2=0x563ce5372210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 -- 172.21.15.90:0/213543600 shutdown_connections 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 -- 172.21.15.90:0/213543600 wait complete. 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 -- start start 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.349+0000 7f0e15a7c740 1 Processor -- start 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce53521d0 0x563ce536fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce536c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e15a7c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce535ea80 0x563ce536c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c1aef90 con 0x563ce5352af0 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c1ae860 con 0x563ce53521d0 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e15a7c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c1ab9a0 con 0x563ce535ea80 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e15a7c740 10 client.0 _ll_register_callbacks cb 0x7f0df40042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e03400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce536c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e03400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce536c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41680/0 (socket says 172.21.15.90:41680) 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e03400640 1 -- 172.21.15.90:0/363948343 learned_addr learned my addr 172.21.15.90:0/363948343 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.350+0000 7f0e11000640 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce535ea80 0x563ce536c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e11000640 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce53521d0 0x563ce536fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e03400640 1 -- 172.21.15.90:0/363948343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce535ea80 msgr2=0x563ce536c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e03400640 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce535ea80 0x563ce536c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e03400640 1 -- 172.21.15.90:0/363948343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce53521d0 msgr2=0x563ce536fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e03400640 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce53521d0 0x563ce536fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e03400640 1 -- 172.21.15.90:0/363948343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce53e3e30 con 0x563ce5352af0 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e11000640 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce53521d0 0x563ce536fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e03400640 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce536c000 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0df401bea0 tx=0x7f0df400b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9899a841bd86d127 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0dfa000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0dfa000640 1 -- 172.21.15.90:0/363948343 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df4035020 con 0x563ce5352af0 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0dfa000640 1 -- 172.21.15.90:0/363948343 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9490+0+0 (secure 0 0 0) 0x7f0df4017040 con 0x563ce5352af0 2024-09-10T22:18:50.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e15a7c740 1 -- 172.21.15.90:0/363948343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0df4003df0 con 0x563ce5352af0 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0dfa000640 1 -- 172.21.15.90:0/363948343 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0df402c070 con 0x563ce5352af0 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e15a7c740 10 client.16287 fetch_fsmap learned FSMap version 48 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.351+0000 7f0e15a7c740 1 -- 172.21.15.90:0/363948343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563ce53e4f30 con 0x563ce5352af0 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.352+0000 7f0dfa000640 1 -- 172.21.15.90:0/363948343 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0df4003df0 con 0x563ce5352af0 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.352+0000 7f0dfa000640 1 -- 172.21.15.90:0/363948343 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 8434+0+0 (secure 0 0 0) 0x7f0df4004ac0 con 0x563ce5352af0 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.352+0000 7f0e15a7c740 10 client.16287 fetch_fsmap finished waiting for FSMap version 48 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.352+0000 7f0e15a7c740 10 client.16287 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.352+0000 7f0e15a7c740 20 client.16287 populate_metadata read hostname 'smithi090' 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.352+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x563ce53e5620 0x563ce53e7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.352+0000 7f0e15a7c740 4 client.16287 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a2ed6f05", "release": true, "await": true}] 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.352+0000 7f0e15a7c740 1 -- 172.21.15.90:0/363948343 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a2ed6f05", "release": true, "await": true}) -- 0x563ce52c6c90 con 0x563ce53e5620 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.353+0000 7f0e11000640 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x563ce53e5620 0x563ce53e7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:18:50.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.353+0000 7f0dfa000640 10 client.16287 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.353+0000 7f0e11000640 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x563ce53e5620 0x563ce53e7a40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=968e6ae865e246af server_cookie=229773c58006f36 in_seq=0 out_seq=0 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0dfa000640 1 -- 172.21.15.90:0/363948343 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563ce52c6c90 con 0x563ce53e5620 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0dfa000640 10 client.16287 handle_command_reply: tid=0 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 client.16287 shutdown 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 -- 172.21.15.90:0/363948343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 msgr2=0x563ce536c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce536c000 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0df401bea0 tx=0x7f0df400b590 comp rx=0 tx=0).stop 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 -- 172.21.15.90:0/363948343 shutdown_connections 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x563ce53e5620 0x563ce53e7a40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x563ce535ea80 0x563ce536c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563ce5352af0 0x563ce536c000 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 --2- 172.21.15.90:0/363948343 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x563ce53521d0 0x563ce536fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 -- 172.21.15.90:0/363948343 >> 172.21.15.90:0/363948343 conn(0x563ce5350000 msgr2=0x563ce5372a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0dfa000640 0 client.16287 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:18:50.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 -- 172.21.15.90:0/363948343 shutdown_connections 2024-09-10T22:18:50.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.389+0000 7f0e15a7c740 1 -- 172.21.15.90:0/363948343 wait complete. 2024-09-10T22:18:50.654 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:18:50.654 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 48, 2024-09-10T22:18:50.654 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:18:50.654 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 48, 2024-09-10T22:18:50.654 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:18:50.654 INFO:teuthology.orchestra.run.smithi090.stdout: "a2ed6f05": { 2024-09-10T22:18:50.654 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 48, 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 142.8, 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:18:50.655 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:50.656 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:50.656 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:50.656 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:50.656 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:18:50.656 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:18:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e15a7c740 20 client.16287 trim_cache size 0 max 16384 2024-09-10T22:18:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 -- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0df00b5580 msgr2=0x7f0df00b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0df00b5580 0x7f0df00b7a40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x563ce5355350 tx=0x7f0dfc10a860 comp rx=0 tx=0).stop 2024-09-10T22:18:50.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 -- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 msgr2=0x7f0e0c19b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:18:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 0x7f0e0c19b130 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc02a930 tx=0x7f0dfc003c20 comp rx=0 tx=0).stop 2024-09-10T22:18:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 -- 172.21.15.90:0/4241513335 shutdown_connections 2024-09-10T22:18:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f0df00b5580 0x7f0df00b7a40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0e0c10b180 0x7f0e0c1a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0e0c10a800 0x7f0e0c19b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 --2- 172.21.15.90:0/4241513335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e0c104df0 0x7f0e0c19b130 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:18:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 -- 172.21.15.90:0/4241513335 >> 172.21.15.90:0/4241513335 conn(0x7f0e0c100840 msgr2=0x7f0e0c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:18:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.390+0000 7f0e11a00640 1 -- 172.21.15.90:0/4241513335 shutdown_connections 2024-09-10T22:18:50.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:18:50.391+0000 7f0e11a00640 1 -- 172.21.15.90:0/4241513335 wait complete. 2024-09-10T22:18:50.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7646 5.85 MB/sec warmup 101 sec latency 21673.061 ms 2024-09-10T22:18:51.582 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a2ed6f05', total seconds elapsed: 27.1 2024-09-10T22:18:51.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7793 5.89 MB/sec warmup 102 sec latency 1460.667 ms 2024-09-10T22:18:52.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:51 smithi171 ceph-mon[34138]: pgmap v885: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 KiB/s wr, 13 op/s 2024-09-10T22:18:52.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:51 smithi090 ceph-mon[30350]: pgmap v885: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 KiB/s wr, 13 op/s 2024-09-10T22:18:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:51 smithi154 ceph-mon[34495]: pgmap v885: 129 pgs: 129 active+clean; 483 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 47 KiB/s wr, 13 op/s 2024-09-10T22:18:52.861 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7926 5.93 MB/sec warmup 103 sec latency 2373.492 ms 2024-09-10T22:18:53.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8043 5.90 MB/sec warmup 104 sec latency 56.144 ms 2024-09-10T22:18:54.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:53 smithi171 ceph-mon[34138]: pgmap v886: 129 pgs: 129 active+clean; 485 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 505 KiB/s wr, 43 op/s 2024-09-10T22:18:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:53 smithi090 ceph-mon[30350]: pgmap v886: 129 pgs: 129 active+clean; 485 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 505 KiB/s wr, 43 op/s 2024-09-10T22:18:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:53 smithi154 ceph-mon[34495]: pgmap v886: 129 pgs: 129 active+clean; 485 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 505 KiB/s wr, 43 op/s 2024-09-10T22:18:54.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8156 5.86 MB/sec warmup 105 sec latency 56.488 ms 2024-09-10T22:18:55.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8285 5.84 MB/sec warmup 106 sec latency 77.057 ms 2024-09-10T22:18:56.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:55 smithi171 ceph-mon[34138]: pgmap v887: 129 pgs: 129 active+clean; 489 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-09-10T22:18:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:55 smithi090 ceph-mon[30350]: pgmap v887: 129 pgs: 129 active+clean; 489 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-09-10T22:18:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:55 smithi154 ceph-mon[34495]: pgmap v887: 129 pgs: 129 active+clean; 489 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-09-10T22:18:56.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8414 5.80 MB/sec warmup 107 sec latency 80.861 ms 2024-09-10T22:18:57.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8554 5.78 MB/sec warmup 108 sec latency 68.359 ms 2024-09-10T22:18:58.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:57 smithi171 ceph-mon[34138]: pgmap v888: 129 pgs: 129 active+clean; 494 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-09-10T22:18:58.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:57 smithi090 ceph-mon[30350]: pgmap v888: 129 pgs: 129 active+clean; 494 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-09-10T22:18:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:57 smithi154 ceph-mon[34495]: pgmap v888: 129 pgs: 129 active+clean; 494 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-09-10T22:18:58.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8680 5.78 MB/sec warmup 109 sec latency 106.524 ms 2024-09-10T22:18:59.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8807 5.77 MB/sec warmup 110 sec latency 96.769 ms 2024-09-10T22:19:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:18:59 smithi171 ceph-mon[34138]: pgmap v889: 129 pgs: 129 active+clean; 497 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.0 MiB/s wr, 173 op/s 2024-09-10T22:19:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:18:59 smithi090 ceph-mon[30350]: pgmap v889: 129 pgs: 129 active+clean; 497 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.0 MiB/s wr, 173 op/s 2024-09-10T22:19:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:18:59 smithi154 ceph-mon[34495]: pgmap v889: 129 pgs: 129 active+clean; 497 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 2.0 MiB/s wr, 173 op/s 2024-09-10T22:19:00.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8943 5.81 MB/sec warmup 111 sec latency 82.193 ms 2024-09-10T22:19:01.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9121 5.88 MB/sec warmup 112 sec latency 69.977 ms 2024-09-10T22:19:02.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:01 smithi171 ceph-mon[34138]: pgmap v890: 129 pgs: 129 active+clean; 497 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-10T22:19:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:01 smithi090 ceph-mon[30350]: pgmap v890: 129 pgs: 129 active+clean; 497 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-10T22:19:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:01 smithi154 ceph-mon[34495]: pgmap v890: 129 pgs: 129 active+clean; 497 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-10T22:19:02.862 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9302 5.96 MB/sec warmup 113 sec latency 73.850 ms 2024-09-10T22:19:03.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9468 5.96 MB/sec warmup 114 sec latency 78.543 ms 2024-09-10T22:19:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:03 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:04 smithi090 ceph-mon[30350]: pgmap v891: 129 pgs: 129 active+clean; 509 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-09-10T22:19:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:03 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:03 smithi154 ceph-mon[34495]: pgmap v891: 129 pgs: 129 active+clean; 509 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-09-10T22:19:04.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:04 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:04.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:04 smithi171 ceph-mon[34138]: pgmap v891: 129 pgs: 129 active+clean; 509 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-09-10T22:19:04.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9579 5.95 MB/sec warmup 115 sec latency 101.278 ms 2024-09-10T22:19:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:04 smithi090 ceph-mon[30350]: osdmap e109: 12 total, 12 up, 12 in 2024-09-10T22:19:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:04 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:04 smithi154 ceph-mon[34495]: osdmap e109: 12 total, 12 up, 12 in 2024-09-10T22:19:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:04 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:05.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:04 smithi171 ceph-mon[34138]: osdmap e109: 12 total, 12 up, 12 in 2024-09-10T22:19:05.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:04 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:05.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9694 5.96 MB/sec warmup 116 sec latency 74.525 ms 2024-09-10T22:19:06.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:06 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:19:06.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:06 smithi090 ceph-mon[30350]: pgmap v893: 129 pgs: 129 active+clean; 513 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-09-10T22:19:06.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:06 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:19:06.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:06 smithi154 ceph-mon[34495]: pgmap v893: 129 pgs: 129 active+clean; 513 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-09-10T22:19:06.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:06 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:19:06.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:06 smithi171 ceph-mon[34138]: pgmap v893: 129 pgs: 129 active+clean; 513 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 292 op/s 2024-09-10T22:19:06.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9805 5.96 MB/sec warmup 117 sec latency 304.661 ms 2024-09-10T22:19:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:07 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:07 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:07 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:07 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:07 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-09-10T22:19:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:07 smithi090 ceph-mon[30350]: Unable to set osd_memory_target on smithi090 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-10T22:19:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:07 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:19:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:07 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:19:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:07 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:19:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:07 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:07 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:07 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:07 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:07 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-09-10T22:19:07.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:07 smithi154 ceph-mon[34495]: Unable to set osd_memory_target on smithi090 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-10T22:19:07.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:07 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:19:07.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:07 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:19:07.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:07 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:19:07.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:07 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:07 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:07 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:07 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:19:07.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:07 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-09-10T22:19:07.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:07 smithi171 ceph-mon[34138]: Unable to set osd_memory_target on smithi090 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-10T22:19:07.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:07 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:19:07.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:07 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:19:07.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:07 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:19:07.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9966 6.00 MB/sec warmup 118 sec latency 75.385 ms 2024-09-10T22:19:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:08 smithi090 ceph-mon[30350]: pgmap v894: 129 pgs: 129 active+clean; 517 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-09-10T22:19:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:08 smithi154 ceph-mon[34495]: pgmap v894: 129 pgs: 129 active+clean; 517 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-09-10T22:19:08.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:08 smithi171 ceph-mon[34138]: pgmap v894: 129 pgs: 129 active+clean; 517 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-09-10T22:19:08.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10097 6.01 MB/sec warmup 119 sec latency 60.248 ms 2024-09-10T22:19:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:10 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:10 smithi090 ceph-mon[30350]: pgmap v895: 129 pgs: 129 active+clean; 525 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 278 op/s 2024-09-10T22:19:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:10 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:10 smithi154 ceph-mon[34495]: pgmap v895: 129 pgs: 129 active+clean; 525 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 278 op/s 2024-09-10T22:19:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:10 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:10 smithi171 ceph-mon[34138]: pgmap v895: 129 pgs: 129 active+clean; 525 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 278 op/s 2024-09-10T22:19:10.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10342 10.57 MB/sec execute 1 sec latency 62.843 ms 2024-09-10T22:19:11.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:11 smithi090 ceph-mon[30350]: osdmap e110: 12 total, 12 up, 12 in 2024-09-10T22:19:11.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10351 5.64 MB/sec execute 2 sec latency 600.725 ms 2024-09-10T22:19:11.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:11 smithi171 ceph-mon[34138]: osdmap e110: 12 total, 12 up, 12 in 2024-09-10T22:19:12.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:11 smithi154 ceph-mon[34495]: osdmap e110: 12 total, 12 up, 12 in 2024-09-10T22:19:12.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10525 8.07 MB/sec execute 3 sec latency 651.983 ms 2024-09-10T22:19:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:12 smithi171 ceph-mon[34138]: pgmap v897: 129 pgs: 129 active+clean; 528 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 3.2 MiB/s wr, 202 op/s 2024-09-10T22:19:13.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:12 smithi090 ceph-mon[30350]: pgmap v897: 129 pgs: 129 active+clean; 528 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 3.2 MiB/s wr, 202 op/s 2024-09-10T22:19:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:12 smithi154 ceph-mon[34495]: pgmap v897: 129 pgs: 129 active+clean; 528 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 3.2 MiB/s wr, 202 op/s 2024-09-10T22:19:13.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10656 7.82 MB/sec execute 4 sec latency 66.635 ms 2024-09-10T22:19:13.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:13 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:14.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:13 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:14.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:13 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:14.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10769 7.18 MB/sec execute 5 sec latency 77.784 ms 2024-09-10T22:19:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:14 smithi171 ceph-mon[34138]: pgmap v898: 129 pgs: 129 active+clean; 536 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 219 op/s 2024-09-10T22:19:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:14 smithi090 ceph-mon[30350]: pgmap v898: 129 pgs: 129 active+clean; 536 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 219 op/s 2024-09-10T22:19:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:14 smithi154 ceph-mon[34495]: pgmap v898: 129 pgs: 129 active+clean; 536 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 219 op/s 2024-09-10T22:19:15.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10863 6.57 MB/sec execute 6 sec latency 79.203 ms 2024-09-10T22:19:16.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:15 smithi171 ceph-mon[34138]: pgmap v899: 129 pgs: 129 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 225 op/s 2024-09-10T22:19:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:15 smithi090 ceph-mon[30350]: pgmap v899: 129 pgs: 129 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 225 op/s 2024-09-10T22:19:16.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:15 smithi154 ceph-mon[34495]: pgmap v899: 129 pgs: 129 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 225 op/s 2024-09-10T22:19:16.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11018 7.47 MB/sec execute 7 sec latency 68.464 ms 2024-09-10T22:19:17.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11164 7.92 MB/sec execute 8 sec latency 79.239 ms 2024-09-10T22:19:18.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:17 smithi171 ceph-mon[34138]: pgmap v900: 129 pgs: 129 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 234 op/s 2024-09-10T22:19:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:17 smithi090 ceph-mon[30350]: pgmap v900: 129 pgs: 129 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 234 op/s 2024-09-10T22:19:18.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:17 smithi154 ceph-mon[34495]: pgmap v900: 129 pgs: 129 active+clean; 541 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 234 op/s 2024-09-10T22:19:18.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11306 8.12 MB/sec execute 9 sec latency 56.479 ms 2024-09-10T22:19:19.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11412 7.43 MB/sec execute 10 sec latency 65.654 ms 2024-09-10T22:19:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:19 smithi171 ceph-mon[34138]: pgmap v901: 129 pgs: 129 active+clean; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-09-10T22:19:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:19 smithi090 ceph-mon[30350]: pgmap v901: 129 pgs: 129 active+clean; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-09-10T22:19:20.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:19 smithi154 ceph-mon[34495]: pgmap v901: 129 pgs: 129 active+clean; 544 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 234 op/s 2024-09-10T22:19:20.863 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11524 7.14 MB/sec execute 11 sec latency 72.915 ms 2024-09-10T22:19:21.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11619 7.09 MB/sec execute 12 sec latency 363.086 ms 2024-09-10T22:19:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:21 smithi171 ceph-mon[34138]: pgmap v902: 129 pgs: 129 active+clean; 547 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 249 op/s 2024-09-10T22:19:22.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:21 smithi090 ceph-mon[30350]: pgmap v902: 129 pgs: 129 active+clean; 547 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 249 op/s 2024-09-10T22:19:22.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:21 smithi154 ceph-mon[34495]: pgmap v902: 129 pgs: 129 active+clean; 547 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 249 op/s 2024-09-10T22:19:22.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11733 6.64 MB/sec execute 13 sec latency 96.881 ms 2024-09-10T22:19:23.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11864 6.42 MB/sec execute 14 sec latency 71.921 ms 2024-09-10T22:19:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:23 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:23 smithi171 ceph-mon[34138]: pgmap v903: 129 pgs: 129 active+clean; 549 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-10T22:19:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:23 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:23 smithi090 ceph-mon[30350]: pgmap v903: 129 pgs: 129 active+clean; 549 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-10T22:19:24.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:23 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:24.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:23 smithi154 ceph-mon[34495]: pgmap v903: 129 pgs: 129 active+clean; 549 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 265 op/s 2024-09-10T22:19:24.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11988 6.26 MB/sec execute 15 sec latency 138.693 ms 2024-09-10T22:19:25.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 12131 6.35 MB/sec execute 16 sec latency 64.971 ms 2024-09-10T22:19:26.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:25 smithi171 ceph-mon[34138]: pgmap v904: 129 pgs: 129 active+clean; 548 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 272 op/s 2024-09-10T22:19:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:25 smithi090 ceph-mon[30350]: pgmap v904: 129 pgs: 129 active+clean; 548 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 272 op/s 2024-09-10T22:19:26.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:25 smithi154 ceph-mon[34495]: pgmap v904: 129 pgs: 129 active+clean; 548 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 272 op/s 2024-09-10T22:19:26.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 12234 6.36 MB/sec execute 17 sec latency 357.568 ms 2024-09-10T22:19:27.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 12394 6.69 MB/sec execute 18 sec latency 81.856 ms 2024-09-10T22:19:28.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:27 smithi171 ceph-mon[34138]: pgmap v905: 129 pgs: 129 active+clean; 547 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-09-10T22:19:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:27 smithi090 ceph-mon[30350]: pgmap v905: 129 pgs: 129 active+clean; 547 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-09-10T22:19:28.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:27 smithi154 ceph-mon[34495]: pgmap v905: 129 pgs: 129 active+clean; 547 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-09-10T22:19:28.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 12537 6.62 MB/sec execute 19 sec latency 86.167 ms 2024-09-10T22:19:29.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 12676 6.73 MB/sec execute 20 sec latency 70.982 ms 2024-09-10T22:19:30.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:29 smithi171 ceph-mon[34138]: pgmap v906: 129 pgs: 129 active+clean; 552 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 291 op/s 2024-09-10T22:19:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:29 smithi090 ceph-mon[30350]: pgmap v906: 129 pgs: 129 active+clean; 552 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 291 op/s 2024-09-10T22:19:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:29 smithi154 ceph-mon[34495]: pgmap v906: 129 pgs: 129 active+clean; 552 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 291 op/s 2024-09-10T22:19:30.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 12793 6.51 MB/sec execute 21 sec latency 71.624 ms 2024-09-10T22:19:31.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 12876 6.55 MB/sec execute 22 sec latency 487.480 ms 2024-09-10T22:19:32.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:31 smithi171 ceph-mon[34138]: pgmap v907: 129 pgs: 129 active+clean; 552 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-09-10T22:19:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:31 smithi090 ceph-mon[30350]: pgmap v907: 129 pgs: 129 active+clean; 552 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-09-10T22:19:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:31 smithi154 ceph-mon[34495]: pgmap v907: 129 pgs: 129 active+clean; 552 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-09-10T22:19:32.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 13043 6.60 MB/sec execute 23 sec latency 78.612 ms 2024-09-10T22:19:33.864 INFO:tasks.workunit.client.0.smithi090.stdout: 10 13211 6.63 MB/sec execute 24 sec latency 58.309 ms 2024-09-10T22:19:34.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:33 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:34.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:33 smithi171 ceph-mon[34138]: pgmap v908: 129 pgs: 129 active+clean; 557 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-09-10T22:19:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:33 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:33 smithi090 ceph-mon[30350]: pgmap v908: 129 pgs: 129 active+clean; 557 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-09-10T22:19:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:33 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:33 smithi154 ceph-mon[34495]: pgmap v908: 129 pgs: 129 active+clean; 557 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 268 op/s 2024-09-10T22:19:34.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 13335 6.63 MB/sec execute 25 sec latency 68.468 ms 2024-09-10T22:19:35.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 13441 6.58 MB/sec execute 26 sec latency 73.782 ms 2024-09-10T22:19:36.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:35 smithi171 ceph-mon[34138]: pgmap v909: 129 pgs: 129 active+clean; 559 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 265 op/s 2024-09-10T22:19:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:35 smithi090 ceph-mon[30350]: pgmap v909: 129 pgs: 129 active+clean; 559 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 265 op/s 2024-09-10T22:19:36.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:35 smithi154 ceph-mon[34495]: pgmap v909: 129 pgs: 129 active+clean; 559 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 265 op/s 2024-09-10T22:19:36.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 13586 6.84 MB/sec execute 27 sec latency 58.286 ms 2024-09-10T22:19:37.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 13752 7.05 MB/sec execute 28 sec latency 54.170 ms 2024-09-10T22:19:38.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 13953 7.28 MB/sec execute 29 sec latency 76.303 ms 2024-09-10T22:19:38.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:38 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:38.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:38 smithi171 ceph-mon[34138]: pgmap v910: 129 pgs: 129 active+clean; 565 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2024-09-10T22:19:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:38 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:38 smithi090 ceph-mon[30350]: pgmap v910: 129 pgs: 129 active+clean; 565 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2024-09-10T22:19:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:38 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:19:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:38 smithi154 ceph-mon[34495]: pgmap v910: 129 pgs: 129 active+clean; 565 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2024-09-10T22:19:39.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14085 7.22 MB/sec execute 30 sec latency 84.078 ms 2024-09-10T22:19:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:39 smithi171 ceph-mon[34138]: pgmap v911: 129 pgs: 129 active+clean; 573 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 298 op/s 2024-09-10T22:19:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:39 smithi090 ceph-mon[30350]: pgmap v911: 129 pgs: 129 active+clean; 573 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 298 op/s 2024-09-10T22:19:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:39 smithi154 ceph-mon[34495]: pgmap v911: 129 pgs: 129 active+clean; 573 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.6 MiB/s wr, 298 op/s 2024-09-10T22:19:40.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14185 7.14 MB/sec execute 31 sec latency 68.444 ms 2024-09-10T22:19:41.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14294 7.22 MB/sec execute 32 sec latency 248.009 ms 2024-09-10T22:19:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:41 smithi171 ceph-mon[34138]: pgmap v912: 129 pgs: 129 active+clean; 574 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-09-10T22:19:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:41 smithi090 ceph-mon[30350]: pgmap v912: 129 pgs: 129 active+clean; 574 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-09-10T22:19:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:41 smithi154 ceph-mon[34495]: pgmap v912: 129 pgs: 129 active+clean; 574 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 230 op/s 2024-09-10T22:19:42.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14420 7.20 MB/sec execute 33 sec latency 63.597 ms 2024-09-10T22:19:43.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14563 7.27 MB/sec execute 34 sec latency 57.331 ms 2024-09-10T22:19:44.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:43 smithi171 ceph-mon[34138]: pgmap v913: 129 pgs: 129 active+clean; 577 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 271 op/s 2024-09-10T22:19:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:43 smithi090 ceph-mon[30350]: pgmap v913: 129 pgs: 129 active+clean; 577 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 271 op/s 2024-09-10T22:19:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:43 smithi154 ceph-mon[34495]: pgmap v913: 129 pgs: 129 active+clean; 577 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 271 op/s 2024-09-10T22:19:44.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14690 7.26 MB/sec execute 35 sec latency 50.785 ms 2024-09-10T22:19:45.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14797 7.11 MB/sec execute 36 sec latency 75.899 ms 2024-09-10T22:19:46.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:45 smithi171 ceph-mon[34138]: pgmap v914: 129 pgs: 129 active+clean; 580 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 275 op/s 2024-09-10T22:19:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:45 smithi090 ceph-mon[30350]: pgmap v914: 129 pgs: 129 active+clean; 580 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 275 op/s 2024-09-10T22:19:46.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:45 smithi154 ceph-mon[34495]: pgmap v914: 129 pgs: 129 active+clean; 580 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 275 op/s 2024-09-10T22:19:46.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14850 6.95 MB/sec execute 37 sec latency 570.136 ms 2024-09-10T22:19:47.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14937 6.89 MB/sec execute 38 sec latency 790.945 ms 2024-09-10T22:19:48.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:47 smithi171 ceph-mon[34138]: pgmap v915: 129 pgs: 129 active+clean; 582 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-09-10T22:19:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:47 smithi090 ceph-mon[30350]: pgmap v915: 129 pgs: 129 active+clean; 582 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-09-10T22:19:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:47 smithi154 ceph-mon[34495]: pgmap v915: 129 pgs: 129 active+clean; 582 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-09-10T22:19:48.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15071 6.82 MB/sec execute 39 sec latency 59.542 ms 2024-09-10T22:19:49.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15204 6.71 MB/sec execute 40 sec latency 111.442 ms 2024-09-10T22:19:50.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:49 smithi171 ceph-mon[34138]: pgmap v916: 129 pgs: 129 active+clean; 584 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 299 op/s 2024-09-10T22:19:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:49 smithi090 ceph-mon[30350]: pgmap v916: 129 pgs: 129 active+clean; 584 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 299 op/s 2024-09-10T22:19:50.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:49 smithi154 ceph-mon[34495]: pgmap v916: 129 pgs: 129 active+clean; 584 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 299 op/s 2024-09-10T22:19:50.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15347 6.78 MB/sec execute 41 sec latency 89.275 ms 2024-09-10T22:19:51.865 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15467 6.79 MB/sec execute 42 sec latency 140.606 ms 2024-09-10T22:19:52.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:51 smithi171 ceph-mon[34138]: pgmap v917: 129 pgs: 129 active+clean; 585 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-10T22:19:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:51 smithi090 ceph-mon[30350]: pgmap v917: 129 pgs: 129 active+clean; 585 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-10T22:19:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:51 smithi154 ceph-mon[34495]: pgmap v917: 129 pgs: 129 active+clean; 585 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-10T22:19:52.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15534 6.66 MB/sec execute 43 sec latency 649.800 ms 2024-09-10T22:19:53.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15672 6.70 MB/sec execute 44 sec latency 73.404 ms 2024-09-10T22:19:54.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:53 smithi171 ceph-mon[34138]: pgmap v918: 129 pgs: 129 active+clean; 581 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 279 op/s 2024-09-10T22:19:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:53 smithi090 ceph-mon[30350]: pgmap v918: 129 pgs: 129 active+clean; 581 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 279 op/s 2024-09-10T22:19:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:53 smithi154 ceph-mon[34495]: pgmap v918: 129 pgs: 129 active+clean; 581 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 279 op/s 2024-09-10T22:19:54.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15828 6.76 MB/sec execute 45 sec latency 63.607 ms 2024-09-10T22:19:55.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15971 6.83 MB/sec execute 46 sec latency 300.619 ms 2024-09-10T22:19:56.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:55 smithi171 ceph-mon[34138]: pgmap v919: 129 pgs: 129 active+clean; 579 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 270 op/s 2024-09-10T22:19:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:55 smithi090 ceph-mon[30350]: pgmap v919: 129 pgs: 129 active+clean; 579 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 270 op/s 2024-09-10T22:19:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:55 smithi154 ceph-mon[34495]: pgmap v919: 129 pgs: 129 active+clean; 579 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 270 op/s 2024-09-10T22:19:56.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16101 6.81 MB/sec execute 47 sec latency 202.179 ms 2024-09-10T22:19:57.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16182 6.78 MB/sec execute 48 sec latency 585.635 ms 2024-09-10T22:19:58.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:57 smithi171 ceph-mon[34138]: pgmap v920: 129 pgs: 129 active+clean; 580 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-09-10T22:19:58.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:57 smithi090 ceph-mon[30350]: pgmap v920: 129 pgs: 129 active+clean; 580 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-09-10T22:19:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:57 smithi154 ceph-mon[34495]: pgmap v920: 129 pgs: 129 active+clean; 580 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 228 op/s 2024-09-10T22:19:58.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16312 6.71 MB/sec execute 49 sec latency 72.969 ms 2024-09-10T22:19:59.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16430 6.64 MB/sec execute 50 sec latency 59.804 ms 2024-09-10T22:20:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:19:59 smithi171 ceph-mon[34138]: pgmap v921: 129 pgs: 129 active+clean; 586 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-09-10T22:20:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:19:59 smithi090 ceph-mon[30350]: pgmap v921: 129 pgs: 129 active+clean; 586 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-09-10T22:20:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:19:59 smithi154 ceph-mon[34495]: pgmap v921: 129 pgs: 129 active+clean; 586 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 281 op/s 2024-09-10T22:20:00.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16565 6.65 MB/sec execute 51 sec latency 80.347 ms 2024-09-10T22:20:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:00 smithi171 ceph-mon[34138]: overall HEALTH_OK 2024-09-10T22:20:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:00 smithi090 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T22:20:01.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:00 smithi154 ceph-mon[34495]: overall HEALTH_OK 2024-09-10T22:20:01.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16705 6.68 MB/sec execute 52 sec latency 105.757 ms 2024-09-10T22:20:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:01 smithi090 ceph-mon[30350]: pgmap v922: 129 pgs: 129 active+clean; 586 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.7 MiB/s wr, 213 op/s 2024-09-10T22:20:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:01 smithi154 ceph-mon[34495]: pgmap v922: 129 pgs: 129 active+clean; 586 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.7 MiB/s wr, 213 op/s 2024-09-10T22:20:02.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:01 smithi171 ceph-mon[34138]: pgmap v922: 129 pgs: 129 active+clean; 586 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 2.7 MiB/s wr, 213 op/s 2024-09-10T22:20:02.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16853 6.70 MB/sec execute 53 sec latency 63.109 ms 2024-09-10T22:20:03.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16992 6.78 MB/sec execute 54 sec latency 97.778 ms 2024-09-10T22:20:04.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 17120 6.77 MB/sec execute 55 sec latency 83.623 ms 2024-09-10T22:20:04.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:04 smithi154 ceph-mon[34495]: osdmap e111: 12 total, 12 up, 12 in 2024-09-10T22:20:04.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:04 smithi154 ceph-mon[34495]: pgmap v924: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 598 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-09-10T22:20:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:04 smithi171 ceph-mon[34138]: osdmap e111: 12 total, 12 up, 12 in 2024-09-10T22:20:04.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:04 smithi171 ceph-mon[34138]: pgmap v924: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 598 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-09-10T22:20:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:04 smithi090 ceph-mon[30350]: osdmap e111: 12 total, 12 up, 12 in 2024-09-10T22:20:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:04 smithi090 ceph-mon[30350]: pgmap v924: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 598 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.7 MiB/s wr, 267 op/s 2024-09-10T22:20:05.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 17307 6.95 MB/sec execute 56 sec latency 86.805 ms 2024-09-10T22:20:06.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:05 smithi171 ceph-mon[34138]: pgmap v925: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 606 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-09-10T22:20:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:05 smithi090 ceph-mon[30350]: pgmap v925: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 606 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-09-10T22:20:06.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:05 smithi154 ceph-mon[34495]: pgmap v925: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 606 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 258 op/s 2024-09-10T22:20:06.866 INFO:tasks.workunit.client.0.smithi090.stdout: 10 17442 6.95 MB/sec execute 57 sec latency 93.068 ms 2024-09-10T22:20:07.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:06 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:20:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:06 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:20:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:06 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:20:07.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 17552 6.98 MB/sec execute 58 sec latency 85.415 ms 2024-09-10T22:20:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi171 to 176.4M 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: Unable to set osd_memory_target on smithi171 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi154 to 176.4M 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: Unable to set osd_memory_target on smithi154 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:20:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:08 smithi090 ceph-mon[30350]: pgmap v926: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 611 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-10T22:20:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi171 to 176.4M 2024-09-10T22:20:08.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: Unable to set osd_memory_target on smithi171 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:20:08.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi154 to 176.4M 2024-09-10T22:20:08.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: Unable to set osd_memory_target on smithi154 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:20:08.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:08 smithi154 ceph-mon[34495]: pgmap v926: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 611 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi171 to 176.4M 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: Unable to set osd_memory_target on smithi171 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:20:08.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi154 to 176.4M 2024-09-10T22:20:08.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: Unable to set osd_memory_target on smithi154 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:20:08.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:08 smithi171 ceph-mon[34138]: pgmap v926: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 611 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 262 op/s 2024-09-10T22:20:08.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 17683 7.04 MB/sec execute 59 sec latency 69.165 ms 2024-09-10T22:20:09.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:09 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:20:09.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:09 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:20:09.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:09 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:20:09.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:09 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:20:09.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:09 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:20:09.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:09 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:20:09.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:09 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:20:09.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:09 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:20:09.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:09 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:20:09.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 17814 7.08 MB/sec execute 60 sec latency 72.715 ms 2024-09-10T22:20:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:10 smithi090 ceph-mon[30350]: pgmap v927: 129 pgs: 129 active+clean; 624 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 276 op/s 2024-09-10T22:20:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:10 smithi154 ceph-mon[34495]: pgmap v927: 129 pgs: 129 active+clean; 624 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 276 op/s 2024-09-10T22:20:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:10 smithi171 ceph-mon[34138]: pgmap v927: 129 pgs: 129 active+clean; 624 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 276 op/s 2024-09-10T22:20:10.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 17939 7.08 MB/sec execute 61 sec latency 70.471 ms 2024-09-10T22:20:11.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18049 7.02 MB/sec execute 62 sec latency 67.632 ms 2024-09-10T22:20:11.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:11 smithi171 ceph-mon[34138]: osdmap e112: 12 total, 12 up, 12 in 2024-09-10T22:20:12.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:11 smithi090 ceph-mon[30350]: osdmap e112: 12 total, 12 up, 12 in 2024-09-10T22:20:12.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:11 smithi154 ceph-mon[34495]: osdmap e112: 12 total, 12 up, 12 in 2024-09-10T22:20:12.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18154 6.97 MB/sec execute 63 sec latency 226.095 ms 2024-09-10T22:20:12.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:12 smithi171 ceph-mon[34138]: pgmap v929: 129 pgs: 129 active+clean; 624 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 251 op/s 2024-09-10T22:20:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:12 smithi090 ceph-mon[30350]: pgmap v929: 129 pgs: 129 active+clean; 624 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 251 op/s 2024-09-10T22:20:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:12 smithi154 ceph-mon[34495]: pgmap v929: 129 pgs: 129 active+clean; 624 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 251 op/s 2024-09-10T22:20:13.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18265 6.93 MB/sec execute 64 sec latency 68.169 ms 2024-09-10T22:20:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:13 smithi171 ceph-mon[34138]: pgmap v930: 129 pgs: 129 active+clean; 631 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 237 op/s 2024-09-10T22:20:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:13 smithi090 ceph-mon[30350]: pgmap v930: 129 pgs: 129 active+clean; 631 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 237 op/s 2024-09-10T22:20:14.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:13 smithi154 ceph-mon[34495]: pgmap v930: 129 pgs: 129 active+clean; 631 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 237 op/s 2024-09-10T22:20:14.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18386 6.87 MB/sec execute 65 sec latency 83.076 ms 2024-09-10T22:20:15.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18508 6.80 MB/sec execute 66 sec latency 85.683 ms 2024-09-10T22:20:16.083 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.8 2024-09-10T22:20:16.083 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:20:16.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:15 smithi171 ceph-mon[34138]: pgmap v931: 129 pgs: 129 active+clean; 636 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 260 op/s 2024-09-10T22:20:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:15 smithi090 ceph-mon[30350]: pgmap v931: 129 pgs: 129 active+clean; 636 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 260 op/s 2024-09-10T22:20:16.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:15 smithi154 ceph-mon[34495]: pgmap v931: 129 pgs: 129 active+clean; 636 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 260 op/s 2024-09-10T22:20:16.673 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:20:16.868 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18628 6.76 MB/sec execute 67 sec latency 106.593 ms 2024-09-10T22:20:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.373+0000 7f7d13200640 1 -- 172.21.15.90:0/684678095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14108620 msgr2=0x7f7d14108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.374+0000 7f7d13200640 1 --2- 172.21.15.90:0/684678095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14108620 0x7f7d14108a60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009860 tx=0x7f7cfc02a340 comp rx=0 tx=0).stop 2024-09-10T22:20:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.374+0000 7f7d13200640 1 -- 172.21.15.90:0/684678095 shutdown_connections 2024-09-10T22:20:17.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.374+0000 7f7d13200640 1 --2- 172.21.15.90:0/684678095 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d14108fa0 0x7f7d1410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.374+0000 7f7d13200640 1 --2- 172.21.15.90:0/684678095 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14108620 0x7f7d14108a60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.374+0000 7f7d13200640 1 --2- 172.21.15.90:0/684678095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d14102c10 0x7f7d14102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.374+0000 7f7d13200640 1 -- 172.21.15.90:0/684678095 >> 172.21.15.90:0/684678095 conn(0x7f7d140fe660 msgr2=0x7f7d14100a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:17.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.374+0000 7f7d13200640 1 -- 172.21.15.90:0/684678095 shutdown_connections 2024-09-10T22:20:17.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.374+0000 7f7d13200640 1 -- 172.21.15.90:0/684678095 wait complete. 2024-09-10T22:20:17.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.375+0000 7f7d13200640 1 Processor -- start 2024-09-10T22:20:17.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.375+0000 7f7d13200640 1 -- start start 2024-09-10T22:20:17.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.376+0000 7f7d13200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d14102c10 0x7f7d14198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:17.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.376+0000 7f7d13200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d14108620 0x7f7d141994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:17.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.376+0000 7f7d11e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d14102c10 0x7f7d14198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:17.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.376+0000 7f7d11e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d14102c10 0x7f7d14198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44982/0 (socket says 172.21.15.90:44982) 2024-09-10T22:20:17.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.376+0000 7f7d11e00640 1 -- 172.21.15.90:0/3805144897 learned_addr learned my addr 172.21.15.90:0/3805144897 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:17.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.376+0000 7f7d13200640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14108fa0 0x7f7d1419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:17.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14112aa0 con 0x7f7d14102c10 2024-09-10T22:20:17.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14112920 con 0x7f7d14108fa0 2024-09-10T22:20:17.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14112c20 con 0x7f7d14108620 2024-09-10T22:20:17.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d0be00640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d14108620 0x7f7d141994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:17.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d12800640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14108fa0 0x7f7d1419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:17.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d11e00640 1 -- 172.21.15.90:0/3805144897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d14108620 msgr2=0x7f7d141994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:17.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d11e00640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d14108620 0x7f7d141994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d11e00640 1 -- 172.21.15.90:0/3805144897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14108fa0 msgr2=0x7f7d1419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:17.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d11e00640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14108fa0 0x7f7d1419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.377+0000 7f7d11e00640 1 -- 172.21.15.90:0/3805144897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1419f040 con 0x7f7d14102c10 2024-09-10T22:20:17.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.378+0000 7f7d12800640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14108fa0 0x7f7d1419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:20:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.378+0000 7f7d11e00640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d14102c10 0x7f7d14198f90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9f0 tx=0x7f7d0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.378+0000 7f7d0aa00640 1 -- 172.21.15.90:0/3805144897 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400cd80 con 0x7f7d14102c10 2024-09-10T22:20:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.378+0000 7f7d0aa00640 1 -- 172.21.15.90:0/3805144897 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7d040108c0 con 0x7f7d14102c10 2024-09-10T22:20:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.378+0000 7f7d0aa00640 1 -- 172.21.15.90:0/3805144897 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d040187c0 con 0x7f7d14102c10 2024-09-10T22:20:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.379+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1419f330 con 0x7f7d14102c10 2024-09-10T22:20:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.379+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d141a59f0 con 0x7f7d14102c10 2024-09-10T22:20:17.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.381+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d14103ec0 con 0x7f7d14102c10 2024-09-10T22:20:17.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.384+0000 7f7d0aa00640 1 -- 172.21.15.90:0/3805144897 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7d04010430 con 0x7f7d14102c10 2024-09-10T22:20:17.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.386+0000 7f7d0aa00640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7cf80b5650 0x7f7cf80b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:17.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.386+0000 7f7d0aa00640 1 -- 172.21.15.90:0/3805144897 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9490+0+0 (secure 0 0 0) 0x7f7d040e66d0 con 0x7f7d14102c10 2024-09-10T22:20:17.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.387+0000 7f7d0be00640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7cf80b5650 0x7f7cf80b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:17.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.388+0000 7f7d0be00640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7cf80b5650 0x7f7cf80b7b10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc005cd0 tx=0x7f7cfc000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:17.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.392+0000 7f7d0aa00640 1 -- 172.21.15.90:0/3805144897 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7d040ace80 con 0x7f7d14102c10 2024-09-10T22:20:17.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.797+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d1410ade0 con 0x7f7d14102c10 2024-09-10T22:20:17.802 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:20:17.802 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":55,"btime":"2024-09-10T22:19:37:723878+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:19:37.673705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:20:17.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.798+0000 7f7d0aa00640 1 -- 172.21.15.90:0/3805144897 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+12021 (secure 0 0 0) 0x7f7d040ac780 con 0x7f7d14102c10 2024-09-10T22:20:17.802 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 55 2024-09-10T22:20:17.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.802+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7cf80b5650 msgr2=0x7f7cf80b7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:17.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.802+0000 7f7d13200640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7cf80b5650 0x7f7cf80b7b10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc005cd0 tx=0x7f7cfc000bc0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.802+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d14102c10 msgr2=0x7f7d14198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:17.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.802+0000 7f7d13200640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d14102c10 0x7f7d14198f90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9f0 tx=0x7f7d0400eeb0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.803+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 shutdown_connections 2024-09-10T22:20:17.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.803+0000 7f7d13200640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f7cf80b5650 0x7f7cf80b7b10 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc005cd0 tx=0x7f7cfc000bc0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.803+0000 7f7d13200640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7d14108fa0 0x7f7d1419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.803+0000 7f7d13200640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7d14108620 0x7f7d141994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.803+0000 7f7d13200640 1 --2- 172.21.15.90:0/3805144897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7d14102c10 0x7f7d14198f90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:17.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.803+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 >> 172.21.15.90:0/3805144897 conn(0x7f7d140fe660 msgr2=0x7f7d14100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:17.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.804+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 shutdown_connections 2024-09-10T22:20:17.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:17.804+0000 7f7d13200640 1 -- 172.21.15.90:0/3805144897 wait complete. 2024-09-10T22:20:17.869 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18749 6.77 MB/sec execute 68 sec latency 90.718 ms 2024-09-10T22:20:18.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:17 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3805144897' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:20:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:17 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3805144897' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:20:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:17 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3805144897' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:20:18.843 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 154.8 --await-for 45.0' 2024-09-10T22:20:18.843 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 154.8 --await-for 45.0 2024-09-10T22:20:18.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:18 smithi090 ceph-mon[30350]: pgmap v932: 129 pgs: 129 active+clean; 637 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2024-09-10T22:20:18.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18872 6.77 MB/sec execute 69 sec latency 104.314 ms 2024-09-10T22:20:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:18 smithi154 ceph-mon[34495]: pgmap v932: 129 pgs: 129 active+clean; 637 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2024-09-10T22:20:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:18 smithi171 ceph-mon[34138]: pgmap v932: 129 pgs: 129 active+clean; 637 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 263 op/s 2024-09-10T22:20:19.440 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:20:19.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19008 6.74 MB/sec execute 70 sec latency 109.305 ms 2024-09-10T22:20:20.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:19 smithi090 ceph-mon[30350]: pgmap v933: 129 pgs: 129 active+clean; 637 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-09-10T22:20:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.230+0000 7f946e000640 1 -- 172.21.15.90:0/2665341916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9468113af0 msgr2=0x7f9468115ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.230+0000 7f946e000640 1 --2- 172.21.15.90:0/2665341916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9468113af0 0x7f9468115ee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9460009ca0 tx=0x7f946002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.230+0000 7f946e000640 1 -- 172.21.15.90:0/2665341916 shutdown_connections 2024-09-10T22:20:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.230+0000 7f946e000640 1 --2- 172.21.15.90:0/2665341916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9468113af0 0x7f9468115ee0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.230+0000 7f946e000640 1 --2- 172.21.15.90:0/2665341916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9468075e60 0x7f94681135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.230+0000 7f946e000640 1 --2- 172.21.15.90:0/2665341916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468075540 0x7f9468075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.232 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.230+0000 7f946e000640 1 -- 172.21.15.90:0/2665341916 >> 172.21.15.90:0/2665341916 conn(0x7f94680fe820 msgr2=0x7f9468100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:20.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.231+0000 7f946e000640 1 -- 172.21.15.90:0/2665341916 shutdown_connections 2024-09-10T22:20:20.233 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.231+0000 7f946e000640 1 -- 172.21.15.90:0/2665341916 wait complete. 2024-09-10T22:20:20.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.232+0000 7f946e000640 1 Processor -- start 2024-09-10T22:20:20.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.233+0000 7f946e000640 1 -- start start 2024-09-10T22:20:20.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.233+0000 7f946e000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9468075540 0x7f946819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.233+0000 7f946e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468075e60 0x7f946819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.233+0000 7f946e000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9468113af0 0x7f94681a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.233+0000 7f946e000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9468119250 con 0x7f9468075e60 2024-09-10T22:20:20.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.233+0000 7f946e000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94681190d0 con 0x7f9468113af0 2024-09-10T22:20:20.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.233+0000 7f946e000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94681193d0 con 0x7f9468075540 2024-09-10T22:20:20.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.234+0000 7f9467e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9468113af0 0x7f94681a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.234+0000 7f9467e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9468113af0 0x7f94681a50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:34104/0 (socket says 172.21.15.90:34104) 2024-09-10T22:20:20.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.234+0000 7f9467e00640 1 -- 172.21.15.90:0/331280728 learned_addr learned my addr 172.21.15.90:0/331280728 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:20.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.234+0000 7f9467400640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9468075540 0x7f946819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.235+0000 7f9466a00640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468075e60 0x7f946819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.235+0000 7f9467400640 1 -- 172.21.15.90:0/331280728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9468113af0 msgr2=0x7f94681a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:20.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.235+0000 7f9467400640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9468113af0 0x7f94681a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.235+0000 7f9467400640 1 -- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468075e60 msgr2=0x7f946819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.235+0000 7f9467400640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468075e60 0x7f946819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.235+0000 7f9467400640 1 -- 172.21.15.90:0/331280728 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94681a5890 con 0x7f9468075540 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.235+0000 7f9466a00640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468075e60 0x7f946819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.236+0000 7f9467400640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9468075540 0x7f946819f7e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f945800e9f0 tx=0x7f945800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.236+0000 7f945fe00640 1 -- 172.21.15.90:0/331280728 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945800cd80 con 0x7f9468075540 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.236+0000 7f945fe00640 1 -- 172.21.15.90:0/331280728 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94580035f0 con 0x7f9468075540 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.236+0000 7f945fe00640 1 -- 172.21.15.90:0/331280728 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458010690 con 0x7f9468075540 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.236+0000 7f946e000640 1 -- 172.21.15.90:0/331280728 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94681a5b80 con 0x7f9468075540 2024-09-10T22:20:20.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.236+0000 7f946e000640 1 -- 172.21.15.90:0/331280728 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94681ac240 con 0x7f9468075540 2024-09-10T22:20:20.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.241+0000 7f9470a8b740 1 Processor -- start 2024-09-10T22:20:20.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.241+0000 7f945fe00640 1 -- 172.21.15.90:0/331280728 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9458004020 con 0x7f9468075540 2024-09-10T22:20:20.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.241+0000 7f9470a8b740 1 -- start start 2024-09-10T22:20:20.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.242+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.242+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x5589418884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.242+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 0x55894188ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.242+0000 7f9470a8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5589417f1ca0 con 0x558941888a30 2024-09-10T22:20:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.242+0000 7f9470a8b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5589417e10d0 con 0x55894187c180 2024-09-10T22:20:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.242+0000 7f9470a8b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5589417e4ee0 con 0x55894187caa0 2024-09-10T22:20:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.243+0000 7f9467e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.243+0000 7f9467e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:34120/0 (socket says 172.21.15.90:34120) 2024-09-10T22:20:20.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.243+0000 7f9467e00640 1 -- 172.21.15.90:0/3762678568 learned_addr learned my addr 172.21.15.90:0/3762678568 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:20.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.243+0000 7f9467400640 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x5589418884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.243+0000 7f9467400640 1 -- 172.21.15.90:0/3762678568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 msgr2=0x55894187c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:20.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.243+0000 7f9467400640 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.243+0000 7f9467400640 1 -- 172.21.15.90:0/3762678568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 msgr2=0x55894188ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:20.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.244+0000 7f9466a00640 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 0x55894188ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.244+0000 7f9467400640 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 0x55894188ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.244+0000 7f9467400640 1 -- 172.21.15.90:0/3762678568 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558941793400 con 0x55894187caa0 2024-09-10T22:20:20.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.244+0000 7f9466a00640 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 0x55894188ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:20:20.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.244+0000 7f9467400640 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x5589418884f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f94580e3940 tx=0x7f94580ecef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15d4ca21b2f22be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:20.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.245+0000 7f9453e00640 1 -- 172.21.15.90:0/3762678568 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94580f5070 con 0x55894187caa0 2024-09-10T22:20:20.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.245+0000 7f9453e00640 1 -- 172.21.15.90:0/3762678568 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94580e6e20 con 0x55894187caa0 2024-09-10T22:20:20.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.245+0000 7f9453e00640 1 -- 172.21.15.90:0/3762678568 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94580f1420 con 0x55894187caa0 2024-09-10T22:20:20.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.246+0000 7f945fe00640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f944c0b55a0 0x7f944c0b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.246+0000 7f945fe00640 1 -- 172.21.15.90:0/331280728 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9490+0+0 (secure 0 0 0) 0x7f94580e6620 con 0x7f9468075540 2024-09-10T22:20:20.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.247+0000 7f9470a8b740 1 -- 172.21.15.90:0/3762678568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 msgr2=0x5589418884f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:20.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.247+0000 7f9470a8b740 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x5589418884f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f94580e3940 tx=0x7f94580ecef0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.247+0000 7f9467e00640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f944c0b55a0 0x7f944c0b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.247+0000 7f9470a8b740 1 -- 172.21.15.90:0/3762678568 shutdown_connections 2024-09-10T22:20:20.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.247+0000 7f9470a8b740 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 0x55894188ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.247+0000 7f9470a8b740 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x5589418884f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.247+0000 7f9470a8b740 1 --2- 172.21.15.90:0/3762678568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.247+0000 7f9470a8b740 1 -- 172.21.15.90:0/3762678568 >> 172.21.15.90:0/3762678568 conn(0x558941879fb0 msgr2=0x55894187b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:20.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.249+0000 7f9467e00640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f944c0b55a0 0x7f944c0b7a60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9460004af0 tx=0x7f9460004a80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:20.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.249+0000 7f9470a8b740 1 -- 172.21.15.90:0/3762678568 shutdown_connections 2024-09-10T22:20:20.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.249+0000 7f9470a8b740 1 -- 172.21.15.90:0/3762678568 wait complete. 2024-09-10T22:20:20.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.249+0000 7f9470a8b740 1 -- start start 2024-09-10T22:20:20.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.250+0000 7f9470a8b740 1 Processor -- start 2024-09-10T22:20:20.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.250+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187c180 0x558941888320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941885770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558941888a30 0x558941885da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9470a8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5589416427a0 con 0x55894187c180 2024-09-10T22:20:20.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9470a8b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558941786a10 con 0x558941888a30 2024-09-10T22:20:20.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9470a8b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55894188b6b0 con 0x55894187caa0 2024-09-10T22:20:20.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9470a8b740 10 client.0 _ll_register_callbacks cb 0x558941796be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:20:20.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9466a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941885770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9466a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941885770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:36472/0 (socket says 172.21.15.90:36472) 2024-09-10T22:20:20.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9466a00640 1 -- 172.21.15.90:0/2044035825 learned_addr learned my addr 172.21.15.90:0/2044035825 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:20.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9467e00640 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558941888a30 0x558941885da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.251+0000 7f9467400640 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187c180 0x558941888320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.252+0000 7f9466a00640 1 -- 172.21.15.90:0/2044035825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558941888a30 msgr2=0x558941885da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:20.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.252+0000 7f9466a00640 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558941888a30 0x558941885da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.252+0000 7f9466a00640 1 -- 172.21.15.90:0/2044035825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187c180 msgr2=0x558941888320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:20.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.252+0000 7f9466a00640 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187c180 0x558941888320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.252+0000 7f9466a00640 1 -- 172.21.15.90:0/2044035825 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55894154eb90 con 0x55894187caa0 2024-09-10T22:20:20.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.252+0000 7f9466a00640 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941885770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9454011660 tx=0x7f9454011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf3d9ce12f4d0a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:20.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.252+0000 7f9450c00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:20:20.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.252+0000 7f9450c00640 1 -- 172.21.15.90:0/2044035825 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454004220 con 0x55894187caa0 2024-09-10T22:20:20.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.253+0000 7f9450c00640 1 -- 172.21.15.90:0/2044035825 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9490+0+0 (secure 0 0 0) 0x7f94540043c0 con 0x55894187caa0 2024-09-10T22:20:20.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.253+0000 7f9470a8b740 1 -- 172.21.15.90:0/2044035825 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94580e6e20 con 0x55894187caa0 2024-09-10T22:20:20.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.253+0000 7f9450c00640 1 -- 172.21.15.90:0/2044035825 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9454011e60 con 0x55894187caa0 2024-09-10T22:20:20.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.253+0000 7f9470a8b740 10 client.25909 fetch_fsmap learned FSMap version 55 2024-09-10T22:20:20.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.253+0000 7f9470a8b740 1 -- 172.21.15.90:0/2044035825 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558941896fe0 con 0x55894187caa0 2024-09-10T22:20:20.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.254+0000 7f9450c00640 1 -- 172.21.15.90:0/2044035825 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f94580e6e20 con 0x55894187caa0 2024-09-10T22:20:20.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.254+0000 7f9450c00640 1 -- 172.21.15.90:0/2044035825 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 55) ==== 8450+0+0 (secure 0 0 0) 0x7f9454003ba0 con 0x55894187caa0 2024-09-10T22:20:20.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.254+0000 7f9470a8b740 10 client.25909 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T22:20:20.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.254+0000 7f9470a8b740 10 client.25909 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:20:20.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.255+0000 7f9470a8b740 20 client.25909 populate_metadata read hostname 'smithi090' 2024-09-10T22:20:20.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.255+0000 7f9470a8b740 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x558941898590 0x55894189aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.255+0000 7f9470a8b740 4 client.25909 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:20:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.255+0000 7f9470a8b740 1 -- 172.21.15.90:0/2044035825 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589417f3bf0 con 0x558941898590 2024-09-10T22:20:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.255+0000 7f9467400640 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x558941898590 0x55894189aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.256+0000 7f9467400640 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x558941898590 0x55894189aa10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f6037344e22ab4 server_cookie=9f246be4a3bfcdfb in_seq=0 out_seq=0 2024-09-10T22:20:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.256+0000 7f9450c00640 10 client.25909 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:20:20.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.257+0000 7f9450c00640 1 -- 172.21.15.90:0/2044035825 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f94580b5e50 con 0x558941898590 2024-09-10T22:20:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.257+0000 7f9450c00640 10 client.25909 handle_command_reply: tid=0 2024-09-10T22:20:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.257+0000 7f9470a8b740 1 client.25909 shutdown 2024-09-10T22:20:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.258+0000 7f9470a8b740 1 -- 172.21.15.90:0/2044035825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 msgr2=0x558941885770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.258+0000 7f9470a8b740 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941885770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9454011660 tx=0x7f9454011b20 comp rx=0 tx=0).stop 2024-09-10T22:20:20.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.258+0000 7f9470a8b740 1 -- 172.21.15.90:0/2044035825 shutdown_connections 2024-09-10T22:20:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.258+0000 7f9470a8b740 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x558941898590 0x55894189aa10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.258+0000 7f9470a8b740 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x558941888a30 0x558941885da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.258+0000 7f9470a8b740 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941885770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.258+0000 7f9470a8b740 1 --2- 172.21.15.90:0/2044035825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187c180 0x558941888320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.258+0000 7f9470a8b740 1 -- 172.21.15.90:0/2044035825 >> 172.21.15.90:0/2044035825 conn(0x558941879fb0 msgr2=0x55894187fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.258+0000 7f9450c00640 0 client.25909 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:20:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.259+0000 7f9470a8b740 1 -- 172.21.15.90:0/2044035825 shutdown_connections 2024-09-10T22:20:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.259+0000 7f9470a8b740 1 -- 172.21.15.90:0/2044035825 wait complete. 2024-09-10T22:20:20.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.259+0000 7f9470a8b740 20 client.25909 trim_cache size 0 max 16384 2024-09-10T22:20:20.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.291+0000 7f9470a8b740 1 Processor -- start 2024-09-10T22:20:20.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.291+0000 7f9470a8b740 1 -- start start 2024-09-10T22:20:20.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.292+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.292+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187caa0 0x558941883ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.292+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558941888a30 0x5589418874b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.292+0000 7f9470a8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b64f0 con 0x55894187caa0 2024-09-10T22:20:20.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.292+0000 7f9470a8b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b8330 con 0x55894187c180 2024-09-10T22:20:20.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.292+0000 7f9470a8b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b6c80 con 0x558941888a30 2024-09-10T22:20:20.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.292+0000 7f9467400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.292+0000 7f9467400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:34150/0 (socket says 172.21.15.90:34150) 2024-09-10T22:20:20.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.292+0000 7f9467400640 1 -- 172.21.15.90:0/1178627169 learned_addr learned my addr 172.21.15.90:0/1178627169 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:20.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.293+0000 7f9467e00640 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558941888a30 0x5589418874b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.293+0000 7f9466a00640 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187caa0 0x558941883ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.293+0000 7f9467400640 1 -- 172.21.15.90:0/1178627169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558941888a30 msgr2=0x5589418874b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:20.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.293+0000 7f9467400640 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558941888a30 0x5589418874b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.293+0000 7f9467400640 1 -- 172.21.15.90:0/1178627169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187caa0 msgr2=0x558941883ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:20.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.293+0000 7f9467400640 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187caa0 0x558941883ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.293+0000 7f9467400640 1 -- 172.21.15.90:0/1178627169 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589418810c0 con 0x55894187c180 2024-09-10T22:20:20.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.293+0000 7f9467e00640 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558941888a30 0x5589418874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:20:20.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.293+0000 7f9466a00640 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187caa0 0x558941883ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:20:20.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.294+0000 7f9467400640 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187b6e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f94580a8c40 tx=0x7f9458003d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78141c190032ada4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:20.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.295+0000 7f9453400640 1 -- 172.21.15.90:0/1178627169 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94580f5070 con 0x55894187c180 2024-09-10T22:20:20.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.295+0000 7f9453400640 1 -- 172.21.15.90:0/1178627169 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5589417f3bf0 con 0x55894187c180 2024-09-10T22:20:20.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.295+0000 7f9453400640 1 -- 172.21.15.90:0/1178627169 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94580ac180 con 0x55894187c180 2024-09-10T22:20:20.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.296+0000 7f9470a8b740 1 -- 172.21.15.90:0/1178627169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 msgr2=0x55894187b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:20.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.296+0000 7f9470a8b740 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187b6e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f94580a8c40 tx=0x7f9458003d10 comp rx=0 tx=0).stop 2024-09-10T22:20:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.299+0000 7f9470a8b740 1 -- 172.21.15.90:0/1178627169 shutdown_connections 2024-09-10T22:20:20.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.299+0000 7f9470a8b740 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x558941888a30 0x5589418874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.299+0000 7f9470a8b740 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55894187caa0 0x558941883ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.299+0000 7f9470a8b740 1 --2- 172.21.15.90:0/1178627169 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x55894187b6e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.299+0000 7f9470a8b740 1 -- 172.21.15.90:0/1178627169 >> 172.21.15.90:0/1178627169 conn(0x558941879fb0 msgr2=0x558941886da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.299+0000 7f9470a8b740 1 -- 172.21.15.90:0/1178627169 shutdown_connections 2024-09-10T22:20:20.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.299+0000 7f9470a8b740 1 -- 172.21.15.90:0/1178627169 wait complete. 2024-09-10T22:20:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.299+0000 7f9470a8b740 1 -- start start 2024-09-10T22:20:20.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.300+0000 7f9470a8b740 1 Processor -- start 2024-09-10T22:20:20.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.301+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x558941890970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.301+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941890eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.302+0000 7f9470a8b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 0x558941881fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.302+0000 7f9470a8b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b7ba0 con 0x558941888a30 2024-09-10T22:20:20.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.302+0000 7f9470a8b740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b7410 con 0x55894187c180 2024-09-10T22:20:20.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.302+0000 7f9470a8b740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b4510 con 0x55894187caa0 2024-09-10T22:20:20.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.302+0000 7f9470a8b740 10 client.0 _ll_register_callbacks cb 0x7f9454009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.302+0000 7f9467400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x558941890970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.302+0000 7f9466a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941890eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.302+0000 7f9466a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941890eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:36490/0 (socket says 172.21.15.90:36490) 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.302+0000 7f9466a00640 1 -- 172.21.15.90:0/4377155 learned_addr learned my addr 172.21.15.90:0/4377155 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.303+0000 7f9466a00640 1 -- 172.21.15.90:0/4377155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 msgr2=0x558941890970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.303+0000 7f9466a00640 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x558941890970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.303+0000 7f9466a00640 1 -- 172.21.15.90:0/4377155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 msgr2=0x558941881fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.303+0000 7f9466a00640 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 0x558941881fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.303+0000 7f9466a00640 1 -- 172.21.15.90:0/4377155 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589418846d0 con 0x55894187caa0 2024-09-10T22:20:20.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.303+0000 7f9467400640 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x558941890970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:20:20.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.304+0000 7f9466a00640 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941890eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9454009770 tx=0x7f9454005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e2ec23a871d79e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:20.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.304+0000 7f9450c00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:20:20.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.304+0000 7f9450c00640 1 -- 172.21.15.90:0/4377155 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94540129f0 con 0x55894187caa0 2024-09-10T22:20:20.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.305+0000 7f9450c00640 1 -- 172.21.15.90:0/4377155 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9490+0+0 (secure 0 0 0) 0x7f9454012b90 con 0x55894187caa0 2024-09-10T22:20:20.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.305+0000 7f9470a8b740 1 -- 172.21.15.90:0/4377155 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589417f3bf0 con 0x55894187caa0 2024-09-10T22:20:20.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.305+0000 7f9450c00640 1 -- 172.21.15.90:0/4377155 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f945403b050 con 0x55894187caa0 2024-09-10T22:20:20.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.306+0000 7f9470a8b740 10 client.25915 fetch_fsmap learned FSMap version 55 2024-09-10T22:20:20.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.306+0000 7f9470a8b740 1 -- 172.21.15.90:0/4377155 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55894187c8d0 con 0x55894187caa0 2024-09-10T22:20:20.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.306+0000 7f9450c00640 1 -- 172.21.15.90:0/4377155 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5589417f3bf0 con 0x55894187caa0 2024-09-10T22:20:20.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.307+0000 7f9450c00640 1 -- 172.21.15.90:0/4377155 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 55) ==== 8450+0+0 (secure 0 0 0) 0x7f94540035f0 con 0x55894187caa0 2024-09-10T22:20:20.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.307+0000 7f9470a8b740 10 client.25915 fetch_fsmap finished waiting for FSMap version 55 2024-09-10T22:20:20.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.307+0000 7f9470a8b740 10 client.25915 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:20:20.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.307+0000 7f9470a8b740 20 client.25915 populate_metadata read hostname 'smithi090' 2024-09-10T22:20:20.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.307+0000 7f9470a8b740 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5589419071b0 0x5589419095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:20.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.307+0000 7f9470a8b740 4 client.25915 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.8, "await_for": 45.0}] 2024-09-10T22:20:20.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.307+0000 7f9470a8b740 1 -- 172.21.15.90:0/4377155 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.8, "await_for": 45.0}) -- 0x5589417ef6a0 con 0x5589419071b0 2024-09-10T22:20:20.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.308+0000 7f9467400640 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5589419071b0 0x5589419095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:20.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.309+0000 7f9467400640 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5589419071b0 0x5589419095d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e52314732fbf10d server_cookie=a9d2dbf77d85f76e in_seq=0 out_seq=0 2024-09-10T22:20:20.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.309+0000 7f9450c00640 10 client.25915 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:20:20.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:19 smithi171 ceph-mon[34138]: pgmap v933: 129 pgs: 129 active+clean; 637 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-09-10T22:20:20.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9450c00640 1 -- 172.21.15.90:0/4377155 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5589417ef6a0 con 0x5589419071b0 2024-09-10T22:20:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9450c00640 10 client.25915 handle_command_reply: tid=0 2024-09-10T22:20:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9470a8b740 1 client.25915 shutdown 2024-09-10T22:20:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9470a8b740 1 -- 172.21.15.90:0/4377155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 msgr2=0x558941890eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9470a8b740 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941890eb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9454009770 tx=0x7f9454005c50 comp rx=0 tx=0).stop 2024-09-10T22:20:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9470a8b740 1 -- 172.21.15.90:0/4377155 shutdown_connections 2024-09-10T22:20:20.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9470a8b740 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5589419071b0 0x5589419095d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9470a8b740 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x558941888a30 0x558941881fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9470a8b740 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55894187caa0 0x558941890eb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9470a8b740 1 --2- 172.21.15.90:0/4377155 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55894187c180 0x558941890970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9450c00640 0 client.25915 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:20:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.713+0000 7f9470a8b740 1 -- 172.21.15.90:0/4377155 >> 172.21.15.90:0/4377155 conn(0x558941879fb0 msgr2=0x5589418875c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.714+0000 7f9470a8b740 1 -- 172.21.15.90:0/4377155 shutdown_connections 2024-09-10T22:20:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.714+0000 7f9470a8b740 1 -- 172.21.15.90:0/4377155 wait complete. 2024-09-10T22:20:20.717 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.714+0000 7f9470a8b740 20 client.25915 trim_cache size 0 max 16384 2024-09-10T22:20:20.718 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:20:20.718 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 55, 2024-09-10T22:20:20.718 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:20:20.718 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 50, 2024-09-10T22:20:20.718 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:20:20.718 INFO:teuthology.orchestra.run.smithi090.stdout: "d927a075": { 2024-09-10T22:20:20.719 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 50, 2024-09-10T22:20:20.719 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:20:20.719 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:20:20.719 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:20:20.719 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:20:20.719 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:20:20.719 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:20:20.719 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 154.8, 2024-09-10T22:20:20.719 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:20:20.720 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:20:20.720 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:20:20.720 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:20:20.720 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:20:20.720 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:20:20.720 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:20.720 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:20.720 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:20.720 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:20.721 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:20.721 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:20:20.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 -- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f944c0b55a0 msgr2=0x7f944c0b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:20.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f944c0b55a0 0x7f944c0b7a60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9460004af0 tx=0x7f9460004a80 comp rx=0 tx=0).stop 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 -- 172.21.15.90:0/331280728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9468075540 msgr2=0x7f946819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9468075540 0x7f946819f7e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f945800e9f0 tx=0x7f945800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 -- 172.21.15.90:0/331280728 shutdown_connections 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f944c0b55a0 0x7f944c0b7a60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9468113af0 0x7f94681a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9468075e60 0x7f946819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 --2- 172.21.15.90:0/331280728 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9468075540 0x7f946819f7e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.715+0000 7f946e000640 1 -- 172.21.15.90:0/331280728 >> 172.21.15.90:0/331280728 conn(0x7f94680fe820 msgr2=0x7f94680ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.716+0000 7f946e000640 1 -- 172.21.15.90:0/331280728 shutdown_connections 2024-09-10T22:20:20.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:20.716+0000 7f946e000640 1 -- 172.21.15.90:0/331280728 wait complete. 2024-09-10T22:20:20.869 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19093 6.79 MB/sec execute 71 sec latency 559.105 ms 2024-09-10T22:20:21.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 6.70 MB/sec execute 72 sec latency 1559.193 ms 2024-09-10T22:20:21.995 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd927a075', quiesce took 5.9 seconds. Will release after: 28.9 2024-09-10T22:20:22.176 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:19 smithi154 ceph-mon[34495]: pgmap v933: 129 pgs: 129 active+clean; 637 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-09-10T22:20:22.177 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:21 smithi154 ceph-mon[34495]: pgmap v934: 129 pgs: 129 active+clean; 637 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 247 op/s 2024-09-10T22:20:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:21 smithi171 ceph-mon[34138]: pgmap v934: 129 pgs: 129 active+clean; 637 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 247 op/s 2024-09-10T22:20:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:21 smithi090 ceph-mon[30350]: pgmap v934: 129 pgs: 129 active+clean; 637 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 247 op/s 2024-09-10T22:20:22.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 6.60 MB/sec execute 73 sec latency 2559.281 ms 2024-09-10T22:20:23.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 6.51 MB/sec execute 74 sec latency 3559.352 ms 2024-09-10T22:20:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:23 smithi090 ceph-mon[30350]: pgmap v935: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-09-10T22:20:24.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:23 smithi154 ceph-mon[34495]: pgmap v935: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-09-10T22:20:24.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:23 smithi171 ceph-mon[34138]: pgmap v935: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 252 op/s 2024-09-10T22:20:24.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 6.43 MB/sec execute 75 sec latency 4559.446 ms 2024-09-10T22:20:25.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 6.34 MB/sec execute 76 sec latency 5559.540 ms 2024-09-10T22:20:26.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:25 smithi154 ceph-mon[34495]: pgmap v936: 129 pgs: 129 active+clean; 633 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 236 op/s 2024-09-10T22:20:26.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:25 smithi171 ceph-mon[34138]: pgmap v936: 129 pgs: 129 active+clean; 633 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 236 op/s 2024-09-10T22:20:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:25 smithi090 ceph-mon[30350]: pgmap v936: 129 pgs: 129 active+clean; 633 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 236 op/s 2024-09-10T22:20:26.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 6.26 MB/sec execute 77 sec latency 6559.663 ms 2024-09-10T22:20:27.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 6.18 MB/sec execute 78 sec latency 7559.810 ms 2024-09-10T22:20:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:27 smithi090 ceph-mon[30350]: pgmap v937: 129 pgs: 129 active+clean; 633 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-09-10T22:20:28.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:27 smithi154 ceph-mon[34495]: pgmap v937: 129 pgs: 129 active+clean; 633 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-09-10T22:20:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:27 smithi171 ceph-mon[34138]: pgmap v937: 129 pgs: 129 active+clean; 633 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-09-10T22:20:28.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 6.10 MB/sec execute 79 sec latency 8559.952 ms 2024-09-10T22:20:29.870 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 6.03 MB/sec execute 80 sec latency 9560.046 ms 2024-09-10T22:20:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:29 smithi090 ceph-mon[30350]: pgmap v938: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-10T22:20:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:29 smithi154 ceph-mon[34495]: pgmap v938: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-10T22:20:30.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:29 smithi171 ceph-mon[34138]: pgmap v938: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.9 MiB/s wr, 168 op/s 2024-09-10T22:20:30.871 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.95 MB/sec execute 81 sec latency 10560.175 ms 2024-09-10T22:20:31.871 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.88 MB/sec execute 82 sec latency 11560.311 ms 2024-09-10T22:20:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:31 smithi090 ceph-mon[30350]: pgmap v939: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 991 KiB/s wr, 87 op/s 2024-09-10T22:20:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:32 smithi154 ceph-mon[34495]: pgmap v939: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 991 KiB/s wr, 87 op/s 2024-09-10T22:20:32.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:31 smithi171 ceph-mon[34138]: pgmap v939: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 991 KiB/s wr, 87 op/s 2024-09-10T22:20:32.871 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.81 MB/sec execute 83 sec latency 12560.451 ms 2024-09-10T22:20:33.871 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.74 MB/sec execute 84 sec latency 13560.568 ms 2024-09-10T22:20:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:33 smithi090 ceph-mon[30350]: pgmap v940: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 785 KiB/s wr, 67 op/s 2024-09-10T22:20:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:33 smithi154 ceph-mon[34495]: pgmap v940: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 785 KiB/s wr, 67 op/s 2024-09-10T22:20:34.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:33 smithi171 ceph-mon[34138]: pgmap v940: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 785 KiB/s wr, 67 op/s 2024-09-10T22:20:34.871 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.67 MB/sec execute 85 sec latency 14560.693 ms 2024-09-10T22:20:35.871 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.61 MB/sec execute 86 sec latency 15560.841 ms 2024-09-10T22:20:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:35 smithi090 ceph-mon[30350]: pgmap v941: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 361 KiB/s wr, 46 op/s 2024-09-10T22:20:36.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:35 smithi154 ceph-mon[34495]: pgmap v941: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 361 KiB/s wr, 46 op/s 2024-09-10T22:20:36.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:35 smithi171 ceph-mon[34138]: pgmap v941: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 361 KiB/s wr, 46 op/s 2024-09-10T22:20:36.871 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.54 MB/sec execute 87 sec latency 16560.951 ms 2024-09-10T22:20:37.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.48 MB/sec execute 88 sec latency 17561.069 ms 2024-09-10T22:20:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:37 smithi090 ceph-mon[30350]: pgmap v942: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 153 KiB/s wr, 14 op/s 2024-09-10T22:20:38.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:37 smithi154 ceph-mon[34495]: pgmap v942: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 153 KiB/s wr, 14 op/s 2024-09-10T22:20:38.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:37 smithi171 ceph-mon[34138]: pgmap v942: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 153 KiB/s wr, 14 op/s 2024-09-10T22:20:38.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.42 MB/sec execute 89 sec latency 18561.149 ms 2024-09-10T22:20:39.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.36 MB/sec execute 90 sec latency 19561.294 ms 2024-09-10T22:20:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:39 smithi090 ceph-mon[30350]: pgmap v943: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 120 KiB/s wr, 18 op/s 2024-09-10T22:20:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:40 smithi154 ceph-mon[34495]: pgmap v943: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 120 KiB/s wr, 18 op/s 2024-09-10T22:20:40.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:40 smithi171 ceph-mon[34138]: pgmap v943: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 120 KiB/s wr, 18 op/s 2024-09-10T22:20:40.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.30 MB/sec execute 91 sec latency 20561.422 ms 2024-09-10T22:20:41.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.24 MB/sec execute 92 sec latency 21561.501 ms 2024-09-10T22:20:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:41 smithi090 ceph-mon[30350]: pgmap v944: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-09-10T22:20:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:41 smithi154 ceph-mon[34495]: pgmap v944: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-09-10T22:20:42.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:41 smithi171 ceph-mon[34138]: pgmap v944: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-09-10T22:20:42.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.18 MB/sec execute 93 sec latency 22561.627 ms 2024-09-10T22:20:43.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.13 MB/sec execute 94 sec latency 23561.695 ms 2024-09-10T22:20:44.300 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:44 smithi171 ceph-mon[34138]: pgmap v945: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-10T22:20:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:44 smithi090 ceph-mon[30350]: pgmap v945: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-10T22:20:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:44 smithi154 ceph-mon[34495]: pgmap v945: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 28 KiB/s wr, 11 op/s 2024-09-10T22:20:44.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.07 MB/sec execute 95 sec latency 24561.832 ms 2024-09-10T22:20:45.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 5.02 MB/sec execute 96 sec latency 25561.922 ms 2024-09-10T22:20:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:46 smithi090 ceph-mon[30350]: pgmap v946: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-10T22:20:46.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:46 smithi154 ceph-mon[34495]: pgmap v946: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-10T22:20:46.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:46 smithi171 ceph-mon[34138]: pgmap v946: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-09-10T22:20:46.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 4.97 MB/sec execute 97 sec latency 26562.004 ms 2024-09-10T22:20:47.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 4.92 MB/sec execute 98 sec latency 27562.102 ms 2024-09-10T22:20:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:48 smithi090 ceph-mon[30350]: pgmap v947: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-10T22:20:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:48 smithi154 ceph-mon[34495]: pgmap v947: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-10T22:20:48.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:48 smithi171 ceph-mon[34138]: pgmap v947: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-10T22:20:48.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 4.87 MB/sec execute 99 sec latency 28562.256 ms 2024-09-10T22:20:49.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 4.82 MB/sec execute 100 sec latency 29562.400 ms 2024-09-10T22:20:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:50 smithi090 ceph-mon[30350]: pgmap v948: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-10T22:20:50.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:50 smithi154 ceph-mon[34495]: pgmap v948: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-10T22:20:50.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:50 smithi171 ceph-mon[34138]: pgmap v948: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-10T22:20:50.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 4.77 MB/sec execute 101 sec latency 30562.469 ms 2024-09-10T22:20:50.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:20:51.282 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:20:51.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 4.73 MB/sec execute 102 sec latency 31562.562 ms 2024-09-10T22:20:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.925+0000 7f753d200640 1 -- 172.21.15.90:0/4191780981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538109150 msgr2=0x7f753810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.925+0000 7f753d200640 1 --2- 172.21.15.90:0/4191780981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538109150 0x7f753810f950 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7530009ca0 tx=0x7f753002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:20:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.925+0000 7f753d200640 1 -- 172.21.15.90:0/4191780981 shutdown_connections 2024-09-10T22:20:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.925+0000 7f753d200640 1 --2- 172.21.15.90:0/4191780981 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7538109150 0x7f753810f950 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.925+0000 7f753d200640 1 --2- 172.21.15.90:0/4191780981 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f75381087d0 0x7f7538108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.925+0000 7f753d200640 1 --2- 172.21.15.90:0/4191780981 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7538102dc0 0x7f75381031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.925+0000 7f753d200640 1 -- 172.21.15.90:0/4191780981 >> 172.21.15.90:0/4191780981 conn(0x7f75380fe830 msgr2=0x7f7538100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.926+0000 7f753d200640 1 -- 172.21.15.90:0/4191780981 shutdown_connections 2024-09-10T22:20:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.926+0000 7f753d200640 1 -- 172.21.15.90:0/4191780981 wait complete. 2024-09-10T22:20:51.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.926+0000 7f753d200640 1 Processor -- start 2024-09-10T22:20:51.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f753d200640 1 -- start start 2024-09-10T22:20:51.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f753d200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7538102dc0 0x7f753819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f753d200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75381087d0 0x7f753819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f753d200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7538109150 0x7f75381a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f753d200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7538112d10 con 0x7f75381087d0 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f753d200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7538112b90 con 0x7f7538109150 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f753d200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7538112e90 con 0x7f7538102dc0 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f752fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75381087d0 0x7f753819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f752fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75381087d0 0x7f753819b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47916/0 (socket says 172.21.15.90:47916) 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f752fe00640 1 -- 172.21.15.90:0/3596336919 learned_addr learned my addr 172.21.15.90:0/3596336919 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f7535e00640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7538102dc0 0x7f753819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.927+0000 7f7536800640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7538109150 0x7f75381a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.928+0000 7f752fe00640 1 -- 172.21.15.90:0/3596336919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7538102dc0 msgr2=0x7f753819b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.928+0000 7f752fe00640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7538102dc0 0x7f753819b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.928+0000 7f752fe00640 1 -- 172.21.15.90:0/3596336919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7538109150 msgr2=0x7f75381a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:51.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.928+0000 7f752fe00640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7538109150 0x7f75381a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:51.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.928+0000 7f752fe00640 1 -- 172.21.15.90:0/3596336919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75381a14c0 con 0x7f75381087d0 2024-09-10T22:20:51.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.928+0000 7f752fe00640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75381087d0 0x7f753819b950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f752000d770 tx=0x7f752000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:51.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.928+0000 7f752ea00640 1 -- 172.21.15.90:0/3596336919 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520017070 con 0x7f75381087d0 2024-09-10T22:20:51.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.929+0000 7f752ea00640 1 -- 172.21.15.90:0/3596336919 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7520009820 con 0x7f75381087d0 2024-09-10T22:20:51.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.929+0000 7f752ea00640 1 -- 172.21.15.90:0/3596336919 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7520004b20 con 0x7f75381087d0 2024-09-10T22:20:51.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.929+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75381a17b0 con 0x7f75381087d0 2024-09-10T22:20:51.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.929+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75381a7e70 con 0x7f75381087d0 2024-09-10T22:20:51.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.930+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75381031a0 con 0x7f75381087d0 2024-09-10T22:20:51.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.932+0000 7f752ea00640 1 -- 172.21.15.90:0/3596336919 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7520004cc0 con 0x7f75381087d0 2024-09-10T22:20:51.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.934+0000 7f752ea00640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f751c0b54d0 0x7f751c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:51.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.934+0000 7f752ea00640 1 -- 172.21.15.90:0/3596336919 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9490+0+0 (secure 0 0 0) 0x7f75200e60f0 con 0x7f75381087d0 2024-09-10T22:20:51.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.934+0000 7f7535e00640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f751c0b54d0 0x7f751c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:51.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.935+0000 7f7535e00640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f751c0b54d0 0x7f751c0b7990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7528009990 tx=0x7f7528009110 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:51.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:51.935+0000 7f752ea00640 1 -- 172.21.15.90:0/3596336919 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f75200ac8c0 con 0x7f75381087d0 2024-09-10T22:20:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:52 smithi090 ceph-mon[30350]: pgmap v949: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 10 op/s 2024-09-10T22:20:52.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.163+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f753819c540 con 0x7f75381087d0 2024-09-10T22:20:52.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.163+0000 7f752ea00640 1 -- 172.21.15.90:0/3596336919 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+12021 (secure 0 0 0) 0x7f75200ac1c0 con 0x7f75381087d0 2024-09-10T22:20:52.165 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:20:52.166 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":55,"btime":"2024-09-10T22:19:37:723878+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:19:37.673705+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:20:52.166 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 55 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f751c0b54d0 msgr2=0x7f751c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f751c0b54d0 0x7f751c0b7990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7528009990 tx=0x7f7528009110 comp rx=0 tx=0).stop 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75381087d0 msgr2=0x7f753819b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75381087d0 0x7f753819b950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f752000d770 tx=0x7f752000db70 comp rx=0 tx=0).stop 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 shutdown_connections 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f751c0b54d0 0x7f751c0b7990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7538109150 0x7f75381a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75381087d0 0x7f753819b950 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 --2- 172.21.15.90:0/3596336919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7538102dc0 0x7f753819b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 >> 172.21.15.90:0/3596336919 conn(0x7f75380fe830 msgr2=0x7f7538101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 shutdown_connections 2024-09-10T22:20:52.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:52.166+0000 7f753d200640 1 -- 172.21.15.90:0/3596336919 wait complete. 2024-09-10T22:20:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:52 smithi154 ceph-mon[34495]: pgmap v949: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 10 op/s 2024-09-10T22:20:52.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:52 smithi171 ceph-mon[34138]: pgmap v949: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 10 op/s 2024-09-10T22:20:52.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 4.68 MB/sec execute 103 sec latency 32562.964 ms 2024-09-10T22:20:53.154 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id d927a075 --release --await' 2024-09-10T22:20:53.155 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id d927a075 --release --await 2024-09-10T22:20:53.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3596336919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:20:53.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3596336919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:20:53.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3596336919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:20:53.526 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:20:53.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19097 4.64 MB/sec execute 104 sec latency 33563.091 ms 2024-09-10T22:20:54.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:54 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:20:54.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:54 smithi090 ceph-mon[30350]: pgmap v950: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 10 op/s 2024-09-10T22:20:54.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.146+0000 7f35e7200640 1 -- 172.21.15.90:0/3176115564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e8104df0 msgr2=0x7f35e81051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:54.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.146+0000 7f35e7200640 1 --2- 172.21.15.90:0/3176115564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e8104df0 0x7f35e81051d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009ca0 tx=0x7f35d802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.147+0000 7f35e7200640 1 -- 172.21.15.90:0/3176115564 shutdown_connections 2024-09-10T22:20:54.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.147+0000 7f35e7200640 1 --2- 172.21.15.90:0/3176115564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35e810b180 0x7f35e8111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.147+0000 7f35e7200640 1 --2- 172.21.15.90:0/3176115564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e810a800 0x7f35e810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.147+0000 7f35e7200640 1 --2- 172.21.15.90:0/3176115564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e8104df0 0x7f35e81051d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.147+0000 7f35e7200640 1 -- 172.21.15.90:0/3176115564 >> 172.21.15.90:0/3176115564 conn(0x7f35e8100840 msgr2=0x7f35e8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.147+0000 7f35e7200640 1 -- 172.21.15.90:0/3176115564 shutdown_connections 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.147+0000 7f35e7200640 1 -- 172.21.15.90:0/3176115564 wait complete. 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.148+0000 7f35e7200640 1 Processor -- start 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.148+0000 7f35e7200640 1 -- start start 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.148+0000 7f35e7200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e8104df0 0x7f35e819b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.148+0000 7f35e7200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35e810a800 0x7f35e819b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.148+0000 7f35e7200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e810b180 0x7f35e81a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.148+0000 7f35e7200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8114d40 con 0x7f35e810b180 2024-09-10T22:20:54.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.148+0000 7f35e7200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8114bc0 con 0x7f35e8104df0 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.148+0000 7f35e7200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8114ec0 con 0x7f35e810a800 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.148+0000 7f35e6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e810b180 0x7f35e81a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35e6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e810b180 0x7f35e81a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47934/0 (socket says 172.21.15.90:47934) 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35e6800640 1 -- 172.21.15.90:0/3765013723 learned_addr learned my addr 172.21.15.90:0/3765013723 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35e5e00640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e8104df0 0x7f35e819b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35dd400640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35e810a800 0x7f35e819b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35e6800640 1 -- 172.21.15.90:0/3765013723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35e810a800 msgr2=0x7f35e819b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35e6800640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35e810a800 0x7f35e819b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35e6800640 1 -- 172.21.15.90:0/3765013723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e8104df0 msgr2=0x7f35e819b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35e6800640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e8104df0 0x7f35e819b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35e6800640 1 -- 172.21.15.90:0/3765013723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e81a11a0 con 0x7f35e810b180 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.149+0000 7f35e6800640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e810b180 0x7f35e81a09d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d750 tx=0x7f35e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.150+0000 7f35df400640 1 -- 172.21.15.90:0/3765013723 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0017070 con 0x7f35e810b180 2024-09-10T22:20:54.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.150+0000 7f35df400640 1 -- 172.21.15.90:0/3765013723 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35e00054b0 con 0x7f35e810b180 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.150+0000 7f35e7200640 1 -- 172.21.15.90:0/3765013723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e81a1490 con 0x7f35e810b180 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.150+0000 7f35df400640 1 -- 172.21.15.90:0/3765013723 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0004b90 con 0x7f35e810b180 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.150+0000 7f35e7200640 1 -- 172.21.15.90:0/3765013723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a7b50 con 0x7f35e810b180 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.153+0000 7f35efc57740 1 Processor -- start 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.153+0000 7f35df400640 1 -- 172.21.15.90:0/3765013723 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f35e0004d30 con 0x7f35e810b180 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35efc57740 1 -- start start 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35efc57740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e651d0 0x5601a3e655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35efc57740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e71540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35efc57740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e73e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35efc57740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5601a3ddc290 con 0x5601a3e71a80 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35efc57740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5601a3dcb6c0 con 0x5601a3e65af0 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35efc57740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5601a3dcf4d0 con 0x5601a3e651d0 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35dd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e73e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35dd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e73e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47936/0 (socket says 172.21.15.90:47936) 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35dd400640 1 -- 172.21.15.90:0/2456834433 learned_addr learned my addr 172.21.15.90:0/2456834433 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35dd400640 1 --2- 172.21.15.90:0/2456834433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e651d0 0x5601a3e655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.154+0000 7f35e5e00640 1 --2- 172.21.15.90:0/2456834433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e71540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.155+0000 7f35dd400640 1 -- 172.21.15.90:0/2456834433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e651d0 msgr2=0x5601a3e655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.155+0000 7f35dd400640 1 --2- 172.21.15.90:0/2456834433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e651d0 0x5601a3e655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.155+0000 7f35dd400640 1 -- 172.21.15.90:0/2456834433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 msgr2=0x5601a3e71540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.155+0000 7f35dd400640 1 --2- 172.21.15.90:0/2456834433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e71540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.155+0000 7f35dd400640 1 -- 172.21.15.90:0/2456834433 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a3d58460 con 0x5601a3e71a80 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.155+0000 7f35dd400640 1 --2- 172.21.15.90:0/2456834433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e73e70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f35d00127f0 tx=0x7f35d000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36ab4e0376a93bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.155+0000 7f35d7400640 1 -- 172.21.15.90:0/2456834433 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0012e30 con 0x5601a3e71a80 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.155+0000 7f35d7400640 1 -- 172.21.15.90:0/2456834433 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35d0010c00 con 0x5601a3e71a80 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.155+0000 7f35d7400640 1 -- 172.21.15.90:0/2456834433 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d001e9d0 con 0x5601a3e71a80 2024-09-10T22:20:54.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35df400640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35cc0b5500 0x7f35cc0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35efc57740 1 -- 172.21.15.90:0/2456834433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 msgr2=0x5601a3e73e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35efc57740 1 --2- 172.21.15.90:0/2456834433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e73e70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f35d00127f0 tx=0x7f35d000b7c0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35df400640 1 -- 172.21.15.90:0/3765013723 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9490+0+0 (secure 0 0 0) 0x7f35e00e5df0 con 0x7f35e810b180 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35e5e00640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35cc0b5500 0x7f35cc0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35efc57740 1 -- 172.21.15.90:0/2456834433 shutdown_connections 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35efc57740 1 --2- 172.21.15.90:0/2456834433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e73e70 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35efc57740 1 --2- 172.21.15.90:0/2456834433 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e71540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35efc57740 1 --2- 172.21.15.90:0/2456834433 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e651d0 0x5601a3e655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35efc57740 1 -- 172.21.15.90:0/2456834433 >> 172.21.15.90:0/2456834433 conn(0x5601a3e63000 msgr2=0x5601a3e641a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35efc57740 1 -- 172.21.15.90:0/2456834433 shutdown_connections 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.156+0000 7f35efc57740 1 -- 172.21.15.90:0/2456834433 wait complete. 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35efc57740 1 -- start start 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35e5e00640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35cc0b5500 0x7f35cc0b79c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006010 tx=0x7f35d8031000 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:54.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35efc57740 1 Processor -- start 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35efc57740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e6ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35efc57740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e6efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35efc57740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 0x5601a3e6f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35efc57740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5601a3c2cbb0 con 0x5601a3e651d0 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35efc57740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5601a3d666c0 con 0x5601a3e65af0 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35efc57740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5601a3e74730 con 0x5601a3e71a80 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.157+0000 7f35efc57740 10 client.0 _ll_register_callbacks cb 0x5601a3d81490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e6ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e6ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47948/0 (socket says 172.21.15.90:47948) 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e5e00640 1 -- 172.21.15.90:0/32552061 learned_addr learned my addr 172.21.15.90:0/32552061 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e6800640 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 0x5601a3e6f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35dd400640 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e6efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e6800640 1 -- 172.21.15.90:0/32552061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 msgr2=0x5601a3e6efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e6800640 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e6efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e6800640 1 -- 172.21.15.90:0/32552061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 msgr2=0x5601a3e6ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e6800640 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e6ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e6800640 1 -- 172.21.15.90:0/32552061 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a3b29b00 con 0x5601a3e71a80 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35e6800640 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 0x5601a3e6f4f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f35e00e3110 tx=0x7f35e00ab970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a84c3e0f29ce5728 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.158+0000 7f35cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35cbe00640 1 -- 172.21.15.90:0/32552061 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00ae030 con 0x5601a3e71a80 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35efc57740 1 -- 172.21.15.90:0/32552061 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35d0010c00 con 0x5601a3e71a80 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35cbe00640 1 -- 172.21.15.90:0/32552061 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9490+0+0 (secure 0 0 0) 0x7f35e00930b0 con 0x5601a3e71a80 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35cbe00640 1 -- 172.21.15.90:0/32552061 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f35e0020070 con 0x5601a3e71a80 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35efc57740 10 client.25933 fetch_fsmap learned FSMap version 56 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35efc57740 1 -- 172.21.15.90:0/32552061 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5601a3ba0960 con 0x5601a3e71a80 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35cbe00640 1 -- 172.21.15.90:0/32552061 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35d0010c00 con 0x5601a3e71a80 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35cbe00640 1 -- 172.21.15.90:0/32552061 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f35e0005050 con 0x5601a3e71a80 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35efc57740 10 client.25933 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T22:20:54.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35efc57740 10 client.25933 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35efc57740 20 client.25933 populate_metadata read hostname 'smithi090' 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.159+0000 7f35efc57740 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5601a3e816b0 0x5601a3e83ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.160+0000 7f35efc57740 4 client.25933 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.160+0000 7f35efc57740 1 -- 172.21.15.90:0/32552061 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601a3dde1e0 con 0x5601a3e816b0 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.160+0000 7f35e5e00640 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5601a3e816b0 0x5601a3e83ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.161+0000 7f35cbe00640 10 client.25933 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.161+0000 7f35e5e00640 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5601a3e816b0 0x5601a3e83ac0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a605fd8d0f1cab server_cookie=70a3c7f35eab22ac in_seq=0 out_seq=0 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35cbe00640 1 -- 172.21.15.90:0/32552061 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5601a3dde1e0 con 0x5601a3e816b0 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35cbe00640 10 client.25933 handle_command_reply: tid=0 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 client.25933 shutdown 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 -- 172.21.15.90:0/32552061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 msgr2=0x5601a3e6f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 0x5601a3e6f4f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f35e00e3110 tx=0x7f35e00ab970 comp rx=0 tx=0).stop 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 -- 172.21.15.90:0/32552061 shutdown_connections 2024-09-10T22:20:54.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5601a3e816b0 0x5601a3e83ac0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 0x5601a3e6f4f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35cbe00640 0 client.25933 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e6efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 --2- 172.21.15.90:0/32552061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e6ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 -- 172.21.15.90:0/32552061 >> 172.21.15.90:0/32552061 conn(0x5601a3e63000 msgr2=0x5601a3e68b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 -- 172.21.15.90:0/32552061 shutdown_connections 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 1 -- 172.21.15.90:0/32552061 wait complete. 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.162+0000 7f35efc57740 20 client.25933 trim_cache size 0 max 16384 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.180+0000 7f35efc57740 1 Processor -- start 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.180+0000 7f35efc57740 1 -- start start 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.180+0000 7f35efc57740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e651d0 0x5601a3e6f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.180+0000 7f35efc57740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e65af0 0x5601a3e6f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.180+0000 7f35efc57740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e6bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.180+0000 7f35efc57740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81b1d60 con 0x5601a3e71a80 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.180+0000 7f35efc57740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81b3b40 con 0x5601a3e651d0 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.180+0000 7f35efc57740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81b24f0 con 0x5601a3e65af0 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e6bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e5e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e651d0 0x5601a3e6f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35dd400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e65af0 0x5601a3e6f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e6bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47956/0 (socket says 172.21.15.90:47956) 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e6800640 1 -- 172.21.15.90:0/4260530588 learned_addr learned my addr 172.21.15.90:0/4260530588 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e6800640 1 -- 172.21.15.90:0/4260530588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e65af0 msgr2=0x5601a3e6f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e6800640 1 --2- 172.21.15.90:0/4260530588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e65af0 0x5601a3e6f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e6800640 1 -- 172.21.15.90:0/4260530588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e651d0 msgr2=0x5601a3e6f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e6800640 1 --2- 172.21.15.90:0/4260530588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e651d0 0x5601a3e6f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e6800640 1 -- 172.21.15.90:0/4260530588 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a3e84140 con 0x5601a3e71a80 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e5e00640 1 --2- 172.21.15.90:0/4260530588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e651d0 0x5601a3e6f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:20:54.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.181+0000 7f35e6800640 1 --2- 172.21.15.90:0/4260530588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e6bca0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f35e00a84c0 tx=0x7f35e00ab840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66180c4bd140e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35d6a00640 1 -- 172.21.15.90:0/4260530588 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0014030 con 0x5601a3e71a80 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35d6a00640 1 -- 172.21.15.90:0/4260530588 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35e00b0920 con 0x5601a3e71a80 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35d6a00640 1 -- 172.21.15.90:0/4260530588 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e00ae070 con 0x5601a3e71a80 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35efc57740 1 -- 172.21.15.90:0/4260530588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 msgr2=0x5601a3e6bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35efc57740 1 --2- 172.21.15.90:0/4260530588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e6bca0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f35e00a84c0 tx=0x7f35e00ab840 comp rx=0 tx=0).stop 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35efc57740 1 -- 172.21.15.90:0/4260530588 shutdown_connections 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35efc57740 1 --2- 172.21.15.90:0/4260530588 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e71a80 0x5601a3e6bca0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35efc57740 1 --2- 172.21.15.90:0/4260530588 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e65af0 0x5601a3e6f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35efc57740 1 --2- 172.21.15.90:0/4260530588 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e651d0 0x5601a3e6f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.182+0000 7f35efc57740 1 -- 172.21.15.90:0/4260530588 >> 172.21.15.90:0/4260530588 conn(0x5601a3e63000 msgr2=0x5601a3e6e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.183+0000 7f35efc57740 1 -- 172.21.15.90:0/4260530588 shutdown_connections 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.183+0000 7f35efc57740 1 -- 172.21.15.90:0/4260530588 wait complete. 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.183+0000 7f35efc57740 1 -- start start 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.183+0000 7f35efc57740 1 Processor -- start 2024-09-10T22:20:54.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35efc57740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e76fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35efc57740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35efc57740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 0x5601a3ef6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35efc57740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81b33b0 con 0x5601a3e651d0 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35efc57740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81b2c80 con 0x5601a3e65af0 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35efc57740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f35e81afd60 con 0x5601a3e71a80 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35efc57740 10 client.0 _ll_register_callbacks cb 0x7f35e0005660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35e5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e76fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35e5e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e76fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47960/0 (socket says 172.21.15.90:47960) 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35e5e00640 1 -- 172.21.15.90:0/3436554737 learned_addr learned my addr 172.21.15.90:0/3436554737 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35dd400640 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35e6800640 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 0x5601a3ef6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35e5e00640 1 -- 172.21.15.90:0/3436554737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 msgr2=0x5601a3ef6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:54.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35e5e00640 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 0x5601a3ef6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35e5e00640 1 -- 172.21.15.90:0/3436554737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 msgr2=0x5601a3e774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35e5e00640 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e774e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35e5e00640 1 -- 172.21.15.90:0/3436554737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a3e7d960 con 0x5601a3e651d0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.184+0000 7f35dd400640 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.185+0000 7f35e5e00640 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e76fa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005e30 tx=0x7f35d80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db05f7d35d8e4eca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.185+0000 7f35cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.185+0000 7f35cbe00640 1 -- 172.21.15.90:0/3436554737 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d8003dc0 con 0x5601a3e651d0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.185+0000 7f35cbe00640 1 -- 172.21.15.90:0/3436554737 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9490+0+0 (secure 0 0 0) 0x7f35d8004040 con 0x5601a3e651d0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.185+0000 7f35efc57740 1 -- 172.21.15.90:0/3436554737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e00b0920 con 0x5601a3e651d0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.185+0000 7f35cbe00640 1 -- 172.21.15.90:0/3436554737 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f35d8004240 con 0x5601a3e651d0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.185+0000 7f35efc57740 10 client.16323 fetch_fsmap learned FSMap version 56 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.185+0000 7f35efc57740 1 -- 172.21.15.90:0/3436554737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5601a3ef84b0 con 0x5601a3e651d0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.185+0000 7f35cbe00640 1 -- 172.21.15.90:0/3436554737 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f35e00b0920 con 0x5601a3e651d0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.186+0000 7f35cbe00640 1 -- 172.21.15.90:0/3436554737 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f35d8030b60 con 0x5601a3e651d0 2024-09-10T22:20:54.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.186+0000 7f35efc57740 10 client.16323 fetch_fsmap finished waiting for FSMap version 56 2024-09-10T22:20:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.186+0000 7f35efc57740 10 client.16323 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:20:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.186+0000 7f35efc57740 20 client.16323 populate_metadata read hostname 'smithi090' 2024-09-10T22:20:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.186+0000 7f35efc57740 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5601a3ef8c70 0x5601a3efb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:20:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.186+0000 7f35efc57740 4 client.16323 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d927a075", "release": true, "await": true}] 2024-09-10T22:20:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.186+0000 7f35efc57740 1 -- 172.21.15.90:0/3436554737 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d927a075", "release": true, "await": true}) -- 0x5601a3dd9c90 con 0x5601a3ef8c70 2024-09-10T22:20:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.186+0000 7f35dd400640 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5601a3ef8c70 0x5601a3efb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:20:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.187+0000 7f35dd400640 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5601a3ef8c70 0x5601a3efb090 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d10d7e22558c572 server_cookie=d435bd0257445463 in_seq=0 out_seq=0 2024-09-10T22:20:54.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.187+0000 7f35cbe00640 10 client.16323 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:20:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.223+0000 7f35cbe00640 1 -- 172.21.15.90:0/3436554737 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5601a3dd9c90 con 0x5601a3ef8c70 2024-09-10T22:20:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.223+0000 7f35cbe00640 10 client.16323 handle_command_reply: tid=0 2024-09-10T22:20:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 client.16323 shutdown 2024-09-10T22:20:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 -- 172.21.15.90:0/3436554737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 msgr2=0x5601a3e76fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e76fa0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f35d8005e30 tx=0x7f35d80095c0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 -- 172.21.15.90:0/3436554737 shutdown_connections 2024-09-10T22:20:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5601a3ef8c70 0x5601a3efb090 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x5601a3e71a80 0x5601a3ef6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x5601a3e65af0 0x5601a3e774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 --2- 172.21.15.90:0/3436554737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5601a3e651d0 0x5601a3e76fa0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35cbe00640 0 client.16323 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:20:54.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 -- 172.21.15.90:0/3436554737 >> 172.21.15.90:0/3436554737 conn(0x5601a3e63000 msgr2=0x5601a3e7d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:54.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 -- 172.21.15.90:0/3436554737 shutdown_connections 2024-09-10T22:20:54.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 1 -- 172.21.15.90:0/3436554737 wait complete. 2024-09-10T22:20:54.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.224+0000 7f35efc57740 20 client.16323 trim_cache size 0 max 16384 2024-09-10T22:20:54.227 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:20:54.227 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 56, 2024-09-10T22:20:54.227 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 52, 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "d927a075": { 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 52, 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 154.8, 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:20:54.228 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:20:54.229 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 -- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35cc0b5500 msgr2=0x7f35cc0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35cc0b5500 0x7f35cc0b79c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006010 tx=0x7f35d8031000 comp rx=0 tx=0).stop 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 -- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e810b180 msgr2=0x7f35e81a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e810b180 0x7f35e81a09d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d750 tx=0x7f35e000dc10 comp rx=0 tx=0).stop 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 -- 172.21.15.90:0/3765013723 shutdown_connections 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f35cc0b5500 0x7f35cc0b79c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35e810b180 0x7f35e81a09d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f35e810a800 0x7f35e819b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 --2- 172.21.15.90:0/3765013723 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f35e8104df0 0x7f35e819b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:20:54.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.225+0000 7f35e7200640 1 -- 172.21.15.90:0/3765013723 >> 172.21.15.90:0/3765013723 conn(0x7f35e8100840 msgr2=0x7f35e8102000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:20:54.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.226+0000 7f35e7200640 1 -- 172.21.15.90:0/3765013723 shutdown_connections 2024-09-10T22:20:54.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:20:54.226+0000 7f35e7200640 1 -- 172.21.15.90:0/3765013723 wait complete. 2024-09-10T22:20:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:54 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:20:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:54 smithi154 ceph-mon[34495]: pgmap v950: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 10 op/s 2024-09-10T22:20:54.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:54 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:20:54.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:54 smithi171 ceph-mon[34138]: pgmap v950: 129 pgs: 129 active+clean; 634 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.0 KiB/s wr, 10 op/s 2024-09-10T22:20:54.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19187 4.63 MB/sec execute 105 sec latency 33943.252 ms 2024-09-10T22:20:55.303 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd927a075', total seconds elapsed: 39.2 2024-09-10T22:20:55.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19325 4.67 MB/sec execute 106 sec latency 76.828 ms 2024-09-10T22:20:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:56 smithi090 ceph-mon[30350]: pgmap v951: 129 pgs: 129 active+clean; 635 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 83 KiB/s wr, 21 op/s 2024-09-10T22:20:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:56 smithi154 ceph-mon[34495]: pgmap v951: 129 pgs: 129 active+clean; 635 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 83 KiB/s wr, 21 op/s 2024-09-10T22:20:56.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:56 smithi171 ceph-mon[34138]: pgmap v951: 129 pgs: 129 active+clean; 635 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 83 KiB/s wr, 21 op/s 2024-09-10T22:20:56.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19493 4.71 MB/sec execute 107 sec latency 60.832 ms 2024-09-10T22:20:57.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19635 4.71 MB/sec execute 108 sec latency 70.058 ms 2024-09-10T22:20:58.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:20:58 smithi090 ceph-mon[30350]: pgmap v952: 129 pgs: 129 active+clean; 636 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 216 KiB/s wr, 24 op/s 2024-09-10T22:20:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:20:58 smithi154 ceph-mon[34495]: pgmap v952: 129 pgs: 129 active+clean; 636 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 216 KiB/s wr, 24 op/s 2024-09-10T22:20:58.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:20:58 smithi171 ceph-mon[34138]: pgmap v952: 129 pgs: 129 active+clean; 636 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 216 KiB/s wr, 24 op/s 2024-09-10T22:20:58.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19779 4.70 MB/sec execute 109 sec latency 62.596 ms 2024-09-10T22:20:59.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19927 4.73 MB/sec execute 110 sec latency 56.646 ms 2024-09-10T22:21:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:00 smithi090 ceph-mon[30350]: pgmap v953: 129 pgs: 129 active+clean; 643 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1020 KiB/s wr, 84 op/s 2024-09-10T22:21:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:00 smithi154 ceph-mon[34495]: pgmap v953: 129 pgs: 129 active+clean; 643 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1020 KiB/s wr, 84 op/s 2024-09-10T22:21:00.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:00 smithi171 ceph-mon[34138]: pgmap v953: 129 pgs: 129 active+clean; 643 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1020 KiB/s wr, 84 op/s 2024-09-10T22:21:00.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20042 4.72 MB/sec execute 111 sec latency 85.863 ms 2024-09-10T22:21:01.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20051 4.68 MB/sec execute 112 sec latency 960.058 ms 2024-09-10T22:21:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:02 smithi090 ceph-mon[30350]: pgmap v954: 129 pgs: 129 active+clean; 644 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-09-10T22:21:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:02 smithi154 ceph-mon[34495]: pgmap v954: 129 pgs: 129 active+clean; 644 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-09-10T22:21:02.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:02 smithi171 ceph-mon[34138]: pgmap v954: 129 pgs: 129 active+clean; 644 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-09-10T22:21:02.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20051 4.64 MB/sec execute 113 sec latency 1960.238 ms 2024-09-10T22:21:03.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20090 4.61 MB/sec execute 114 sec latency 2673.154 ms 2024-09-10T22:21:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:04 smithi090 ceph-mon[30350]: pgmap v955: 129 pgs: 129 active+clean; 646 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-09-10T22:21:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:04 smithi154 ceph-mon[34495]: pgmap v955: 129 pgs: 129 active+clean; 646 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-09-10T22:21:04.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:04 smithi171 ceph-mon[34138]: pgmap v955: 129 pgs: 129 active+clean; 646 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-09-10T22:21:04.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20234 4.67 MB/sec execute 115 sec latency 262.988 ms 2024-09-10T22:21:05.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20362 4.71 MB/sec execute 116 sec latency 217.076 ms 2024-09-10T22:21:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:06 smithi090 ceph-mon[30350]: pgmap v956: 129 pgs: 129 active+clean; 651 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 167 op/s 2024-09-10T22:21:06.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:06 smithi154 ceph-mon[34495]: pgmap v956: 129 pgs: 129 active+clean; 651 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 167 op/s 2024-09-10T22:21:06.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:06 smithi171 ceph-mon[34138]: pgmap v956: 129 pgs: 129 active+clean; 651 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 167 op/s 2024-09-10T22:21:06.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20517 4.74 MB/sec execute 117 sec latency 72.012 ms 2024-09-10T22:21:07.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20686 4.83 MB/sec execute 118 sec latency 66.915 ms 2024-09-10T22:21:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:08 smithi090 ceph-mon[30350]: pgmap v957: 129 pgs: 129 active+clean; 657 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-09-10T22:21:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:08 smithi154 ceph-mon[34495]: pgmap v957: 129 pgs: 129 active+clean; 657 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-09-10T22:21:08.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:08 smithi171 ceph-mon[34138]: pgmap v957: 129 pgs: 129 active+clean; 657 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 172 op/s 2024-09-10T22:21:08.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20806 4.83 MB/sec execute 119 sec latency 79.613 ms 2024-09-10T22:21:09.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:09 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:21:09.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:09 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:21:09.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:09 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:21:09.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20947 4.86 MB/sec execute 120 sec latency 91.825 ms 2024-09-10T22:21:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:10 smithi090 ceph-mon[30350]: osdmap e113: 12 total, 12 up, 12 in 2024-09-10T22:21:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:10 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:21:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:10 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:21:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:10 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:21:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:10 smithi090 ceph-mon[30350]: pgmap v959: 129 pgs: 129 active+clean; 671 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2024-09-10T22:21:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:10 smithi154 ceph-mon[34495]: osdmap e113: 12 total, 12 up, 12 in 2024-09-10T22:21:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:10 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:21:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:10 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:21:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:10 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:21:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:10 smithi154 ceph-mon[34495]: pgmap v959: 129 pgs: 129 active+clean; 671 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2024-09-10T22:21:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:10 smithi171 ceph-mon[34138]: osdmap e113: 12 total, 12 up, 12 in 2024-09-10T22:21:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:10 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:21:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:10 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:21:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:10 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:21:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:10 smithi171 ceph-mon[34138]: pgmap v959: 129 pgs: 129 active+clean; 671 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 204 op/s 2024-09-10T22:21:10.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21059 4.91 MB/sec execute 121 sec latency 67.780 ms 2024-09-10T22:21:11.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21157 4.89 MB/sec execute 122 sec latency 74.399 ms 2024-09-10T22:21:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:12 smithi090 ceph-mon[30350]: pgmap v960: 129 pgs: 129 active+clean; 675 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 214 op/s 2024-09-10T22:21:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:12 smithi154 ceph-mon[34495]: pgmap v960: 129 pgs: 129 active+clean; 675 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 214 op/s 2024-09-10T22:21:12.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:12 smithi171 ceph-mon[34138]: pgmap v960: 129 pgs: 129 active+clean; 675 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 214 op/s 2024-09-10T22:21:12.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21304 4.98 MB/sec execute 123 sec latency 293.748 ms 2024-09-10T22:21:13.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21424 4.97 MB/sec execute 124 sec latency 88.493 ms 2024-09-10T22:21:14.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21555 4.98 MB/sec execute 125 sec latency 67.286 ms 2024-09-10T22:21:14.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:14 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:14.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:14 smithi154 ceph-mon[34495]: pgmap v961: 129 pgs: 129 active+clean; 683 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 254 op/s 2024-09-10T22:21:14.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:14 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:14.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:14 smithi171 ceph-mon[34138]: pgmap v961: 129 pgs: 129 active+clean; 683 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 254 op/s 2024-09-10T22:21:15.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:14 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:15.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:14 smithi090 ceph-mon[30350]: pgmap v961: 129 pgs: 129 active+clean; 683 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.9 MiB/s wr, 254 op/s 2024-09-10T22:21:15.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21659 4.96 MB/sec execute 126 sec latency 75.381 ms 2024-09-10T22:21:16.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21770 4.96 MB/sec execute 127 sec latency 66.991 ms 2024-09-10T22:21:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:16 smithi090 ceph-mon[30350]: osdmap e114: 12 total, 12 up, 12 in 2024-09-10T22:21:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:16 smithi090 ceph-mon[30350]: pgmap v963: 129 pgs: 129 active+clean; 684 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 277 op/s 2024-09-10T22:21:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:16 smithi154 ceph-mon[34495]: osdmap e114: 12 total, 12 up, 12 in 2024-09-10T22:21:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:16 smithi154 ceph-mon[34495]: pgmap v963: 129 pgs: 129 active+clean; 684 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 277 op/s 2024-09-10T22:21:17.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:16 smithi171 ceph-mon[34138]: osdmap e114: 12 total, 12 up, 12 in 2024-09-10T22:21:17.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:16 smithi171 ceph-mon[34138]: pgmap v963: 129 pgs: 129 active+clean; 684 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 277 op/s 2024-09-10T22:21:17.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21885 4.96 MB/sec execute 128 sec latency 79.683 ms 2024-09-10T22:21:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:18 smithi090 ceph-mon[30350]: pgmap v964: 129 pgs: 129 active+clean; 691 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 271 op/s 2024-09-10T22:21:18.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:18 smithi154 ceph-mon[34495]: pgmap v964: 129 pgs: 129 active+clean; 691 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 271 op/s 2024-09-10T22:21:18.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:18 smithi171 ceph-mon[34138]: pgmap v964: 129 pgs: 129 active+clean; 691 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 271 op/s 2024-09-10T22:21:18.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22023 4.98 MB/sec execute 129 sec latency 80.569 ms 2024-09-10T22:21:19.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22160 4.99 MB/sec execute 130 sec latency 94.489 ms 2024-09-10T22:21:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:20 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:20 smithi090 ceph-mon[30350]: pgmap v965: 129 pgs: 129 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 278 op/s 2024-09-10T22:21:20.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:20 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:20.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:20 smithi154 ceph-mon[34495]: pgmap v965: 129 pgs: 129 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 278 op/s 2024-09-10T22:21:20.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:20 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:20.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:20 smithi171 ceph-mon[34138]: pgmap v965: 129 pgs: 129 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 278 op/s 2024-09-10T22:21:20.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22305 5.00 MB/sec execute 131 sec latency 73.994 ms 2024-09-10T22:21:21.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22421 4.99 MB/sec execute 132 sec latency 98.607 ms 2024-09-10T22:21:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:22 smithi090 ceph-mon[30350]: pgmap v966: 129 pgs: 129 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 251 op/s 2024-09-10T22:21:22.421 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:22 smithi154 ceph-mon[34495]: pgmap v966: 129 pgs: 129 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 251 op/s 2024-09-10T22:21:22.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:22 smithi171 ceph-mon[34138]: pgmap v966: 129 pgs: 129 active+clean; 694 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 251 op/s 2024-09-10T22:21:22.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22548 5.01 MB/sec execute 133 sec latency 81.140 ms 2024-09-10T22:21:23.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22682 5.04 MB/sec execute 134 sec latency 67.435 ms 2024-09-10T22:21:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:24 smithi090 ceph-mon[30350]: pgmap v967: 129 pgs: 129 active+clean; 694 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 265 op/s 2024-09-10T22:21:24.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:24 smithi154 ceph-mon[34495]: pgmap v967: 129 pgs: 129 active+clean; 694 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 265 op/s 2024-09-10T22:21:24.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:24 smithi171 ceph-mon[34138]: pgmap v967: 129 pgs: 129 active+clean; 694 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.9 MiB/s wr, 265 op/s 2024-09-10T22:21:24.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22819 5.04 MB/sec execute 135 sec latency 74.299 ms 2024-09-10T22:21:25.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22945 5.02 MB/sec execute 136 sec latency 66.423 ms 2024-09-10T22:21:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:26 smithi090 ceph-mon[30350]: pgmap v968: 129 pgs: 129 active+clean; 694 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 292 op/s 2024-09-10T22:21:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:26 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:26.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:26 smithi154 ceph-mon[34495]: pgmap v968: 129 pgs: 129 active+clean; 694 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 292 op/s 2024-09-10T22:21:26.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:26 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:26.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:26 smithi171 ceph-mon[34138]: pgmap v968: 129 pgs: 129 active+clean; 694 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.2 MiB/s wr, 292 op/s 2024-09-10T22:21:26.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:26 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:26.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 23103 5.05 MB/sec execute 137 sec latency 60.405 ms 2024-09-10T22:21:27.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 23241 5.04 MB/sec execute 138 sec latency 71.369 ms 2024-09-10T22:21:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:28 smithi090 ceph-mon[30350]: pgmap v969: 129 pgs: 129 active+clean; 695 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 268 op/s 2024-09-10T22:21:28.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:28 smithi154 ceph-mon[34495]: pgmap v969: 129 pgs: 129 active+clean; 695 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 268 op/s 2024-09-10T22:21:28.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:28 smithi171 ceph-mon[34138]: pgmap v969: 129 pgs: 129 active+clean; 695 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.0 MiB/s wr, 268 op/s 2024-09-10T22:21:28.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 23340 5.05 MB/sec execute 139 sec latency 498.196 ms 2024-09-10T22:21:29.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 23456 5.05 MB/sec execute 140 sec latency 61.064 ms 2024-09-10T22:21:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:30 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:30 smithi090 ceph-mon[30350]: pgmap v970: 129 pgs: 129 active+clean; 701 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-09-10T22:21:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:30 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:30 smithi154 ceph-mon[34495]: pgmap v970: 129 pgs: 129 active+clean; 701 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-09-10T22:21:30.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:30 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:30.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:30 smithi171 ceph-mon[34138]: pgmap v970: 129 pgs: 129 active+clean; 701 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 284 op/s 2024-09-10T22:21:30.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 23601 5.05 MB/sec execute 141 sec latency 60.539 ms 2024-09-10T22:21:31.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 23762 5.13 MB/sec execute 142 sec latency 83.506 ms 2024-09-10T22:21:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:32 smithi090 ceph-mon[30350]: pgmap v971: 129 pgs: 129 active+clean; 702 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-09-10T22:21:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:32 smithi154 ceph-mon[34495]: pgmap v971: 129 pgs: 129 active+clean; 702 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-09-10T22:21:32.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:32 smithi171 ceph-mon[34138]: pgmap v971: 129 pgs: 129 active+clean; 702 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 234 op/s 2024-09-10T22:21:32.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 23921 5.16 MB/sec execute 143 sec latency 78.605 ms 2024-09-10T22:21:33.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24104 5.22 MB/sec execute 144 sec latency 116.395 ms 2024-09-10T22:21:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:34 smithi090 ceph-mon[30350]: pgmap v972: 129 pgs: 129 active+clean; 709 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 286 op/s 2024-09-10T22:21:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:34 smithi154 ceph-mon[34495]: pgmap v972: 129 pgs: 129 active+clean; 709 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 286 op/s 2024-09-10T22:21:34.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:34 smithi171 ceph-mon[34138]: pgmap v972: 129 pgs: 129 active+clean; 709 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.3 MiB/s wr, 286 op/s 2024-09-10T22:21:34.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24175 5.23 MB/sec execute 145 sec latency 500.768 ms 2024-09-10T22:21:35.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24321 5.26 MB/sec execute 146 sec latency 109.891 ms 2024-09-10T22:21:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:36 smithi090 ceph-mon[30350]: pgmap v973: 129 pgs: 129 active+clean; 718 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 269 op/s 2024-09-10T22:21:36.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:36 smithi154 ceph-mon[34495]: pgmap v973: 129 pgs: 129 active+clean; 718 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 269 op/s 2024-09-10T22:21:36.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:36 smithi171 ceph-mon[34138]: pgmap v973: 129 pgs: 129 active+clean; 718 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 269 op/s 2024-09-10T22:21:36.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24444 5.28 MB/sec execute 147 sec latency 84.514 ms 2024-09-10T22:21:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:37 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:37.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:37 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:37.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:37 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:37.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24566 5.29 MB/sec execute 148 sec latency 66.425 ms 2024-09-10T22:21:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:38 smithi090 ceph-mon[30350]: pgmap v974: 129 pgs: 129 active+clean; 719 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 221 op/s 2024-09-10T22:21:38.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:38 smithi154 ceph-mon[34495]: pgmap v974: 129 pgs: 129 active+clean; 719 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 221 op/s 2024-09-10T22:21:38.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:38 smithi171 ceph-mon[34138]: pgmap v974: 129 pgs: 129 active+clean; 719 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.1 MiB/s wr, 221 op/s 2024-09-10T22:21:38.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24722 5.33 MB/sec execute 149 sec latency 71.963 ms 2024-09-10T22:21:39.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24836 5.32 MB/sec execute 150 sec latency 169.877 ms 2024-09-10T22:21:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:40 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:40 smithi090 ceph-mon[30350]: pgmap v975: 129 pgs: 129 active+clean; 727 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 259 op/s 2024-09-10T22:21:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:40 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:40.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:40 smithi154 ceph-mon[34495]: pgmap v975: 129 pgs: 129 active+clean; 727 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 259 op/s 2024-09-10T22:21:40.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:40 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:40.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:40 smithi171 ceph-mon[34138]: pgmap v975: 129 pgs: 129 active+clean; 727 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.0 MiB/s wr, 259 op/s 2024-09-10T22:21:40.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24939 5.30 MB/sec execute 151 sec latency 62.119 ms 2024-09-10T22:21:41.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25051 5.31 MB/sec execute 152 sec latency 112.651 ms 2024-09-10T22:21:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:42 smithi090 ceph-mon[30350]: pgmap v976: 129 pgs: 129 active+clean; 732 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 234 op/s 2024-09-10T22:21:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:42 smithi154 ceph-mon[34495]: pgmap v976: 129 pgs: 129 active+clean; 732 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 234 op/s 2024-09-10T22:21:42.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:42 smithi171 ceph-mon[34138]: pgmap v976: 129 pgs: 129 active+clean; 732 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 234 op/s 2024-09-10T22:21:42.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25115 5.28 MB/sec execute 153 sec latency 839.999 ms 2024-09-10T22:21:43.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25115 5.24 MB/sec execute 154 sec latency 1840.151 ms 2024-09-10T22:21:44.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:44 smithi171 ceph-mon[34138]: pgmap v977: 129 pgs: 129 active+clean; 735 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 249 op/s 2024-09-10T22:21:44.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:44 smithi090 ceph-mon[30350]: pgmap v977: 129 pgs: 129 active+clean; 735 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 249 op/s 2024-09-10T22:21:44.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25169 5.22 MB/sec execute 155 sec latency 2403.583 ms 2024-09-10T22:21:44.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:44 smithi154 ceph-mon[34495]: pgmap v977: 129 pgs: 129 active+clean; 735 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 249 op/s 2024-09-10T22:21:45.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25315 5.24 MB/sec execute 156 sec latency 110.014 ms 2024-09-10T22:21:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:46 smithi090 ceph-mon[30350]: pgmap v978: 129 pgs: 129 active+clean; 733 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 230 op/s 2024-09-10T22:21:46.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:46 smithi154 ceph-mon[34495]: pgmap v978: 129 pgs: 129 active+clean; 733 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 230 op/s 2024-09-10T22:21:46.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:46 smithi171 ceph-mon[34138]: pgmap v978: 129 pgs: 129 active+clean; 733 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 230 op/s 2024-09-10T22:21:46.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25456 5.24 MB/sec execute 157 sec latency 70.948 ms 2024-09-10T22:21:47.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25593 5.27 MB/sec execute 158 sec latency 90.691 ms 2024-09-10T22:21:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:48 smithi090 ceph-mon[30350]: pgmap v979: 129 pgs: 129 active+clean; 730 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 193 op/s 2024-09-10T22:21:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:48 smithi154 ceph-mon[34495]: pgmap v979: 129 pgs: 129 active+clean; 730 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 193 op/s 2024-09-10T22:21:48.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:48 smithi171 ceph-mon[34138]: pgmap v979: 129 pgs: 129 active+clean; 730 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 193 op/s 2024-09-10T22:21:48.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25724 5.26 MB/sec execute 159 sec latency 71.472 ms 2024-09-10T22:21:49.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25790 5.26 MB/sec execute 160 sec latency 553.148 ms 2024-09-10T22:21:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:50 smithi090 ceph-mon[30350]: pgmap v980: 129 pgs: 129 active+clean; 731 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 265 op/s 2024-09-10T22:21:50.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:50 smithi154 ceph-mon[34495]: pgmap v980: 129 pgs: 129 active+clean; 731 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 265 op/s 2024-09-10T22:21:50.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:50 smithi171 ceph-mon[34138]: pgmap v980: 129 pgs: 129 active+clean; 731 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 265 op/s 2024-09-10T22:21:50.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25932 5.30 MB/sec execute 161 sec latency 69.813 ms 2024-09-10T22:21:51.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26050 5.32 MB/sec execute 162 sec latency 74.092 ms 2024-09-10T22:21:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:52 smithi090 ceph-mon[30350]: pgmap v981: 129 pgs: 129 active+clean; 734 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-09-10T22:21:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:52 smithi154 ceph-mon[34495]: pgmap v981: 129 pgs: 129 active+clean; 734 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-09-10T22:21:52.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:52 smithi171 ceph-mon[34138]: pgmap v981: 129 pgs: 129 active+clean; 734 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-09-10T22:21:52.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26176 5.31 MB/sec execute 163 sec latency 76.589 ms 2024-09-10T22:21:53.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26298 5.29 MB/sec execute 164 sec latency 61.129 ms 2024-09-10T22:21:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:54 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:54 smithi090 ceph-mon[30350]: pgmap v982: 129 pgs: 129 active+clean; 737 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-09-10T22:21:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:54 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:54 smithi154 ceph-mon[34495]: pgmap v982: 129 pgs: 129 active+clean; 737 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-09-10T22:21:54.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:54 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:54.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:54 smithi171 ceph-mon[34138]: pgmap v982: 129 pgs: 129 active+clean; 737 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-09-10T22:21:54.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26363 5.27 MB/sec execute 165 sec latency 580.416 ms 2024-09-10T22:21:55.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26481 5.25 MB/sec execute 166 sec latency 85.746 ms 2024-09-10T22:21:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:56 smithi090 ceph-mon[30350]: pgmap v983: 129 pgs: 129 active+clean; 741 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-09-10T22:21:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:56 smithi154 ceph-mon[34495]: pgmap v983: 129 pgs: 129 active+clean; 741 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-09-10T22:21:56.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:56 smithi171 ceph-mon[34138]: pgmap v983: 129 pgs: 129 active+clean; 741 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-09-10T22:21:56.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26623 5.26 MB/sec execute 167 sec latency 79.233 ms 2024-09-10T22:21:57.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26748 5.26 MB/sec execute 168 sec latency 79.454 ms 2024-09-10T22:21:58.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26889 5.27 MB/sec execute 169 sec latency 74.790 ms 2024-09-10T22:21:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:58 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:58 smithi090 ceph-mon[30350]: pgmap v984: 129 pgs: 129 active+clean; 742 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 223 op/s 2024-09-10T22:21:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:58 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:59.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:58 smithi154 ceph-mon[34495]: pgmap v984: 129 pgs: 129 active+clean; 742 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 223 op/s 2024-09-10T22:21:59.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:58 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:21:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:58 smithi171 ceph-mon[34138]: pgmap v984: 129 pgs: 129 active+clean; 742 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.2 MiB/s wr, 223 op/s 2024-09-10T22:21:59.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27019 5.31 MB/sec execute 170 sec latency 63.337 ms 2024-09-10T22:21:59.933 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T22:21:59.933 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.0 2024-09-10T22:21:59.934 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:22:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:21:59 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:21:59 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:21:59 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:00.537 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:22:00.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27142 5.31 MB/sec execute 171 sec latency 99.327 ms 2024-09-10T22:22:01.044 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:00 smithi090 ceph-mon[30350]: pgmap v985: 129 pgs: 129 active+clean; 747 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-09-10T22:22:01.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:00 smithi154 ceph-mon[34495]: pgmap v985: 129 pgs: 129 active+clean; 747 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-09-10T22:22:01.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:00 smithi171 ceph-mon[34138]: pgmap v985: 129 pgs: 129 active+clean; 747 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 270 op/s 2024-09-10T22:22:01.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.404+0000 7f3cece00640 1 -- 172.21.15.90:0/3558634753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce8104e10 msgr2=0x7f3ce81051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:01.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.404+0000 7f3cece00640 1 --2- 172.21.15.90:0/3558634753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce8104e10 0x7f3ce81051f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009900 tx=0x7f3cd802a410 comp rx=0 tx=0).stop 2024-09-10T22:22:01.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.404+0000 7f3cece00640 1 -- 172.21.15.90:0/3558634753 shutdown_connections 2024-09-10T22:22:01.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.404+0000 7f3cece00640 1 --2- 172.21.15.90:0/3558634753 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce810b1a0 0x7f3ce81119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:01.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.404+0000 7f3cece00640 1 --2- 172.21.15.90:0/3558634753 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ce810a820 0x7f3ce810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:01.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.404+0000 7f3cece00640 1 --2- 172.21.15.90:0/3558634753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce8104e10 0x7f3ce81051f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:01.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.404+0000 7f3cece00640 1 -- 172.21.15.90:0/3558634753 >> 172.21.15.90:0/3558634753 conn(0x7f3ce8100840 msgr2=0x7f3ce8102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:01.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.404+0000 7f3cece00640 1 -- 172.21.15.90:0/3558634753 shutdown_connections 2024-09-10T22:22:01.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.404+0000 7f3cece00640 1 -- 172.21.15.90:0/3558634753 wait complete. 2024-09-10T22:22:01.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.406+0000 7f3cece00640 1 Processor -- start 2024-09-10T22:22:01.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.406+0000 7f3cece00640 1 -- start start 2024-09-10T22:22:01.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3cece00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ce8104e10 0x7f3ce819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:01.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3cece00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce810a820 0x7f3ce819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:01.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3cece00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce810b1a0 0x7f3ce81a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:01.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3cece00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce8114cd0 con 0x7f3ce810a820 2024-09-10T22:22:01.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3cece00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce8114b50 con 0x7f3ce810b1a0 2024-09-10T22:22:01.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3cece00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce8114e50 con 0x7f3ce8104e10 2024-09-10T22:22:01.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3ce7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce810b1a0 0x7f3ce81a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:01.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3ce7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce810b1a0 0x7f3ce81a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:49558/0 (socket says 172.21.15.90:49558) 2024-09-10T22:22:01.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3ce7400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ce8104e10 0x7f3ce819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:01.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3ce7e00640 1 -- 172.21.15.90:0/1773328070 learned_addr learned my addr 172.21.15.90:0/1773328070 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:01.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.407+0000 7f3ce6a00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce810a820 0x7f3ce819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:01.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.408+0000 7f3ce7400640 1 -- 172.21.15.90:0/1773328070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce810b1a0 msgr2=0x7f3ce81a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:01.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.408+0000 7f3ce7400640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce810b1a0 0x7f3ce81a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:01.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.408+0000 7f3ce7400640 1 -- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce810a820 msgr2=0x7f3ce819b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:01.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.408+0000 7f3ce7400640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce810a820 0x7f3ce819b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:01.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.408+0000 7f3ce7400640 1 -- 172.21.15.90:0/1773328070 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce81a1210 con 0x7f3ce8104e10 2024-09-10T22:22:01.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.408+0000 7f3ce7e00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce810b1a0 0x7f3ce81a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:22:01.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.408+0000 7f3ce7400640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ce8104e10 0x7f3ce819b160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8005940 tx=0x7f3cd8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:01.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.409+0000 7f3cdfe00640 1 -- 172.21.15.90:0/1773328070 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd803f070 con 0x7f3ce8104e10 2024-09-10T22:22:01.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.409+0000 7f3cdfe00640 1 -- 172.21.15.90:0/1773328070 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3cd8003a80 con 0x7f3ce8104e10 2024-09-10T22:22:01.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.409+0000 7f3cdfe00640 1 -- 172.21.15.90:0/1773328070 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8035070 con 0x7f3ce8104e10 2024-09-10T22:22:01.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.409+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce81a14a0 con 0x7f3ce8104e10 2024-09-10T22:22:01.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.409+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce81a7bc0 con 0x7f3ce8104e10 2024-09-10T22:22:01.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.411+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce81051f0 con 0x7f3ce8104e10 2024-09-10T22:22:01.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.414+0000 7f3cdfe00640 1 -- 172.21.15.90:0/1773328070 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3cd804b060 con 0x7f3ce8104e10 2024-09-10T22:22:01.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.418+0000 7f3cdfe00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3ccc0b5580 0x7f3ccc0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:01.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.418+0000 7f3ce6a00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3ccc0b5580 0x7f3ccc0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:01.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.420+0000 7f3cdfe00640 1 -- 172.21.15.90:0/1773328070 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9490+0+0 (secure 0 0 0) 0x7f3cd81062a0 con 0x7f3ce8104e10 2024-09-10T22:22:01.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.421+0000 7f3ce6a00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3ccc0b5580 0x7f3ccc0b7a40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8104250 tx=0x7f3cd4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:01.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.422+0000 7f3cdfe00640 1 -- 172.21.15.90:0/1773328070 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3cd80cc9f0 con 0x7f3ce8104e10 2024-09-10T22:22:01.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.820+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ce819c290 con 0x7f3ce8104e10 2024-09-10T22:22:01.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.820+0000 7f3cdfe00640 1 -- 172.21.15.90:0/1773328070 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) ==== 94+0+11992 (secure 0 0 0) 0x7f3cd80d51a0 con 0x7f3ce8104e10 2024-09-10T22:22:01.822 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:22:01.822 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":65,"btime":"2024-09-10T22:21:58:810823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":65,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:21:58.787046+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:22:01.822 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 65 2024-09-10T22:22:01.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3ccc0b5580 msgr2=0x7f3ccc0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:01.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3ccc0b5580 0x7f3ccc0b7a40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8104250 tx=0x7f3cd4005f90 comp rx=0 tx=0).stop 2024-09-10T22:22:01.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ce8104e10 msgr2=0x7f3ce819b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:01.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ce8104e10 0x7f3ce819b160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8005940 tx=0x7f3cd8004040 comp rx=0 tx=0).stop 2024-09-10T22:22:01.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 shutdown_connections 2024-09-10T22:22:01.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3ccc0b5580 0x7f3ccc0b7a40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:01.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3ce810b1a0 0x7f3ce81a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:01.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ce810a820 0x7f3ce819b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:01.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 --2- 172.21.15.90:0/1773328070 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3ce8104e10 0x7f3ce819b160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:01.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.824+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 >> 172.21.15.90:0/1773328070 conn(0x7f3ce8100840 msgr2=0x7f3ce8102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:01.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.825+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 shutdown_connections 2024-09-10T22:22:01.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:01.825+0000 7f3cece00640 1 -- 172.21.15.90:0/1773328070 wait complete. 2024-09-10T22:22:01.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27322 5.37 MB/sec execute 172 sec latency 81.525 ms 2024-09-10T22:22:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:01 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1773328070' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:02.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:01 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1773328070' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:02.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:01 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1773328070' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:02.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27410 5.39 MB/sec execute 173 sec latency 588.124 ms 2024-09-10T22:22:03.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:02 smithi090 ceph-mon[30350]: pgmap v986: 129 pgs: 129 active+clean; 749 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-09-10T22:22:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:02 smithi154 ceph-mon[34495]: pgmap v986: 129 pgs: 129 active+clean; 749 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-09-10T22:22:03.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:02 smithi171 ceph-mon[34138]: pgmap v986: 129 pgs: 129 active+clean; 749 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 213 op/s 2024-09-10T22:22:03.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27572 5.40 MB/sec execute 174 sec latency 573.977 ms 2024-09-10T22:22:04.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:03 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:03 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:04.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:03 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:04.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27660 5.39 MB/sec execute 175 sec latency 150.174 ms 2024-09-10T22:22:05.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:04 smithi090 ceph-mon[30350]: osdmap e115: 12 total, 12 up, 12 in 2024-09-10T22:22:05.021 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:04 smithi090 ceph-mon[30350]: pgmap v988: 129 pgs: 129 active+clean; 755 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-09-10T22:22:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:04 smithi154 ceph-mon[34495]: osdmap e115: 12 total, 12 up, 12 in 2024-09-10T22:22:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:04 smithi154 ceph-mon[34495]: pgmap v988: 129 pgs: 129 active+clean; 755 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-09-10T22:22:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:04 smithi171 ceph-mon[34138]: osdmap e115: 12 total, 12 up, 12 in 2024-09-10T22:22:05.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:04 smithi171 ceph-mon[34138]: pgmap v988: 129 pgs: 129 active+clean; 755 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 241 op/s 2024-09-10T22:22:05.383 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0' 2024-09-10T22:22:05.384 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0 2024-09-10T22:22:05.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27789 5.44 MB/sec execute 176 sec latency 86.365 ms 2024-09-10T22:22:06.006 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:22:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:06 smithi090 ceph-mon[30350]: pgmap v989: 129 pgs: 129 active+clean; 762 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-09-10T22:22:06.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:06 smithi154 ceph-mon[34495]: pgmap v989: 129 pgs: 129 active+clean; 762 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-09-10T22:22:06.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:06 smithi171 ceph-mon[34138]: pgmap v989: 129 pgs: 129 active+clean; 762 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-09-10T22:22:06.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27882 5.43 MB/sec execute 177 sec latency 177.119 ms 2024-09-10T22:22:07.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.507+0000 7f4ef3200640 1 -- 172.21.15.90:0/2877176526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ef40776d0 msgr2=0x7f4ef4077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:07.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.507+0000 7f4ef3200640 1 --2- 172.21.15.90:0/2877176526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ef40776d0 0x7f4ef4077ab0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009900 tx=0x7f4ee402a410 comp rx=0 tx=0).stop 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.507+0000 7f4ef3200640 1 -- 172.21.15.90:0/2877176526 shutdown_connections 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.507+0000 7f4ef3200640 1 --2- 172.21.15.90:0/2877176526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ef4113800 0x7f4ef4115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.507+0000 7f4ef3200640 1 --2- 172.21.15.90:0/2877176526 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ef4077ff0 0x7f4ef41132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.507+0000 7f4ef3200640 1 --2- 172.21.15.90:0/2877176526 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ef40776d0 0x7f4ef4077ab0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.507+0000 7f4ef3200640 1 -- 172.21.15.90:0/2877176526 >> 172.21.15.90:0/2877176526 conn(0x7f4ef4100840 msgr2=0x7f4ef4102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.507+0000 7f4ef3200640 1 -- 172.21.15.90:0/2877176526 shutdown_connections 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.507+0000 7f4ef3200640 1 -- 172.21.15.90:0/2877176526 wait complete. 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.508+0000 7f4ef3200640 1 Processor -- start 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.508+0000 7f4ef3200640 1 -- start start 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.509+0000 7f4ef3200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ef40776d0 0x7f4ef4074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.509+0000 7f4ef3200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ef4077ff0 0x7f4ef4074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.509+0000 7f4ef3200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ef4113800 0x7f4ef406dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.509+0000 7f4ef3200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4119110 con 0x7f4ef4113800 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.509+0000 7f4ef3200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4118f90 con 0x7f4ef4077ff0 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.509+0000 7f4ef3200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef4119290 con 0x7f4ef40776d0 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.509+0000 7f4ef1400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ef4077ff0 0x7f4ef4074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.510+0000 7f4ef1400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ef4077ff0 0x7f4ef4074cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:49576/0 (socket says 172.21.15.90:49576) 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.510+0000 7f4ef1400640 1 -- 172.21.15.90:0/3593600059 learned_addr learned my addr 172.21.15.90:0/3593600059 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.510+0000 7f4ef1e00640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ef40776d0 0x7f4ef4074780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.510+0000 7f4ef2800640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ef4113800 0x7f4ef406dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.510+0000 7f4ef1e00640 1 -- 172.21.15.90:0/3593600059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ef4077ff0 msgr2=0x7f4ef4074cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.510+0000 7f4ef1e00640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ef4077ff0 0x7f4ef4074cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.510+0000 7f4ef1e00640 1 -- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ef4113800 msgr2=0x7f4ef406dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.510+0000 7f4ef1e00640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ef4113800 0x7f4ef406dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.510+0000 7f4ef1e00640 1 -- 172.21.15.90:0/3593600059 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef406e3a0 con 0x7f4ef40776d0 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.511+0000 7f4ef1e00640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ef40776d0 0x7f4ef4074780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ee402a930 tx=0x7f4ee4003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.511+0000 7f4eeaa00640 1 -- 172.21.15.90:0/3593600059 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee403f070 con 0x7f4ef40776d0 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.511+0000 7f4eeaa00640 1 -- 172.21.15.90:0/3593600059 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ee403baf0 con 0x7f4ef40776d0 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.511+0000 7f4eeaa00640 1 -- 172.21.15.90:0/3593600059 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4035030 con 0x7f4ef40776d0 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.511+0000 7f4ef3200640 1 -- 172.21.15.90:0/3593600059 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef406e630 con 0x7f4ef40776d0 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.512+0000 7f4ef3200640 1 -- 172.21.15.90:0/3593600059 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef40735c0 con 0x7f4ef40776d0 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.516+0000 7f4efbddd740 1 Processor -- start 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.517+0000 7f4efbddd740 1 -- start start 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.517+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.517+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.517+0000 7f4eeaa00640 1 -- 172.21.15.90:0/3593600059 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4ee403b660 con 0x7f4ef40776d0 2024-09-10T22:22:07.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.517+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.517+0000 7f4efbddd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c40b93fca0 con 0x55c40b9d6ba0 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.517+0000 7f4efbddd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c40b92f0d0 con 0x55c40b9ca2f0 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.517+0000 7f4efbddd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c40b932ee0 con 0x55c40b9cac10 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef2800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef2800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9ca6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:49580/0 (socket says 172.21.15.90:49580) 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef2800640 1 -- 172.21.15.90:0/4209563941 learned_addr learned my addr 172.21.15.90:0/4209563941 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef1e00640 1 --2- 172.21.15.90:0/4209563941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef1400640 1 --2- 172.21.15.90:0/4209563941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef2800640 1 -- 172.21.15.90:0/4209563941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 msgr2=0x55c40b9d6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef2800640 1 --2- 172.21.15.90:0/4209563941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef2800640 1 -- 172.21.15.90:0/4209563941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 msgr2=0x55c40b9d8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef2800640 1 --2- 172.21.15.90:0/4209563941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.518+0000 7f4ef2800640 1 -- 172.21.15.90:0/4209563941 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c40b8e1400 con 0x55c40b9ca2f0 2024-09-10T22:22:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.519+0000 7f4ef2800640 1 --2- 172.21.15.90:0/4209563941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9ca6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009fc0 tx=0x7f4eec010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef1da7bd4ba6314 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.519+0000 7f4edf400640 1 -- 172.21.15.90:0/4209563941 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec016070 con 0x55c40b9ca2f0 2024-09-10T22:22:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.520+0000 7f4edf400640 1 -- 172.21.15.90:0/4209563941 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4eec004d20 con 0x55c40b9ca2f0 2024-09-10T22:22:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.520+0000 7f4edf400640 1 -- 172.21.15.90:0/4209563941 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec005390 con 0x55c40b9ca2f0 2024-09-10T22:22:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.520+0000 7f4eeaa00640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4ed80b5580 0x7f4ed80b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.520+0000 7f4eeaa00640 1 -- 172.21.15.90:0/3593600059 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9578+0+0 (secure 0 0 0) 0x7f4ee4106bf0 con 0x7f4ef40776d0 2024-09-10T22:22:07.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.520+0000 7f4ef1e00640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4ed80b5580 0x7f4ed80b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.521+0000 7f4efbddd740 1 -- 172.21.15.90:0/4209563941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 msgr2=0x55c40b9ca6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:07.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.521+0000 7f4efbddd740 1 --2- 172.21.15.90:0/4209563941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9ca6d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009fc0 tx=0x7f4eec010800 comp rx=0 tx=0).stop 2024-09-10T22:22:07.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.521+0000 7f4efbddd740 1 -- 172.21.15.90:0/4209563941 shutdown_connections 2024-09-10T22:22:07.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.521+0000 7f4efbddd740 1 --2- 172.21.15.90:0/4209563941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.521+0000 7f4efbddd740 1 --2- 172.21.15.90:0/4209563941 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.521+0000 7f4efbddd740 1 --2- 172.21.15.90:0/4209563941 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9ca6d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.521+0000 7f4efbddd740 1 -- 172.21.15.90:0/4209563941 >> 172.21.15.90:0/4209563941 conn(0x55c40b9c8120 msgr2=0x55c40b9c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.521+0000 7f4ef1e00640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4ed80b5580 0x7f4ed80b7a40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4103fd0 tx=0x7f4ee402a410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.522+0000 7f4efbddd740 1 -- 172.21.15.90:0/4209563941 shutdown_connections 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.522+0000 7f4efbddd740 1 -- 172.21.15.90:0/4209563941 wait complete. 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.523+0000 7f4efbddd740 1 -- start start 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.523+0000 7f4efbddd740 1 Processor -- start 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9d64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4efbddd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c40b7907a0 con 0x55c40b9ca2f0 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4efbddd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55c40b8d4a10 con 0x55c40b9d6ba0 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4efbddd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55c40b9d9820 con 0x55c40b9cac10 2024-09-10T22:22:07.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4efbddd740 10 client.0 _ll_register_callbacks cb 0x55c40b8e4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4ef2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9d64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4ef1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4ef2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9d64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34484/0 (socket says 172.21.15.90:34484) 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4ef1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:52622/0 (socket says 172.21.15.90:52622) 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4ef2800640 1 -- 172.21.15.90:0/1300986422 learned_addr learned my addr 172.21.15.90:0/1300986422 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.524+0000 7f4ef2800640 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.525+0000 7f4ef1400640 1 -- 172.21.15.90:0/1300986422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 msgr2=0x55c40b9d3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.525+0000 7f4ef1400640 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.525+0000 7f4ef1400640 1 -- 172.21.15.90:0/1300986422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 msgr2=0x55c40b9d64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.525+0000 7f4ef1400640 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9d64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.525+0000 7f4ef1400640 1 -- 172.21.15.90:0/1300986422 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c40b69cb90 con 0x55c40b9cac10 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.525+0000 7f4ef1400640 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d3900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ef407a150 tx=0x7f4ee0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ae59c34854ec463 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.525+0000 7f4ed3e00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:22:07.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.526+0000 7f4ed3e00640 1 -- 172.21.15.90:0/1300986422 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee000f8c0 con 0x55c40b9cac10 2024-09-10T22:22:07.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.526+0000 7f4efbddd740 1 -- 172.21.15.90:0/1300986422 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eec004d20 con 0x55c40b9cac10 2024-09-10T22:22:07.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.526+0000 7f4ed3e00640 1 -- 172.21.15.90:0/1300986422 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9578+0+0 (secure 0 0 0) 0x7f4ee000fa60 con 0x55c40b9cac10 2024-09-10T22:22:07.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.526+0000 7f4ed3e00640 1 -- 172.21.15.90:0/1300986422 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4ee000fc60 con 0x55c40b9cac10 2024-09-10T22:22:07.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.526+0000 7f4efbddd740 10 client.25957 fetch_fsmap learned FSMap version 66 2024-09-10T22:22:07.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.526+0000 7f4efbddd740 1 -- 172.21.15.90:0/1300986422 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55c40b9e5230 con 0x55c40b9cac10 2024-09-10T22:22:07.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.527+0000 7f4ed3e00640 1 -- 172.21.15.90:0/1300986422 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4eec004d20 con 0x55c40b9cac10 2024-09-10T22:22:07.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.527+0000 7f4ed3e00640 1 -- 172.21.15.90:0/1300986422 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 66) ==== 8382+0+0 (secure 0 0 0) 0x7f4ee001c420 con 0x55c40b9cac10 2024-09-10T22:22:07.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.527+0000 7f4efbddd740 10 client.25957 fetch_fsmap finished waiting for FSMap version 66 2024-09-10T22:22:07.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.527+0000 7f4efbddd740 10 client.25957 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:22:07.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.527+0000 7f4efbddd740 20 client.25957 populate_metadata read hostname 'smithi090' 2024-09-10T22:22:07.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.527+0000 7f4efbddd740 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c40b9e66f0 0x55c40b9e8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.527+0000 7f4efbddd740 4 client.25957 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:22:07.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.527+0000 7f4efbddd740 1 -- 172.21.15.90:0/1300986422 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c40b941bf0 con 0x55c40b9e66f0 2024-09-10T22:22:07.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.527+0000 7f4ef2800640 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c40b9e66f0 0x55c40b9e8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.528+0000 7f4ed3e00640 10 client.25957 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:07.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.528+0000 7f4ef2800640 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c40b9e66f0 0x55c40b9e8b40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5fafb13e941266 server_cookie=d439685eb9063fe in_seq=0 out_seq=0 2024-09-10T22:22:07.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.529+0000 7f4ed3e00640 1 -- 172.21.15.90:0/1300986422 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c40b941bf0 con 0x55c40b9e66f0 2024-09-10T22:22:07.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.529+0000 7f4ed3e00640 10 client.25957 handle_command_reply: tid=0 2024-09-10T22:22:07.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.529+0000 7f4efbddd740 1 client.25957 shutdown 2024-09-10T22:22:07.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.530+0000 7f4efbddd740 1 -- 172.21.15.90:0/1300986422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 msgr2=0x55c40b9d3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:07.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.530+0000 7f4efbddd740 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d3900 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4ef407a150 tx=0x7f4ee0012b60 comp rx=0 tx=0).stop 2024-09-10T22:22:07.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.530+0000 7f4efbddd740 1 -- 172.21.15.90:0/1300986422 shutdown_connections 2024-09-10T22:22:07.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.530+0000 7f4efbddd740 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c40b9e66f0 0x55c40b9e8b40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.530+0000 7f4efbddd740 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.530+0000 7f4efbddd740 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9d3900 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.530+0000 7f4efbddd740 1 --2- 172.21.15.90:0/1300986422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9d64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.530+0000 7f4efbddd740 1 -- 172.21.15.90:0/1300986422 >> 172.21.15.90:0/1300986422 conn(0x55c40b9c8120 msgr2=0x55c40b9cdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:07.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.530+0000 7f4ed3e00640 0 client.25957 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:07.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.531+0000 7f4efbddd740 1 -- 172.21.15.90:0/1300986422 shutdown_connections 2024-09-10T22:22:07.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.531+0000 7f4efbddd740 1 -- 172.21.15.90:0/1300986422 wait complete. 2024-09-10T22:22:07.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.531+0000 7f4efbddd740 20 client.25957 trim_cache size 0 max 16384 2024-09-10T22:22:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.556+0000 7f4efbddd740 1 Processor -- start 2024-09-10T22:22:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.557+0000 7f4efbddd740 1 -- start start 2024-09-10T22:22:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.557+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9c9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.557+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.557+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.557+0000 7f4efbddd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41beb20 con 0x55c40b9ca2f0 2024-09-10T22:22:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.557+0000 7f4efbddd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41c06c0 con 0x55c40b9d6ba0 2024-09-10T22:22:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.557+0000 7f4efbddd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41bf1f0 con 0x55c40b9cac10 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.557+0000 7f4ef1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.557+0000 7f4ef2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.558+0000 7f4ef1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:52624/0 (socket says 172.21.15.90:52624) 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.558+0000 7f4ef2800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.558+0000 7f4ef1400640 1 -- 172.21.15.90:0/70956158 learned_addr learned my addr 172.21.15.90:0/70956158 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.558+0000 7f4ef1400640 1 -- 172.21.15.90:0/70956158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 msgr2=0x55c40b9d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.558+0000 7f4ef1400640 1 --2- 172.21.15.90:0/70956158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.558+0000 7f4ef1400640 1 -- 172.21.15.90:0/70956158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 msgr2=0x55c40b9c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.558+0000 7f4ef1400640 1 --2- 172.21.15.90:0/70956158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.558+0000 7f4ef1400640 1 -- 172.21.15.90:0/70956158 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c40b9d4260 con 0x55c40b9cac10 2024-09-10T22:22:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.558+0000 7f4ef2800640 1 --2- 172.21.15.90:0/70956158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:22:07.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.559+0000 7f4ef1400640 1 --2- 172.21.15.90:0/70956158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9c7af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ee001df90 tx=0x7f4ee000ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a8dfa65542caebb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:07.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.559+0000 7f4edea00640 1 -- 172.21.15.90:0/70956158 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee001c420 con 0x55c40b9cac10 2024-09-10T22:22:07.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.559+0000 7f4edea00640 1 -- 172.21.15.90:0/70956158 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ee001c5c0 con 0x55c40b9cac10 2024-09-10T22:22:07.560 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.559+0000 7f4edea00640 1 -- 172.21.15.90:0/70956158 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00053c0 con 0x55c40b9cac10 2024-09-10T22:22:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.560+0000 7f4efbddd740 1 -- 172.21.15.90:0/70956158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 msgr2=0x55c40b9c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.560+0000 7f4efbddd740 1 --2- 172.21.15.90:0/70956158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9c7af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4ee001df90 tx=0x7f4ee000ff90 comp rx=0 tx=0).stop 2024-09-10T22:22:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.560+0000 7f4efbddd740 1 -- 172.21.15.90:0/70956158 shutdown_connections 2024-09-10T22:22:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.560+0000 7f4efbddd740 1 --2- 172.21.15.90:0/70956158 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9d6ba0 0x55c40b9d0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.560+0000 7f4efbddd740 1 --2- 172.21.15.90:0/70956158 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9c7af0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.560+0000 7f4efbddd740 1 --2- 172.21.15.90:0/70956158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9ca2f0 0x55c40b9c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.560+0000 7f4efbddd740 1 -- 172.21.15.90:0/70956158 >> 172.21.15.90:0/70956158 conn(0x55c40b9c8120 msgr2=0x55c40b9c9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.561+0000 7f4efbddd740 1 -- 172.21.15.90:0/70956158 shutdown_connections 2024-09-10T22:22:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.561+0000 7f4efbddd740 1 -- 172.21.15.90:0/70956158 wait complete. 2024-09-10T22:22:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.562+0000 7f4efbddd740 1 -- start start 2024-09-10T22:22:07.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.562+0000 7f4efbddd740 1 Processor -- start 2024-09-10T22:22:07.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.563+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9dbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.563+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9dc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.563+0000 7f4efbddd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 0x55c40b9dc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.563+0000 7f4efbddd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41bfff0 con 0x55c40b9d6ba0 2024-09-10T22:22:07.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.563+0000 7f4efbddd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41bf8c0 con 0x55c40b9ca2f0 2024-09-10T22:22:07.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.563+0000 7f4efbddd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef41bcb20 con 0x55c40b9cac10 2024-09-10T22:22:07.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.563+0000 7f4efbddd740 10 client.0 _ll_register_callbacks cb 0x7f4ee0009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:22:07.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.564+0000 7f4ef2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 0x55c40b9dc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.564+0000 7f4ef1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9dc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.564+0000 7f4ef1400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9dc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:52640/0 (socket says 172.21.15.90:52640) 2024-09-10T22:22:07.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.564+0000 7f4ef1400640 1 -- 172.21.15.90:0/2300510041 learned_addr learned my addr 172.21.15.90:0/2300510041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:07.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.566+0000 7f4ef2800640 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9dbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.566+0000 7f4ef1400640 1 -- 172.21.15.90:0/2300510041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 msgr2=0x55c40b9dbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.566+0000 7f4ef1400640 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9dbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.566+0000 7f4ef1400640 1 -- 172.21.15.90:0/2300510041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 msgr2=0x55c40b9dc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.566+0000 7f4ef1400640 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 0x55c40b9dc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.566+0000 7f4ef1400640 1 -- 172.21.15.90:0/2300510041 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c40ba53900 con 0x55c40b9cac10 2024-09-10T22:22:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.566+0000 7f4ef1400640 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9dc200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ee002c040 tx=0x7f4ee001ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725af67e1a6f77f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:07.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.566+0000 7f4ed3e00640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:22:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.567+0000 7f4ed3e00640 1 -- 172.21.15.90:0/2300510041 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee001c420 con 0x55c40b9cac10 2024-09-10T22:22:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.567+0000 7f4ed3e00640 1 -- 172.21.15.90:0/2300510041 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9578+0+0 (secure 0 0 0) 0x7f4ee0018040 con 0x55c40b9cac10 2024-09-10T22:22:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.568+0000 7f4efbddd740 1 -- 172.21.15.90:0/2300510041 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee001c5c0 con 0x55c40b9cac10 2024-09-10T22:22:07.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.568+0000 7f4ed3e00640 1 -- 172.21.15.90:0/2300510041 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f4ee0005350 con 0x55c40b9cac10 2024-09-10T22:22:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.568+0000 7f4efbddd740 10 client.25963 fetch_fsmap learned FSMap version 66 2024-09-10T22:22:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.568+0000 7f4efbddd740 1 -- 172.21.15.90:0/2300510041 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55c40ba54be0 con 0x55c40b9cac10 2024-09-10T22:22:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.569+0000 7f4ed3e00640 1 -- 172.21.15.90:0/2300510041 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ee001c5c0 con 0x55c40b9cac10 2024-09-10T22:22:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.569+0000 7f4ed3e00640 1 -- 172.21.15.90:0/2300510041 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 66) ==== 8382+0+0 (secure 0 0 0) 0x7f4ee0004cd0 con 0x55c40b9cac10 2024-09-10T22:22:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.569+0000 7f4efbddd740 10 client.25963 fetch_fsmap finished waiting for FSMap version 66 2024-09-10T22:22:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.569+0000 7f4efbddd740 10 client.25963 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:22:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.569+0000 7f4efbddd740 20 client.25963 populate_metadata read hostname 'smithi090' 2024-09-10T22:22:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.569+0000 7f4efbddd740 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c40ba55150 0x55c40ba57570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:07.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.569+0000 7f4efbddd740 4 client.25963 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}] 2024-09-10T22:22:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.569+0000 7f4efbddd740 1 -- 172.21.15.90:0/2300510041 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}) -- 0x55c40b9e8ed0 con 0x55c40ba55150 2024-09-10T22:22:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.570+0000 7f4ef2800640 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c40ba55150 0x55c40ba57570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.571+0000 7f4ef2800640 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c40ba55150 0x55c40ba57570 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7497c412dd362655 server_cookie=3b982f701a450873 in_seq=0 out_seq=0 2024-09-10T22:22:07.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.571+0000 7f4ed3e00640 10 client.25963 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:07.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27970 5.45 MB/sec execute 178 sec latency 331.855 ms 2024-09-10T22:22:07.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4ed3e00640 1 -- 172.21.15.90:0/2300510041 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c40b9e8ed0 con 0x55c40ba55150 2024-09-10T22:22:07.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4ed3e00640 10 client.25963 handle_command_reply: tid=0 2024-09-10T22:22:07.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4efbddd740 1 client.25963 shutdown 2024-09-10T22:22:07.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4efbddd740 1 -- 172.21.15.90:0/2300510041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 msgr2=0x55c40b9dc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:07.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4efbddd740 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9dc200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ee002c040 tx=0x7f4ee001ccb0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4efbddd740 1 -- 172.21.15.90:0/2300510041 shutdown_connections 2024-09-10T22:22:07.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4efbddd740 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55c40ba55150 0x55c40ba57570 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4efbddd740 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c40b9d6ba0 0x55c40b9dc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4efbddd740 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55c40b9cac10 0x55c40b9dc200 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4efbddd740 1 --2- 172.21.15.90:0/2300510041 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55c40b9ca2f0 0x55c40b9dbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4efbddd740 1 -- 172.21.15.90:0/2300510041 >> 172.21.15.90:0/2300510041 conn(0x55c40b9c8120 msgr2=0x55c40b9d4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:07.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.921+0000 7f4ed3e00640 0 client.25963 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:07.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.922+0000 7f4efbddd740 1 -- 172.21.15.90:0/2300510041 shutdown_connections 2024-09-10T22:22:07.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.922+0000 7f4efbddd740 1 -- 172.21.15.90:0/2300510041 wait complete. 2024-09-10T22:22:07.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.922+0000 7f4efbddd740 20 client.25963 trim_cache size 0 max 16384 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 66, 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 54, 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout: "566a30ea": { 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 54, 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:22:07.924 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 141.0, 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:07.925 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:07.926 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:07.926 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:07.926 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:22:07.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.923+0000 7f4ef3200640 1 -- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4ed80b5580 msgr2=0x7f4ed80b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.923+0000 7f4ef3200640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4ed80b5580 0x7f4ed80b7a40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4103fd0 tx=0x7f4ee402a410 comp rx=0 tx=0).stop 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.923+0000 7f4ef3200640 1 -- 172.21.15.90:0/3593600059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ef40776d0 msgr2=0x7f4ef4074780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.923+0000 7f4ef3200640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ef40776d0 0x7f4ef4074780 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4ee402a930 tx=0x7f4ee4003b90 comp rx=0 tx=0).stop 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.924+0000 7f4ef3200640 1 -- 172.21.15.90:0/3593600059 shutdown_connections 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.924+0000 7f4ef3200640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f4ed80b5580 0x7f4ed80b7a40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.924+0000 7f4ef3200640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4ef4113800 0x7f4ef406dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.924+0000 7f4ef3200640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4ef4077ff0 0x7f4ef4074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.924+0000 7f4ef3200640 1 --2- 172.21.15.90:0/3593600059 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4ef40776d0 0x7f4ef4074780 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.924+0000 7f4ef3200640 1 -- 172.21.15.90:0/3593600059 >> 172.21.15.90:0/3593600059 conn(0x7f4ef4100840 msgr2=0x7f4ef4102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.924+0000 7f4ef3200640 1 -- 172.21.15.90:0/3593600059 shutdown_connections 2024-09-10T22:22:07.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:07.924+0000 7f4ef3200640 1 -- 172.21.15.90:0/3593600059 wait complete. 2024-09-10T22:22:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:08 smithi090 ceph-mon[30350]: pgmap v990: 129 pgs: 129 active+clean; 766 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2024-09-10T22:22:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:08 smithi154 ceph-mon[34495]: pgmap v990: 129 pgs: 129 active+clean; 766 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2024-09-10T22:22:08.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:08 smithi171 ceph-mon[34138]: pgmap v990: 129 pgs: 129 active+clean; 766 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.7 MiB/s wr, 254 op/s 2024-09-10T22:22:08.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.42 MB/sec execute 179 sec latency 1331.930 ms 2024-09-10T22:22:08.986 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '566a30ea', quiesce took 9.1 seconds. Will release after: 11.9 2024-09-10T22:22:09.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.39 MB/sec execute 180 sec latency 2332.051 ms 2024-09-10T22:22:10.268 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:10 smithi090 ceph-mon[30350]: pgmap v991: 129 pgs: 129 active+clean; 773 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 262 op/s 2024-09-10T22:22:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:10 smithi154 ceph-mon[34495]: pgmap v991: 129 pgs: 129 active+clean; 773 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 262 op/s 2024-09-10T22:22:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:10 smithi171 ceph-mon[34138]: pgmap v991: 129 pgs: 129 active+clean; 773 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 262 op/s 2024-09-10T22:22:10.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.36 MB/sec execute 181 sec latency 3332.123 ms 2024-09-10T22:22:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:11 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:22:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:11 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:22:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:11 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:22:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:11 smithi090 ceph-mon[30350]: osdmap e116: 12 total, 12 up, 12 in 2024-09-10T22:22:11.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:11 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:22:11.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:11 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:22:11.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:11 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:22:11.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:11 smithi154 ceph-mon[34495]: osdmap e116: 12 total, 12 up, 12 in 2024-09-10T22:22:11.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:11 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:22:11.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:11 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:22:11.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:11 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:22:11.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:11 smithi171 ceph-mon[34138]: osdmap e116: 12 total, 12 up, 12 in 2024-09-10T22:22:11.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.33 MB/sec execute 182 sec latency 4332.239 ms 2024-09-10T22:22:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:22:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:22:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:22:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:12 smithi090 ceph-mon[30350]: pgmap v993: 129 pgs: 129 active+clean; 773 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 209 op/s 2024-09-10T22:22:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:22:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:22:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:22:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:12 smithi154 ceph-mon[34495]: pgmap v993: 129 pgs: 129 active+clean; 773 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 209 op/s 2024-09-10T22:22:12.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:22:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:22:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:22:12.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:12 smithi171 ceph-mon[34138]: pgmap v993: 129 pgs: 129 active+clean; 773 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 209 op/s 2024-09-10T22:22:12.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.30 MB/sec execute 183 sec latency 5332.380 ms 2024-09-10T22:22:13.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.27 MB/sec execute 184 sec latency 6332.536 ms 2024-09-10T22:22:14.300 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:14 smithi171 ceph-mon[34138]: pgmap v994: 129 pgs: 129 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 174 op/s 2024-09-10T22:22:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:14 smithi090 ceph-mon[30350]: pgmap v994: 129 pgs: 129 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 174 op/s 2024-09-10T22:22:14.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:14 smithi154 ceph-mon[34495]: pgmap v994: 129 pgs: 129 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 174 op/s 2024-09-10T22:22:14.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.24 MB/sec execute 185 sec latency 7332.682 ms 2024-09-10T22:22:15.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.22 MB/sec execute 186 sec latency 8332.847 ms 2024-09-10T22:22:16.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.19 MB/sec execute 187 sec latency 9332.961 ms 2024-09-10T22:22:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:16 smithi090 ceph-mon[30350]: pgmap v995: 129 pgs: 129 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-09-10T22:22:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:16 smithi154 ceph-mon[34495]: pgmap v995: 129 pgs: 129 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-09-10T22:22:17.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:16 smithi171 ceph-mon[34138]: pgmap v995: 129 pgs: 129 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.7 MiB/s wr, 114 op/s 2024-09-10T22:22:17.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.16 MB/sec execute 188 sec latency 10333.077 ms 2024-09-10T22:22:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:18 smithi090 ceph-mon[30350]: pgmap v996: 129 pgs: 129 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-09-10T22:22:18.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:18 smithi154 ceph-mon[34495]: pgmap v996: 129 pgs: 129 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-09-10T22:22:18.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:18 smithi171 ceph-mon[34138]: pgmap v996: 129 pgs: 129 active+clean; 774 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-09-10T22:22:18.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.13 MB/sec execute 189 sec latency 11333.205 ms 2024-09-10T22:22:19.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.11 MB/sec execute 190 sec latency 12333.323 ms 2024-09-10T22:22:20.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.08 MB/sec execute 191 sec latency 13333.393 ms 2024-09-10T22:22:20.886 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:22:21.270 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:22:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:21 smithi090 ceph-mon[30350]: pgmap v997: 129 pgs: 129 active+clean; 775 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 168 KiB/s wr, 26 op/s 2024-09-10T22:22:21.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:21 smithi154 ceph-mon[34495]: pgmap v997: 129 pgs: 129 active+clean; 775 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 168 KiB/s wr, 26 op/s 2024-09-10T22:22:21.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:21 smithi171 ceph-mon[34138]: pgmap v997: 129 pgs: 129 active+clean; 775 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 168 KiB/s wr, 26 op/s 2024-09-10T22:22:21.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.05 MB/sec execute 192 sec latency 14333.461 ms 2024-09-10T22:22:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.856+0000 7f9ad0c00640 1 -- 172.21.15.90:0/4045142690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc104fa0 msgr2=0x7f9acc105380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.856+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/4045142690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc104fa0 0x7f9acc105380 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009900 tx=0x7f9abc02a410 comp rx=0 tx=0).stop 2024-09-10T22:22:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.856+0000 7f9ad0c00640 1 -- 172.21.15.90:0/4045142690 shutdown_connections 2024-09-10T22:22:21.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.856+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/4045142690 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9acc10b330 0x7f9acc111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.856+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/4045142690 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9acc10a9b0 0x7f9acc10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.856+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/4045142690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc104fa0 0x7f9acc105380 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.856+0000 7f9ad0c00640 1 -- 172.21.15.90:0/4045142690 >> 172.21.15.90:0/4045142690 conn(0x7f9acc100a50 msgr2=0x7f9acc102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.856+0000 7f9ad0c00640 1 -- 172.21.15.90:0/4045142690 shutdown_connections 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.856+0000 7f9ad0c00640 1 -- 172.21.15.90:0/4045142690 wait complete. 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.857+0000 7f9ad0c00640 1 Processor -- start 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.857+0000 7f9ad0c00640 1 -- start start 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ad0c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9acc104fa0 0x7f9acc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ad0c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc10a9b0 0x7f9acc19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ad0c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9acc10b330 0x7f9acc1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ad0c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc114df0 con 0x7f9acc10a9b0 2024-09-10T22:22:21.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ad0c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc114c70 con 0x7f9acc10b330 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ad0c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc114f70 con 0x7f9acc104fa0 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ac9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc10a9b0 0x7f9acc19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ac9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc10a9b0 0x7f9acc19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56290/0 (socket says 172.21.15.90:56290) 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ac9400640 1 -- 172.21.15.90:0/2868001884 learned_addr learned my addr 172.21.15.90:0/2868001884 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ac9e00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9acc104fa0 0x7f9acc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9aca800640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9acc10b330 0x7f9acc1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ac9400640 1 -- 172.21.15.90:0/2868001884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9acc104fa0 msgr2=0x7f9acc19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ac9400640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9acc104fa0 0x7f9acc19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ac9400640 1 -- 172.21.15.90:0/2868001884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9acc10b330 msgr2=0x7f9acc1a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ac9400640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9acc10b330 0x7f9acc1a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.858+0000 7f9ac9400640 1 -- 172.21.15.90:0/2868001884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc1a14f0 con 0x7f9acc10a9b0 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.859+0000 7f9ac9400640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc10a9b0 0x7f9acc19b980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800e9f0 tx=0x7f9ab800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:21.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.859+0000 7f9ac2a00640 1 -- 172.21.15.90:0/2868001884 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800cd80 con 0x7f9acc10a9b0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.859+0000 7f9ac2a00640 1 -- 172.21.15.90:0/2868001884 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ab80108c0 con 0x7f9acc10a9b0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.859+0000 7f9ac2a00640 1 -- 172.21.15.90:0/2868001884 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80187c0 con 0x7f9acc10a9b0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.859+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc1a17e0 con 0x7f9acc10a9b0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.859+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc1a7ea0 con 0x7f9acc10a9b0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.860+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc105380 con 0x7f9acc10a9b0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.862+0000 7f9ac2a00640 1 -- 172.21.15.90:0/2868001884 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9ab8010430 con 0x7f9acc10a9b0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.864+0000 7f9ac2a00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ab00b54d0 0x7f9ab00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.864+0000 7f9ac2a00640 1 -- 172.21.15.90:0/2868001884 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7f9ab80e6ba0 con 0x7f9acc10a9b0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.864+0000 7f9ac9e00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ab00b54d0 0x7f9ab00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.865+0000 7f9ac9e00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ab00b54d0 0x7f9ab00b7990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0057c0 tx=0x7f9abc0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:21.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:21.865+0000 7f9ac2a00640 1 -- 172.21.15.90:0/2868001884 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9ab80acca0 con 0x7f9acc10a9b0 2024-09-10T22:22:22.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.086+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9acc02e5c0 con 0x7f9acc10a9b0 2024-09-10T22:22:22.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.086+0000 7f9ac2a00640 1 -- 172.21.15.90:0/2868001884 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+11990 (secure 0 0 0) 0x7f9ab80b16d0 con 0x7f9acc10a9b0 2024-09-10T22:22:22.088 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:22:22.088 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":66,"btime":"2024-09-10T22:22:02:830195+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:22:02.758175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:22:22.089 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 66 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ab00b54d0 msgr2=0x7f9ab00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ab00b54d0 0x7f9ab00b7990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0057c0 tx=0x7f9abc0058d0 comp rx=0 tx=0).stop 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc10a9b0 msgr2=0x7f9acc19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc10a9b0 0x7f9acc19b980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800e9f0 tx=0x7f9ab800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 shutdown_connections 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9ab00b54d0 0x7f9ab00b7990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9acc10b330 0x7f9acc1a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9acc10a9b0 0x7f9acc19b980 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 --2- 172.21.15.90:0/2868001884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9acc104fa0 0x7f9acc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 >> 172.21.15.90:0/2868001884 conn(0x7f9acc100a50 msgr2=0x7f9acc101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 shutdown_connections 2024-09-10T22:22:22.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:22.089+0000 7f9ad0c00640 1 -- 172.21.15.90:0/2868001884 wait complete. 2024-09-10T22:22:22.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.03 MB/sec execute 193 sec latency 15333.501 ms 2024-09-10T22:22:23.016 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 566a30ea --release --await' 2024-09-10T22:22:23.017 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 566a30ea --release --await 2024-09-10T22:22:23.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:23 smithi090 ceph-mon[30350]: pgmap v998: 129 pgs: 129 active+clean; 775 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 146 KiB/s wr, 23 op/s 2024-09-10T22:22:23.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:23 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2868001884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:23.389 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:22:23.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:23 smithi154 ceph-mon[34495]: pgmap v998: 129 pgs: 129 active+clean; 775 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 146 KiB/s wr, 23 op/s 2024-09-10T22:22:23.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:23 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2868001884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:23.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:23 smithi171 ceph-mon[34138]: pgmap v998: 129 pgs: 129 active+clean; 775 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 146 KiB/s wr, 23 op/s 2024-09-10T22:22:23.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:23 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2868001884' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:23.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27975 5.00 MB/sec execute 194 sec latency 16333.568 ms 2024-09-10T22:22:23.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.950+0000 7ff0d8400640 1 -- 172.21.15.90:0/405689004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 msgr2=0x7ff0d010adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.950+0000 7ff0d8400640 1 --2- 172.21.15.90:0/405689004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 0x7ff0d010adf0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009900 tx=0x7ff0bc02a410 comp rx=0 tx=0).stop 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.950+0000 7ff0d8400640 1 -- 172.21.15.90:0/405689004 shutdown_connections 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.950+0000 7ff0d8400640 1 --2- 172.21.15.90:0/405689004 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d010b330 0x7ff0d0111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.950+0000 7ff0d8400640 1 --2- 172.21.15.90:0/405689004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 0x7ff0d010adf0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.950+0000 7ff0d8400640 1 --2- 172.21.15.90:0/405689004 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0d0104fa0 0x7ff0d0105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.950+0000 7ff0d8400640 1 -- 172.21.15.90:0/405689004 >> 172.21.15.90:0/405689004 conn(0x7ff0d0100a50 msgr2=0x7ff0d0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.950+0000 7ff0d8400640 1 -- 172.21.15.90:0/405689004 shutdown_connections 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.950+0000 7ff0d8400640 1 -- 172.21.15.90:0/405689004 wait complete. 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.951+0000 7ff0d8400640 1 Processor -- start 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.951+0000 7ff0d8400640 1 -- start start 2024-09-10T22:22:23.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.951+0000 7ff0d8400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104fa0 0x7ff0d019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.951+0000 7ff0d8400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 0x7ff0d019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.951+0000 7ff0d8400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0d010b330 0x7ff0d01a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.951+0000 7ff0d8400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0114de0 con 0x7ff0d010a9b0 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.951+0000 7ff0d8400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0114c60 con 0x7ff0d0104fa0 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.951+0000 7ff0d8400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0114f60 con 0x7ff0d010b330 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 0x7ff0d019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 0x7ff0d019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56310/0 (socket says 172.21.15.90:56310) 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5000640 1 -- 172.21.15.90:0/3866276976 learned_addr learned my addr 172.21.15.90:0/3866276976 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104fa0 0x7ff0d019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d6400640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0d010b330 0x7ff0d01a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5000640 1 -- 172.21.15.90:0/3866276976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0d010b330 msgr2=0x7ff0d01a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5000640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0d010b330 0x7ff0d01a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5000640 1 -- 172.21.15.90:0/3866276976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104fa0 msgr2=0x7ff0d019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:23.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5000640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104fa0 0x7ff0d019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5000640 1 -- 172.21.15.90:0/3866276976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d01a1400 con 0x7ff0d010a9b0 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.952+0000 7ff0d5000640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 0x7ff0d019b890 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009230 tx=0x7ff0bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.953+0000 7ff0ce000640 1 -- 172.21.15.90:0/3866276976 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc03f030 con 0x7ff0d010a9b0 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.953+0000 7ff0ce000640 1 -- 172.21.15.90:0/3866276976 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0bc003a80 con 0x7ff0d010a9b0 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.953+0000 7ff0ce000640 1 -- 172.21.15.90:0/3866276976 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc035070 con 0x7ff0d010a9b0 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.953+0000 7ff0d8400640 1 -- 172.21.15.90:0/3866276976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01a1690 con 0x7ff0d010a9b0 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.953+0000 7ff0d8400640 1 -- 172.21.15.90:0/3866276976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01a7db0 con 0x7ff0d010a9b0 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.956+0000 7ff0db027740 1 Processor -- start 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.956+0000 7ff0db027740 1 -- start start 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.956+0000 7ff0ce000640 1 -- 172.21.15.90:0/3866276976 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff0bc03bc30 con 0x7ff0d010a9b0 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.956+0000 7ff0db027740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 0x55abc4ac73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.956+0000 7ff0db027740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.956+0000 7ff0db027740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4ad5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.956+0000 7ff0db027740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55abc4a3e290 con 0x55abc4ac7910 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.956+0000 7ff0db027740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55abc4a2d6c0 con 0x55abc4ad38a0 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.956+0000 7ff0db027740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55abc4a314d0 con 0x55abc4ac6ff0 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56326/0 (socket says 172.21.15.90:56326) 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5a00640 1 -- 172.21.15.90:0/4071914839 learned_addr learned my addr 172.21.15.90:0/4071914839 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d6400640 1 --2- 172.21.15.90:0/4071914839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 0x55abc4ac73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5000640 1 --2- 172.21.15.90:0/4071914839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4ad5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5a00640 1 -- 172.21.15.90:0/4071914839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 msgr2=0x55abc4ac73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/4071914839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 0x55abc4ac73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5a00640 1 -- 172.21.15.90:0/4071914839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 msgr2=0x55abc4ad5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/4071914839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4ad5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5a00640 1 -- 172.21.15.90:0/4071914839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc49ba460 con 0x55abc4ac7910 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.957+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/4071914839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad3360 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4003a20 tx=0x7ff0c4003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6f556950e45de0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.958+0000 7ff0c2a00640 1 -- 172.21.15.90:0/4071914839 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4004b20 con 0x55abc4ac7910 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.958+0000 7ff0c2a00640 1 -- 172.21.15.90:0/4071914839 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0c4004cc0 con 0x55abc4ac7910 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.958+0000 7ff0c2a00640 1 -- 172.21.15.90:0/4071914839 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40126b0 con 0x55abc4ac7910 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.958+0000 7ff0ce000640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff0b80b5450 0x7ff0b80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.958+0000 7ff0ce000640 1 -- 172.21.15.90:0/3866276976 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7ff0bc106d40 con 0x7ff0d010a9b0 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.958+0000 7ff0d6400640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff0b80b5450 0x7ff0b80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.958+0000 7ff0db027740 1 -- 172.21.15.90:0/4071914839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 msgr2=0x55abc4ad3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.958+0000 7ff0db027740 1 --2- 172.21.15.90:0/4071914839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad3360 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4003a20 tx=0x7ff0c4003ee0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0db027740 1 -- 172.21.15.90:0/4071914839 shutdown_connections 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0db027740 1 --2- 172.21.15.90:0/4071914839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4ad5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0db027740 1 --2- 172.21.15.90:0/4071914839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad3360 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0db027740 1 --2- 172.21.15.90:0/4071914839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 0x55abc4ac73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0db027740 1 -- 172.21.15.90:0/4071914839 >> 172.21.15.90:0/4071914839 conn(0x55abc4ac4e20 msgr2=0x55abc4ac5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0db027740 1 -- 172.21.15.90:0/4071914839 shutdown_connections 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0db027740 1 -- 172.21.15.90:0/4071914839 wait complete. 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0d6400640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff0b80b5450 0x7ff0b80b7910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800ac90 tx=0x7ff0c800a250 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0db027740 1 -- start start 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.959+0000 7ff0db027740 1 Processor -- start 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0db027740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0db027740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac7910 0x55abc4ad0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0db027740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4ad12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0db027740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55abc488ebb0 con 0x55abc4ac6ff0 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0db027740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55abc49c86c0 con 0x55abc4ad38a0 2024-09-10T22:22:23.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0db027740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55abc4ad6550 con 0x55abc4ac7910 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0db027740 10 client.0 _ll_register_callbacks cb 0x55abc49e3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0d5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0d5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56342/0 (socket says 172.21.15.90:56342) 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0d5a00640 1 -- 172.21.15.90:0/3974800177 learned_addr learned my addr 172.21.15.90:0/3974800177 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.960+0000 7ff0d5000640 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac7910 0x55abc4ad0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0d5a00640 1 -- 172.21.15.90:0/3974800177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac7910 msgr2=0x55abc4ad0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac7910 0x55abc4ad0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0d5a00640 1 -- 172.21.15.90:0/3974800177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 msgr2=0x55abc4ad12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0d6400640 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4ad12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4ad12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0d5a00640 1 -- 172.21.15.90:0/3974800177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc478bb00 con 0x55abc4ac6ff0 2024-09-10T22:22:23.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0d6400640 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4ad12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad0860 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40049d0 tx=0x7ff0c4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f12808ff34f56b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0b7400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0b7400640 1 -- 172.21.15.90:0/3974800177 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4005280 con 0x55abc4ac6ff0 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0b7400640 1 -- 172.21.15.90:0/3974800177 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7ff0c4005420 con 0x55abc4ac6ff0 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.961+0000 7ff0db027740 1 -- 172.21.15.90:0/3974800177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c4004cc0 con 0x55abc4ac6ff0 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0b7400640 1 -- 172.21.15.90:0/3974800177 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff0c4004b00 con 0x55abc4ac6ff0 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0db027740 10 client.16365 fetch_fsmap learned FSMap version 67 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0db027740 1 -- 172.21.15.90:0/3974800177 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55abc4802960 con 0x55abc4ac6ff0 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0b7400640 1 -- 172.21.15.90:0/3974800177 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0c4004cc0 con 0x55abc4ac6ff0 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0b7400640 1 -- 172.21.15.90:0/3974800177 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7ff0c400f040 con 0x55abc4ac6ff0 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0db027740 10 client.16365 fetch_fsmap finished waiting for FSMap version 67 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0db027740 10 client.16365 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0db027740 20 client.16365 populate_metadata read hostname 'smithi090' 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0db027740 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55abc4ae3290 0x55abc4ae5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0db027740 4 client.16365 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.962+0000 7ff0db027740 1 -- 172.21.15.90:0/3974800177 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abc4a401e0 con 0x55abc4ae3290 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.963+0000 7ff0d5000640 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55abc4ae3290 0x55abc4ae5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.963+0000 7ff0d5000640 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55abc4ae3290 0x55abc4ae5680 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3400175093f5a768 server_cookie=ab0aa0d2e16132a1 in_seq=0 out_seq=0 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.963+0000 7ff0b7400640 10 client.16365 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.964+0000 7ff0b7400640 1 -- 172.21.15.90:0/3974800177 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55abc4a401e0 con 0x55abc4ae3290 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.964+0000 7ff0b7400640 10 client.16365 handle_command_reply: tid=0 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.964+0000 7ff0db027740 1 client.16365 shutdown 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.964+0000 7ff0db027740 1 -- 172.21.15.90:0/3974800177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 msgr2=0x55abc4ad0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad0860 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40049d0 tx=0x7ff0c4005bd0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 1 -- 172.21.15.90:0/3974800177 shutdown_connections 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55abc4ae3290 0x55abc4ae5680 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4ad12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac7910 0x55abc4ad0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0b7400640 0 client.16365 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 1 --2- 172.21.15.90:0/3974800177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad0860 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 1 -- 172.21.15.90:0/3974800177 >> 172.21.15.90:0/3974800177 conn(0x55abc4ac4e20 msgr2=0x55abc4aca9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 1 -- 172.21.15.90:0/3974800177 shutdown_connections 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 1 -- 172.21.15.90:0/3974800177 wait complete. 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.965+0000 7ff0db027740 20 client.16365 trim_cache size 0 max 16384 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.982+0000 7ff0db027740 1 Processor -- start 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.982+0000 7ff0db027740 1 -- start start 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0db027740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4acb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0db027740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ac7910 0x55abc4acda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0db027740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ad38a0 0x55abc4acdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0db027740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b1fb0 con 0x55abc4ac6ff0 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0db027740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b3b20 con 0x55abc4ac7910 2024-09-10T22:22:23.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0db027740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b2680 con 0x55abc4ad38a0 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4acb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d5a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4acb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56346/0 (socket says 172.21.15.90:56346) 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d5a00640 1 -- 172.21.15.90:0/2295402900 learned_addr learned my addr 172.21.15.90:0/2295402900 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d5000640 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ac7910 0x55abc4acda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d6400640 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ad38a0 0x55abc4acdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d5a00640 1 -- 172.21.15.90:0/2295402900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ad38a0 msgr2=0x55abc4acdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ad38a0 0x55abc4acdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d5a00640 1 -- 172.21.15.90:0/2295402900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ac7910 msgr2=0x55abc4acda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ac7910 0x55abc4acda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.983+0000 7ff0d5a00640 1 -- 172.21.15.90:0/2295402900 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc4ae5cb0 con 0x55abc4ac6ff0 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.984+0000 7ff0d5000640 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ac7910 0x55abc4acda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.984+0000 7ff0d6400640 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ad38a0 0x55abc4acdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:22:23.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.984+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4acb0f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40265d0 tx=0x7ff0c4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6db33ba66fdcf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.984+0000 7ff0c2000640 1 -- 172.21.15.90:0/2295402900 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4005960 con 0x55abc4ac6ff0 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.984+0000 7ff0c2000640 1 -- 172.21.15.90:0/2295402900 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0c401e070 con 0x55abc4ac6ff0 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.984+0000 7ff0c2000640 1 -- 172.21.15.90:0/2295402900 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40054e0 con 0x55abc4ac6ff0 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.985+0000 7ff0db027740 1 -- 172.21.15.90:0/2295402900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 msgr2=0x55abc4acb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.985+0000 7ff0db027740 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4acb0f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40265d0 tx=0x7ff0c4009b00 comp rx=0 tx=0).stop 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.985+0000 7ff0db027740 1 -- 172.21.15.90:0/2295402900 shutdown_connections 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.985+0000 7ff0db027740 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ad38a0 0x55abc4acdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.985+0000 7ff0db027740 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ac7910 0x55abc4acda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.985+0000 7ff0db027740 1 --2- 172.21.15.90:0/2295402900 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac6ff0 0x55abc4acb0f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.985+0000 7ff0db027740 1 -- 172.21.15.90:0/2295402900 >> 172.21.15.90:0/2295402900 conn(0x55abc4ac4e20 msgr2=0x55abc4ad26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.985+0000 7ff0db027740 1 -- 172.21.15.90:0/2295402900 shutdown_connections 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.985+0000 7ff0db027740 1 -- 172.21.15.90:0/2295402900 wait complete. 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.986+0000 7ff0db027740 1 -- start start 2024-09-10T22:22:23.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.986+0000 7ff0db027740 1 Processor -- start 2024-09-10T22:22:24.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.986+0000 7ff0db027740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:24.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.986+0000 7ff0db027740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:24.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.986+0000 7ff0db027740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4b4f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:24.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.986+0000 7ff0db027740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b3450 con 0x55abc4ac7910 2024-09-10T22:22:24.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.986+0000 7ff0db027740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01b2d50 con 0x55abc4ad38a0 2024-09-10T22:22:24.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.986+0000 7ff0db027740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01affb0 con 0x55abc4ac6ff0 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.986+0000 7ff0db027740 10 client.0 _ll_register_callbacks cb 0x7ff0c800a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56358/0 (socket says 172.21.15.90:56358) 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5000640 1 -- 172.21.15.90:0/2766563891 learned_addr learned my addr 172.21.15.90:0/2766563891 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d6400640 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4b4f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5000640 1 -- 172.21.15.90:0/2766563891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 msgr2=0x55abc4ad7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5000640 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5000640 1 -- 172.21.15.90:0/2766563891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 msgr2=0x55abc4b4f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5000640 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4b4f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5000640 1 -- 172.21.15.90:0/2766563891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc4ac58a0 con 0x55abc4ac7910 2024-09-10T22:22:24.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d6400640 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4b4f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0d5000640 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad8380 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc104170 tx=0x7ff0bc003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5396ca0c5339737 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.987+0000 7ff0b7400640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0b7400640 1 -- 172.21.15.90:0/2766563891 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc049040 con 0x55abc4ac7910 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0b7400640 1 -- 172.21.15.90:0/2766563891 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7ff0bc0c9110 con 0x55abc4ac7910 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0db027740 1 -- 172.21.15.90:0/2766563891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c401e070 con 0x55abc4ac7910 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0b7400640 1 -- 172.21.15.90:0/2766563891 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7ff0bc0cc9b0 con 0x55abc4ac7910 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0db027740 10 client.16371 fetch_fsmap learned FSMap version 67 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0db027740 1 -- 172.21.15.90:0/2766563891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55abc4b50790 con 0x55abc4ac7910 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0b7400640 1 -- 172.21.15.90:0/2766563891 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff0c401e070 con 0x55abc4ac7910 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0b7400640 1 -- 172.21.15.90:0/2766563891 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 67) ==== 8374+0+0 (secure 0 0 0) 0x7ff0bc0b4140 con 0x55abc4ac7910 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0db027740 10 client.16371 fetch_fsmap finished waiting for FSMap version 67 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0db027740 10 client.16371 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0db027740 20 client.16371 populate_metadata read hostname 'smithi090' 2024-09-10T22:22:24.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0db027740 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55abc4b50ec0 0x55abc4b532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:24.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0db027740 4 client.16371 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "566a30ea", "release": true, "await": true}] 2024-09-10T22:22:24.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.988+0000 7ff0db027740 1 -- 172.21.15.90:0/2766563891 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "566a30ea", "release": true, "await": true}) -- 0x55abc4a3bc90 con 0x55abc4b50ec0 2024-09-10T22:22:24.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.989+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55abc4b50ec0 0x55abc4b532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:24.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.989+0000 7ff0d5a00640 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55abc4b50ec0 0x55abc4b532e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd424852c02e8261 server_cookie=fb013303ab779ec3 in_seq=0 out_seq=0 2024-09-10T22:22:24.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:23.989+0000 7ff0b7400640 10 client.16371 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:24.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0b7400640 1 -- 172.21.15.90:0/2766563891 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55abc4a3bc90 con 0x55abc4b50ec0 2024-09-10T22:22:24.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0b7400640 10 client.16371 handle_command_reply: tid=0 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0db027740 1 client.16371 shutdown 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0db027740 1 -- 172.21.15.90:0/2766563891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 msgr2=0x55abc4ad8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0db027740 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad8380 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc104170 tx=0x7ff0bc003e60 comp rx=0 tx=0).stop 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0db027740 1 -- 172.21.15.90:0/2766563891 shutdown_connections 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0db027740 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55abc4b50ec0 0x55abc4b532e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0db027740 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55abc4ad38a0 0x55abc4b4f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0db027740 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55abc4ac7910 0x55abc4ad8380 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0db027740 1 --2- 172.21.15.90:0/2766563891 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55abc4ac6ff0 0x55abc4ad7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.026+0000 7ff0db027740 1 -- 172.21.15.90:0/2766563891 >> 172.21.15.90:0/2766563891 conn(0x55abc4ac4e20 msgr2=0x55abc4ade770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.027+0000 7ff0b7400640 0 client.16371 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.027+0000 7ff0db027740 1 -- 172.21.15.90:0/2766563891 shutdown_connections 2024-09-10T22:22:24.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.027+0000 7ff0db027740 1 -- 172.21.15.90:0/2766563891 wait complete. 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.027+0000 7ff0db027740 20 client.16371 trim_cache size 0 max 16384 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 67, 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 56, 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout: "566a30ea": { 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 56, 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:22:24.030 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 141.0, 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:24.031 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:22:24.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 -- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff0b80b5450 msgr2=0x7ff0b80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:24.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff0b80b5450 0x7ff0b80b7910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800ac90 tx=0x7ff0c800a250 comp rx=0 tx=0).stop 2024-09-10T22:22:24.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 -- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 msgr2=0x7ff0d019b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:24.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 0x7ff0d019b890 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009230 tx=0x7ff0bc004270 comp rx=0 tx=0).stop 2024-09-10T22:22:24.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 -- 172.21.15.90:0/3866276976 shutdown_connections 2024-09-10T22:22:24.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff0b80b5450 0x7ff0b80b7910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff0d010b330 0x7ff0d01a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0d010a9b0 0x7ff0d019b890 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 --2- 172.21.15.90:0/3866276976 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff0d0104fa0 0x7ff0d019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:24.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 -- 172.21.15.90:0/3866276976 >> 172.21.15.90:0/3866276976 conn(0x7ff0d0100a50 msgr2=0x7ff0d0102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:24.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 -- 172.21.15.90:0/3866276976 shutdown_connections 2024-09-10T22:22:24.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:24.028+0000 7ff0d8400640 1 -- 172.21.15.90:0/3866276976 wait complete. 2024-09-10T22:22:24.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:24 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:24.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:24 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:24.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:24 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:24.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28064 4.99 MB/sec execute 195 sec latency 16492.796 ms 2024-09-10T22:22:25.019 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '566a30ea', total seconds elapsed: 25.1 2024-09-10T22:22:25.019 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.0 2024-09-10T22:22:25.019 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:22:25.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:25 smithi090 ceph-mon[30350]: pgmap v999: 129 pgs: 129 active+clean; 775 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 114 KiB/s wr, 19 op/s 2024-09-10T22:22:25.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:25 smithi154 ceph-mon[34495]: pgmap v999: 129 pgs: 129 active+clean; 775 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 114 KiB/s wr, 19 op/s 2024-09-10T22:22:25.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:25 smithi171 ceph-mon[34138]: pgmap v999: 129 pgs: 129 active+clean; 775 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 114 KiB/s wr, 19 op/s 2024-09-10T22:22:25.585 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:22:25.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28174 4.99 MB/sec execute 196 sec latency 1846.143 ms 2024-09-10T22:22:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.248+0000 7f788ee00640 1 -- 172.21.15.90:0/3231410374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7888101900 msgr2=0x7f7888101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.248+0000 7f788ee00640 1 --2- 172.21.15.90:0/3231410374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7888101900 0x7f7888101ce0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f787c009900 tx=0x7f787c02a400 comp rx=0 tx=0).stop 2024-09-10T22:22:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.249+0000 7f788ee00640 1 -- 172.21.15.90:0/3231410374 shutdown_connections 2024-09-10T22:22:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.249+0000 7f788ee00640 1 --2- 172.21.15.90:0/3231410374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7888106d40 0x7f7888107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.249+0000 7f788ee00640 1 --2- 172.21.15.90:0/3231410374 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7888102220 0x7f78881066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.249+0000 7f788ee00640 1 --2- 172.21.15.90:0/3231410374 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7888101900 0x7f7888101ce0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.249+0000 7f788ee00640 1 -- 172.21.15.90:0/3231410374 >> 172.21.15.90:0/3231410374 conn(0x7f78880fd5b0 msgr2=0x7f78880ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:26.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.251+0000 7f788ee00640 1 -- 172.21.15.90:0/3231410374 shutdown_connections 2024-09-10T22:22:26.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.251+0000 7f788ee00640 1 -- 172.21.15.90:0/3231410374 wait complete. 2024-09-10T22:22:26.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.252+0000 7f788ee00640 1 Processor -- start 2024-09-10T22:22:26.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.252+0000 7f788ee00640 1 -- start start 2024-09-10T22:22:26.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788ee00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7888101900 0x7f788810fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788ee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7888102220 0x7f7888110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788ee00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7888106d40 0x7f7888115450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788ee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7888108420 con 0x7f7888102220 2024-09-10T22:22:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788ee00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78881082a0 con 0x7f7888101900 2024-09-10T22:22:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788ee00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f78881085a0 con 0x7f7888106d40 2024-09-10T22:22:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788da00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7888101900 0x7f788810fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788da00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7888101900 0x7f788810fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:55546/0 (socket says 172.21.15.90:55546) 2024-09-10T22:22:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788da00640 1 -- 172.21.15.90:0/1076760261 learned_addr learned my addr 172.21.15.90:0/1076760261 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788e400640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7888106d40 0x7f7888115450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:26.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.253+0000 7f788d000640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7888102220 0x7f7888110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:26.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.254+0000 7f788da00640 1 -- 172.21.15.90:0/1076760261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7888106d40 msgr2=0x7f7888115450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:26.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.254+0000 7f788da00640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7888106d40 0x7f7888115450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:26.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.254+0000 7f788da00640 1 -- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7888102220 msgr2=0x7f7888110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:26.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.254+0000 7f788da00640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7888102220 0x7f7888110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:26.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.254+0000 7f788da00640 1 -- 172.21.15.90:0/1076760261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7888115c20 con 0x7f7888101900 2024-09-10T22:22:26.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.255+0000 7f788da00640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7888101900 0x7f788810fbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f787c02a920 tx=0x7f787c003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:26.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.255+0000 7f7886000640 1 -- 172.21.15.90:0/1076760261 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c03f070 con 0x7f7888101900 2024-09-10T22:22:26.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.255+0000 7f7886000640 1 -- 172.21.15.90:0/1076760261 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f787c003e40 con 0x7f7888101900 2024-09-10T22:22:26.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.255+0000 7f7886000640 1 -- 172.21.15.90:0/1076760261 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c035030 con 0x7f7888101900 2024-09-10T22:22:26.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.255+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7888115eb0 con 0x7f7888101900 2024-09-10T22:22:26.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.255+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78881b4710 con 0x7f7888101900 2024-09-10T22:22:26.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.258+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78881066f0 con 0x7f7888101900 2024-09-10T22:22:26.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.260+0000 7f7886000640 1 -- 172.21.15.90:0/1076760261 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f787c031400 con 0x7f7888101900 2024-09-10T22:22:26.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.263+0000 7f7886000640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78700b5500 0x7f78700b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:26.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.263+0000 7f7886000640 1 -- 172.21.15.90:0/1076760261 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7f787c106f50 con 0x7f7888101900 2024-09-10T22:22:26.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.263+0000 7f788d000640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78700b5500 0x7f78700b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:26.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.266+0000 7f788d000640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78700b5500 0x7f78700b79c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f78740058c0 tx=0x7f7874005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:26.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.271+0000 7f7886000640 1 -- 172.21.15.90:0/1076760261 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f787c0cd700 con 0x7f7888101900 2024-09-10T22:22:26.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.684+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78881b4910 con 0x7f7888101900 2024-09-10T22:22:26.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.685+0000 7f7886000640 1 -- 172.21.15.90:0/1076760261 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+11987 (secure 0 0 0) 0x7f787c0cd000 con 0x7f7888101900 2024-09-10T22:22:26.687 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:22:26.687 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":67,"btime":"2024-09-10T22:22:23:019898+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":67,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:22:22.760900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:22:26.688 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 67 2024-09-10T22:22:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.688+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78700b5500 msgr2=0x7f78700b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78700b5500 0x7f78700b79c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f78740058c0 tx=0x7f7874005850 comp rx=0 tx=0).stop 2024-09-10T22:22:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7888101900 msgr2=0x7f788810fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:26.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7888101900 0x7f788810fbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f787c02a920 tx=0x7f787c003c20 comp rx=0 tx=0).stop 2024-09-10T22:22:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 shutdown_connections 2024-09-10T22:22:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78700b5500 0x7f78700b79c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7888106d40 0x7f7888115450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7888102220 0x7f7888110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 --2- 172.21.15.90:0/1076760261 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7888101900 0x7f788810fbc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 >> 172.21.15.90:0/1076760261 conn(0x7f78880fd5b0 msgr2=0x7f78881046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 shutdown_connections 2024-09-10T22:22:26.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:26.689+0000 7f788ee00640 1 -- 172.21.15.90:0/1076760261 wait complete. 2024-09-10T22:22:26.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28274 4.97 MB/sec execute 197 sec latency 2846.275 ms 2024-09-10T22:22:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:27 smithi090 ceph-mon[30350]: pgmap v1000: 129 pgs: 129 active+clean; 776 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 222 KiB/s wr, 29 op/s 2024-09-10T22:22:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:27 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1076760261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:27.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:27 smithi154 ceph-mon[34495]: pgmap v1000: 129 pgs: 129 active+clean; 776 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 222 KiB/s wr, 29 op/s 2024-09-10T22:22:27.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:27 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1076760261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:27.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:27 smithi171 ceph-mon[34138]: pgmap v1000: 129 pgs: 129 active+clean; 776 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 222 KiB/s wr, 29 op/s 2024-09-10T22:22:27.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:27 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1076760261' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:27.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28405 4.98 MB/sec execute 198 sec latency 3734.253 ms 2024-09-10T22:22:27.959 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0' 2024-09-10T22:22:27.960 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 141.0 --await-for 45.0 2024-09-10T22:22:28.565 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:22:28.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28518 4.98 MB/sec execute 199 sec latency 82.386 ms 2024-09-10T22:22:29.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.340+0000 7f86a6a00640 1 -- 172.21.15.90:0/2591440148 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0035020 con 0x7f86a810a3b0 2024-09-10T22:22:29.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.341+0000 7f86aec00640 1 -- 172.21.15.90:0/2591440148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a3b0 msgr2=0x7f86a8075070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:29.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.341+0000 7f86aec00640 1 --2- 172.21.15.90:0/2591440148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a3b0 0x7f86a8075070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f86a0009900 tx=0x7f86a002a400 comp rx=0 tx=0).stop 2024-09-10T22:22:29.343 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:29 smithi090 ceph-mon[30350]: pgmap v1001: 129 pgs: 129 active+clean; 778 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 390 KiB/s wr, 27 op/s 2024-09-10T22:22:29.344 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:29 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:29.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.341+0000 7f86aec00640 1 -- 172.21.15.90:0/2591440148 shutdown_connections 2024-09-10T22:22:29.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.341+0000 7f86aec00640 1 --2- 172.21.15.90:0/2591440148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a8075af0 0x7f86a807d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.341+0000 7f86aec00640 1 --2- 172.21.15.90:0/2591440148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86a810a790 0x7f86a80755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.341+0000 7f86aec00640 1 --2- 172.21.15.90:0/2591440148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a3b0 0x7f86a8075070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.341+0000 7f86aec00640 1 -- 172.21.15.90:0/2591440148 >> 172.21.15.90:0/2591440148 conn(0x7f86a806dfc0 msgr2=0x7f86a806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:29.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.343+0000 7f86aec00640 1 -- 172.21.15.90:0/2591440148 shutdown_connections 2024-09-10T22:22:29.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.343+0000 7f86aec00640 1 -- 172.21.15.90:0/2591440148 wait complete. 2024-09-10T22:22:29.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.344+0000 7f86aec00640 1 Processor -- start 2024-09-10T22:22:29.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.345+0000 7f86aec00640 1 -- start start 2024-09-10T22:22:29.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.345+0000 7f86aec00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86a8075af0 0x7f86a8083710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86aec00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a810a3b0 0x7f86a8083c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86aec00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a790 0x7f86a807eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86aec00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8085aa0 con 0x7f86a8075af0 2024-09-10T22:22:29.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86aec00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8085920 con 0x7f86a810a3b0 2024-09-10T22:22:29.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86aec00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f86a8085c20 con 0x7f86a810a790 2024-09-10T22:22:29.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86a7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86a8075af0 0x7f86a8083710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86acc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a790 0x7f86a807eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86acc00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a790 0x7f86a807eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:40354/0 (socket says 172.21.15.90:40354) 2024-09-10T22:22:29.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86acc00640 1 -- 172.21.15.90:0/3531761839 learned_addr learned my addr 172.21.15.90:0/3531761839 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:29.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.346+0000 7f86a7400640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a810a3b0 0x7f86a8083c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.347+0000 7f86acc00640 1 -- 172.21.15.90:0/3531761839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a810a3b0 msgr2=0x7f86a8083c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:29.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.347+0000 7f86acc00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a810a3b0 0x7f86a8083c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.347+0000 7f86acc00640 1 -- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86a8075af0 msgr2=0x7f86a8083710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:29.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.347+0000 7f86acc00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86a8075af0 0x7f86a8083710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.347+0000 7f86acc00640 1 -- 172.21.15.90:0/3531761839 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a807f430 con 0x7f86a810a790 2024-09-10T22:22:29.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.348+0000 7f86a7e00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86a8075af0 0x7f86a8083710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:22:29.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.349+0000 7f86a7400640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a810a3b0 0x7f86a8083c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:22:29.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.350+0000 7f86acc00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a790 0x7f86a807eb70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f869800eb10 tx=0x7f869800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:29.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.350+0000 7f86a4c00640 1 -- 172.21.15.90:0/3531761839 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698004b20 con 0x7f86a810a790 2024-09-10T22:22:29.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.350+0000 7f86a4c00640 1 -- 172.21.15.90:0/3531761839 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8698004cc0 con 0x7f86a810a790 2024-09-10T22:22:29.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.350+0000 7f86a4c00640 1 -- 172.21.15.90:0/3531761839 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8698010690 con 0x7f86a810a790 2024-09-10T22:22:29.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.350+0000 7f86aec00640 1 -- 172.21.15.90:0/3531761839 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a807f720 con 0x7f86a810a790 2024-09-10T22:22:29.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.351+0000 7f86aec00640 1 -- 172.21.15.90:0/3531761839 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a81c5a50 con 0x7f86a810a790 2024-09-10T22:22:29.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.361+0000 7f86a4c00640 1 -- 172.21.15.90:0/3531761839 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8698005020 con 0x7f86a810a790 2024-09-10T22:22:29.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.363+0000 7f86b1855740 1 Processor -- start 2024-09-10T22:22:29.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.364+0000 7f86a4c00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f868c0b54d0 0x7f868c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.364+0000 7f86a4c00640 1 -- 172.21.15.90:0/3531761839 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7f86980e6a70 con 0x7f86a810a790 2024-09-10T22:22:29.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.365+0000 7f86b1855740 1 -- start start 2024-09-10T22:22:29.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.365+0000 7f86b1855740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446180 0x55f441446560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.365+0000 7f86b1855740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 0x55f4414524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.365+0000 7f86b1855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.365+0000 7f86b1855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f4413bbca0 con 0x55f441452a30 2024-09-10T22:22:29.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.365+0000 7f86b1855740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f4413ab0d0 con 0x55f441446aa0 2024-09-10T22:22:29.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.365+0000 7f86b1855740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f4413aeee0 con 0x55f441446180 2024-09-10T22:22:29.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.366+0000 7f86a7400640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f868c0b54d0 0x7f868c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.366+0000 7f86acc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.366+0000 7f86acc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49874/0 (socket says 172.21.15.90:49874) 2024-09-10T22:22:29.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.366+0000 7f86acc00640 1 -- 172.21.15.90:0/2518625200 learned_addr learned my addr 172.21.15.90:0/2518625200 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:29.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.366+0000 7f86a7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446180 0x55f441446560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.367+0000 7f86acc00640 1 -- 172.21.15.90:0/2518625200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446180 msgr2=0x55f441446560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:29.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.367+0000 7f86acc00640 1 --2- 172.21.15.90:0/2518625200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446180 0x55f441446560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.367+0000 7f86acc00640 1 -- 172.21.15.90:0/2518625200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 msgr2=0x55f4414524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:29.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.367+0000 7f86acc00640 1 --2- 172.21.15.90:0/2518625200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 0x55f4414524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.367+0000 7f86acc00640 1 -- 172.21.15.90:0/2518625200 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44135d400 con 0x55f441452a30 2024-09-10T22:22:29.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.368+0000 7f86a7e00640 1 --2- 172.21.15.90:0/2518625200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446180 0x55f441446560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:22:29.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.369+0000 7f86acc00640 1 --2- 172.21.15.90:0/2518625200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454e20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f86980a90f0 tx=0x7f86980ac8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1ee6b670fe72c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:29.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.369+0000 7f86a7400640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f868c0b54d0 0x7f868c0b7990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55f441444e90 tx=0x7f869400a350 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:29.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.370+0000 7f869d200640 1 -- 172.21.15.90:0/2518625200 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86980afe60 con 0x55f441452a30 2024-09-10T22:22:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.370+0000 7f869d200640 1 -- 172.21.15.90:0/2518625200 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86980ade50 con 0x55f441452a30 2024-09-10T22:22:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.370+0000 7f869d200640 1 -- 172.21.15.90:0/2518625200 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86980acb20 con 0x55f441452a30 2024-09-10T22:22:29.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.371+0000 7f86b1855740 1 -- 172.21.15.90:0/2518625200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 msgr2=0x55f441454e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:29.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.371+0000 7f86b1855740 1 --2- 172.21.15.90:0/2518625200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454e20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f86980a90f0 tx=0x7f86980ac8f0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.371+0000 7f86b1855740 1 -- 172.21.15.90:0/2518625200 shutdown_connections 2024-09-10T22:22:29.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.371+0000 7f86b1855740 1 --2- 172.21.15.90:0/2518625200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454e20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.371+0000 7f86b1855740 1 --2- 172.21.15.90:0/2518625200 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 0x55f4414524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.371+0000 7f86b1855740 1 --2- 172.21.15.90:0/2518625200 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446180 0x55f441446560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.371+0000 7f86b1855740 1 -- 172.21.15.90:0/2518625200 >> 172.21.15.90:0/2518625200 conn(0x55f441443fb0 msgr2=0x55f441445150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.373+0000 7f86b1855740 1 -- 172.21.15.90:0/2518625200 shutdown_connections 2024-09-10T22:22:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.373+0000 7f86b1855740 1 -- 172.21.15.90:0/2518625200 wait complete. 2024-09-10T22:22:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.374+0000 7f86b1855740 1 -- start start 2024-09-10T22:22:29.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.374+0000 7f86b1855740 1 Processor -- start 2024-09-10T22:22:29.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86b1855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441452300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86b1855740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f44144f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86b1855740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441452a30 0x55f44144fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86b1855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f44120c7a0 con 0x55f441446180 2024-09-10T22:22:29.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86b1855740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55f441350a10 con 0x55f441452a30 2024-09-10T22:22:29.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86b1855740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55f4414556b0 con 0x55f441446aa0 2024-09-10T22:22:29.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86b1855740 10 client.0 _ll_register_callbacks cb 0x55f441360be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:22:29.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86acc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441452a30 0x55f44144fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86acc00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441452a30 0x55f44144fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:58928/0 (socket says 172.21.15.90:58928) 2024-09-10T22:22:29.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.375+0000 7f86acc00640 1 -- 172.21.15.90:0/3103742538 learned_addr learned my addr 172.21.15.90:0/3103742538 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:29.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.376+0000 7f86a7400640 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f44144f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.376+0000 7f86a7e00640 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441452300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.377+0000 7f86acc00640 1 -- 172.21.15.90:0/3103742538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 msgr2=0x55f44144f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:29.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.377+0000 7f86acc00640 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f44144f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.377+0000 7f86acc00640 1 -- 172.21.15.90:0/3103742538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 msgr2=0x55f441452300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:29.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.377+0000 7f86acc00640 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441452300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.377+0000 7f86acc00640 1 -- 172.21.15.90:0/3103742538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f441118b90 con 0x55f441452a30 2024-09-10T22:22:29.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.378+0000 7f86a7400640 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f44144f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:22:29.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.379+0000 7f86a7e00640 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441452300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:22:29.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.379+0000 7f86acc00640 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441452a30 0x55f44144fd30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86980acab0 tx=0x7f8698004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0e58ebe9aa4be0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:29.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.379+0000 7f8691600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:22:29.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.381+0000 7f8691600640 1 -- 172.21.15.90:0/3103742538 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86980a8290 con 0x55f441452a30 2024-09-10T22:22:29.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.381+0000 7f8691600640 1 -- 172.21.15.90:0/3103742538 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7f86980b10d0 con 0x55f441452a30 2024-09-10T22:22:29.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.382+0000 7f86b1855740 1 -- 172.21.15.90:0/3103742538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86980ade50 con 0x55f441452a30 2024-09-10T22:22:29.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.382+0000 7f8691600640 1 -- 172.21.15.90:0/3103742538 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f86980b1600 con 0x55f441452a30 2024-09-10T22:22:29.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.382+0000 7f86b1855740 10 client.36008 fetch_fsmap learned FSMap version 68 2024-09-10T22:22:29.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.382+0000 7f86b1855740 1 -- 172.21.15.90:0/3103742538 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55f441460f30 con 0x55f441452a30 2024-09-10T22:22:29.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.382+0000 7f8691600640 1 -- 172.21.15.90:0/3103742538 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f86980ade50 con 0x55f441452a30 2024-09-10T22:22:29.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.383+0000 7f8691600640 1 -- 172.21.15.90:0/3103742538 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 68) ==== 8370+0+0 (secure 0 0 0) 0x7f8698016bf0 con 0x55f441452a30 2024-09-10T22:22:29.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.383+0000 7f86b1855740 10 client.36008 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T22:22:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.383+0000 7f86b1855740 10 client.36008 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:22:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.383+0000 7f86b1855740 20 client.36008 populate_metadata read hostname 'smithi090' 2024-09-10T22:22:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.383+0000 7f86b1855740 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55f441462280 0x55f441464670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.383+0000 7f86b1855740 4 client.36008 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:22:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.383+0000 7f86b1855740 1 -- 172.21.15.90:0/3103742538 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4413bdbf0 con 0x55f441462280 2024-09-10T22:22:29.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.384+0000 7f86a7e00640 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55f441462280 0x55f441464670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.387+0000 7f86a7e00640 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55f441462280 0x55f441464670 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef2999846b98f542 server_cookie=e2a356ba189bfa60 in_seq=0 out_seq=0 2024-09-10T22:22:29.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.387+0000 7f8691600640 10 client.36008 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:29.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.388+0000 7f8691600640 1 -- 172.21.15.90:0/3103742538 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f4413bdbf0 con 0x55f441462280 2024-09-10T22:22:29.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.388+0000 7f8691600640 10 client.36008 handle_command_reply: tid=0 2024-09-10T22:22:29.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.388+0000 7f86b1855740 1 client.36008 shutdown 2024-09-10T22:22:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.389+0000 7f86b1855740 1 -- 172.21.15.90:0/3103742538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441452a30 msgr2=0x55f44144fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.389+0000 7f86b1855740 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441452a30 0x55f44144fd30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86980acab0 tx=0x7f8698004ab0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.389+0000 7f86b1855740 1 -- 172.21.15.90:0/3103742538 shutdown_connections 2024-09-10T22:22:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.389+0000 7f86b1855740 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55f441462280 0x55f441464670 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.389+0000 7f86b1855740 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441452a30 0x55f44144fd30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.389+0000 7f86b1855740 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f44144f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.389+0000 7f86b1855740 1 --2- 172.21.15.90:0/3103742538 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441452300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.389+0000 7f86b1855740 1 -- 172.21.15.90:0/3103742538 >> 172.21.15.90:0/3103742538 conn(0x55f441443fb0 msgr2=0x55f441449b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:29.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.389+0000 7f8691600640 0 client.36008 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:29.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.390+0000 7f86b1855740 1 -- 172.21.15.90:0/3103742538 shutdown_connections 2024-09-10T22:22:29.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.390+0000 7f86b1855740 1 -- 172.21.15.90:0/3103742538 wait complete. 2024-09-10T22:22:29.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.391+0000 7f86b1855740 20 client.36008 trim_cache size 0 max 16384 2024-09-10T22:22:29.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:29 smithi154 ceph-mon[34495]: pgmap v1001: 129 pgs: 129 active+clean; 778 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 390 KiB/s wr, 27 op/s 2024-09-10T22:22:29.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:29 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:29.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.425+0000 7f86b1855740 1 Processor -- start 2024-09-10T22:22:29.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.425+0000 7f86b1855740 1 -- start start 2024-09-10T22:22:29.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.425+0000 7f86b1855740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446180 0x55f44144efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.425+0000 7f86b1855740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f441453e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.425+0000 7f86b1855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.426+0000 7f86b1855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81cfca0 con 0x55f441452a30 2024-09-10T22:22:29.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.426+0000 7f86b1855740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81d17e0 con 0x55f441446180 2024-09-10T22:22:29.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.426+0000 7f86b1855740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81d0370 con 0x55f441446aa0 2024-09-10T22:22:29.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.426+0000 7f86a7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446180 0x55f44144efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.426+0000 7f86a7400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f441453e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.426+0000 7f86acc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.426+0000 7f86a7400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f441453e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:40382/0 (socket says 172.21.15.90:40382) 2024-09-10T22:22:29.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.426+0000 7f86a7400640 1 -- 172.21.15.90:0/1985798564 learned_addr learned my addr 172.21.15.90:0/1985798564 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:29.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.427+0000 7f86a7400640 1 -- 172.21.15.90:0/1985798564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446180 msgr2=0x55f44144efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:29.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.427+0000 7f86a7400640 1 --2- 172.21.15.90:0/1985798564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446180 0x55f44144efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.427+0000 7f86a7400640 1 -- 172.21.15.90:0/1985798564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 msgr2=0x55f441454360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:29.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.427+0000 7f86a7400640 1 --2- 172.21.15.90:0/1985798564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.427+0000 7f86a7400640 1 -- 172.21.15.90:0/1985798564 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f44144a490 con 0x55f441446aa0 2024-09-10T22:22:29.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.427+0000 7f86acc00640 1 --2- 172.21.15.90:0/1985798564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:22:29.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.427+0000 7f86a7400640 1 --2- 172.21.15.90:0/1985798564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f441453e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f869400eb50 tx=0x7f8694010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6816659f0c7b6ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:29.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.428+0000 7f8693e00640 1 -- 172.21.15.90:0/1985798564 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694025070 con 0x55f441446aa0 2024-09-10T22:22:29.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.428+0000 7f8693e00640 1 -- 172.21.15.90:0/1985798564 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8694020400 con 0x55f441446aa0 2024-09-10T22:22:29.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.428+0000 7f8693e00640 1 -- 172.21.15.90:0/1985798564 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869400f040 con 0x55f441446aa0 2024-09-10T22:22:29.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 -- 172.21.15.90:0/1985798564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 msgr2=0x55f441453e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:29.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 --2- 172.21.15.90:0/1985798564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f441453e20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f869400eb50 tx=0x7f8694010000 comp rx=0 tx=0).stop 2024-09-10T22:22:29.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 -- 172.21.15.90:0/1985798564 shutdown_connections 2024-09-10T22:22:29.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 --2- 172.21.15.90:0/1985798564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441452a30 0x55f441454360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 --2- 172.21.15.90:0/1985798564 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441446aa0 0x55f441453e20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 --2- 172.21.15.90:0/1985798564 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446180 0x55f44144efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 -- 172.21.15.90:0/1985798564 >> 172.21.15.90:0/1985798564 conn(0x55f441443fb0 msgr2=0x55f44145d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:29.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 -- 172.21.15.90:0/1985798564 shutdown_connections 2024-09-10T22:22:29.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 -- 172.21.15.90:0/1985798564 wait complete. 2024-09-10T22:22:29.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.429+0000 7f86b1855740 1 -- start start 2024-09-10T22:22:29.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.430+0000 7f86b1855740 1 Processor -- start 2024-09-10T22:22:29.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86b1855740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441467e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86b1855740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 0x55f4414642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86b1855740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441452a30 0x55f441464830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86b1855740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81d1110 con 0x55f441446180 2024-09-10T22:22:29.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86b1855740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81d0a40 con 0x55f441446aa0 2024-09-10T22:22:29.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86b1855740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f86a81cdca0 con 0x55f441452a30 2024-09-10T22:22:29.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86b1855740 10 client.0 _ll_register_callbacks cb 0x7f86980ad300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:22:29.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86a7400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 0x55f4414642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86a7400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 0x55f4414642f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:58946/0 (socket says 172.21.15.90:58946) 2024-09-10T22:22:29.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86a7400640 1 -- 172.21.15.90:0/546413614 learned_addr learned my addr 172.21.15.90:0/546413614 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:29.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.431+0000 7f86acc00640 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441452a30 0x55f441464830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.432+0000 7f86a7e00640 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441467e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.432+0000 7f86acc00640 1 -- 172.21.15.90:0/546413614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 msgr2=0x55f4414642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:29.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.432+0000 7f86acc00640 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 0x55f4414642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.432+0000 7f86acc00640 1 -- 172.21.15.90:0/546413614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 msgr2=0x55f441467e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:29.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.432+0000 7f86acc00640 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441467e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.432+0000 7f86acc00640 1 -- 172.21.15.90:0/546413614 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f441465060 con 0x55f441452a30 2024-09-10T22:22:29.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.432+0000 7f86a7e00640 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441467e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:22:29.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.433+0000 7f86acc00640 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441452a30 0x55f441464830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f86980e3e50 tx=0x7f86980ac8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93dd032ed6865bcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:29.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.433+0000 7f8691600640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:22:29.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.433+0000 7f8691600640 1 -- 172.21.15.90:0/546413614 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86980a8290 con 0x55f441452a30 2024-09-10T22:22:29.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.433+0000 7f8691600640 1 -- 172.21.15.90:0/546413614 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7f8698016d80 con 0x55f441452a30 2024-09-10T22:22:29.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.433+0000 7f86b1855740 1 -- 172.21.15.90:0/546413614 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8694020400 con 0x55f441452a30 2024-09-10T22:22:29.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.433+0000 7f8691600640 1 -- 172.21.15.90:0/546413614 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f86980a87f0 con 0x55f441452a30 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.433+0000 7f86b1855740 10 client.26002 fetch_fsmap learned FSMap version 68 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.433+0000 7f86b1855740 1 -- 172.21.15.90:0/546413614 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55f4414cfe00 con 0x55f441452a30 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.434+0000 7f8691600640 1 -- 172.21.15.90:0/546413614 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8694020400 con 0x55f441452a30 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.434+0000 7f8691600640 1 -- 172.21.15.90:0/546413614 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 68) ==== 8370+0+0 (secure 0 0 0) 0x7f86980b10d0 con 0x55f441452a30 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.434+0000 7f86b1855740 10 client.26002 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.434+0000 7f86b1855740 10 client.26002 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.434+0000 7f86b1855740 20 client.26002 populate_metadata read hostname 'smithi090' 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.434+0000 7f86b1855740 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55f4414d03d0 0x55f4414d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.434+0000 7f86b1855740 4 client.26002 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}] 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.434+0000 7f86b1855740 1 -- 172.21.15.90:0/546413614 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.0, "await_for": 45.0}) -- 0x55f4413b96a0 con 0x55f4414d03d0 2024-09-10T22:22:29.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.435+0000 7f86a7e00640 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55f4414d03d0 0x55f4414d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:29.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.435+0000 7f86a7e00640 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55f4414d03d0 0x55f4414d27f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696a057db9768326 server_cookie=4d5669ef933183a8 in_seq=0 out_seq=0 2024-09-10T22:22:29.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.435+0000 7f8691600640 10 client.26002 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:29.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:29 smithi171 ceph-mon[34138]: pgmap v1001: 129 pgs: 129 active+clean; 778 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 390 KiB/s wr, 27 op/s 2024-09-10T22:22:29.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:29 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:22:29.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.824+0000 7f8691600640 1 -- 172.21.15.90:0/546413614 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f4413b96a0 con 0x55f4414d03d0 2024-09-10T22:22:29.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.824+0000 7f8691600640 10 client.26002 handle_command_reply: tid=0 2024-09-10T22:22:29.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.824+0000 7f86b1855740 1 client.26002 shutdown 2024-09-10T22:22:29.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.825+0000 7f86b1855740 1 -- 172.21.15.90:0/546413614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441452a30 msgr2=0x55f441464830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:29.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.825+0000 7f86b1855740 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441452a30 0x55f441464830 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f86980e3e50 tx=0x7f86980ac8f0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.825+0000 7f86b1855740 1 -- 172.21.15.90:0/546413614 shutdown_connections 2024-09-10T22:22:29.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.825+0000 7f86b1855740 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55f4414d03d0 0x55f4414d27f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.825+0000 7f86b1855740 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55f441452a30 0x55f441464830 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.825+0000 7f86b1855740 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55f441446aa0 0x55f4414642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.825+0000 7f86b1855740 1 --2- 172.21.15.90:0/546413614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f441446180 0x55f441467e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.825+0000 7f86b1855740 1 -- 172.21.15.90:0/546413614 >> 172.21.15.90:0/546413614 conn(0x55f441443fb0 msgr2=0x55f44144e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:29.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.825+0000 7f8691600640 0 client.26002 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:29.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.826+0000 7f86b1855740 1 -- 172.21.15.90:0/546413614 shutdown_connections 2024-09-10T22:22:29.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.826+0000 7f86b1855740 1 -- 172.21.15.90:0/546413614 wait complete. 2024-09-10T22:22:29.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.826+0000 7f86b1855740 20 client.26002 trim_cache size 0 max 16384 2024-09-10T22:22:29.829 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:22:29.829 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 68, 2024-09-10T22:22:29.829 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:22:29.829 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 58, 2024-09-10T22:22:29.829 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:22:29.829 INFO:teuthology.orchestra.run.smithi090.stdout: "9c41c224": { 2024-09-10T22:22:29.829 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 58, 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 141.0, 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:29.830 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:29.831 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:29.831 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:29.831 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:29.831 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:22:29.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.827+0000 7f86aec00640 1 -- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f868c0b54d0 msgr2=0x7f868c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:29.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.827+0000 7f86aec00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f868c0b54d0 0x7f868c0b7990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55f441444e90 tx=0x7f869400a350 comp rx=0 tx=0).stop 2024-09-10T22:22:29.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.827+0000 7f86aec00640 1 -- 172.21.15.90:0/3531761839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a790 msgr2=0x7f86a807eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:29.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.827+0000 7f86aec00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a790 0x7f86a807eb70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f869800eb10 tx=0x7f869800c2d0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.828+0000 7f86aec00640 1 -- 172.21.15.90:0/3531761839 shutdown_connections 2024-09-10T22:22:29.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.828+0000 7f86aec00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f868c0b54d0 0x7f868c0b7990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.828+0000 7f86aec00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f86a810a790 0x7f86a807eb70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.828+0000 7f86aec00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f86a810a3b0 0x7f86a8083c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.828+0000 7f86aec00640 1 --2- 172.21.15.90:0/3531761839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f86a8075af0 0x7f86a8083710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:29.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.828+0000 7f86aec00640 1 -- 172.21.15.90:0/3531761839 >> 172.21.15.90:0/3531761839 conn(0x7f86a806dfc0 msgr2=0x7f86a80728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:29.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.828+0000 7f86aec00640 1 -- 172.21.15.90:0/3531761839 shutdown_connections 2024-09-10T22:22:29.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:29.828+0000 7f86aec00640 1 -- 172.21.15.90:0/3531761839 wait complete. 2024-09-10T22:22:29.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.97 MB/sec execute 200 sec latency 454.277 ms 2024-09-10T22:22:30.837 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9c41c224', quiesce took 5.8 seconds. Will release after: 15.2 2024-09-10T22:22:30.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.95 MB/sec execute 201 sec latency 1454.377 ms 2024-09-10T22:22:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:31 smithi090 ceph-mon[30350]: pgmap v1002: 129 pgs: 129 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 121 op/s 2024-09-10T22:22:31.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:31 smithi154 ceph-mon[34495]: pgmap v1002: 129 pgs: 129 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 121 op/s 2024-09-10T22:22:31.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:31 smithi171 ceph-mon[34138]: pgmap v1002: 129 pgs: 129 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 121 op/s 2024-09-10T22:22:31.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.92 MB/sec execute 202 sec latency 2454.455 ms 2024-09-10T22:22:32.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.90 MB/sec execute 203 sec latency 3454.566 ms 2024-09-10T22:22:33.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:33 smithi090 ceph-mon[30350]: pgmap v1003: 129 pgs: 129 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.8 MiB/s wr, 118 op/s 2024-09-10T22:22:33.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:33 smithi154 ceph-mon[34495]: pgmap v1003: 129 pgs: 129 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.8 MiB/s wr, 118 op/s 2024-09-10T22:22:33.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:33 smithi171 ceph-mon[34138]: pgmap v1003: 129 pgs: 129 active+clean; 790 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 KiB/s rd, 1.8 MiB/s wr, 118 op/s 2024-09-10T22:22:33.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.88 MB/sec execute 204 sec latency 4454.655 ms 2024-09-10T22:22:34.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.85 MB/sec execute 205 sec latency 5454.732 ms 2024-09-10T22:22:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:35 smithi090 ceph-mon[30350]: pgmap v1004: 129 pgs: 129 active+clean; 787 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-09-10T22:22:35.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:35 smithi154 ceph-mon[34495]: pgmap v1004: 129 pgs: 129 active+clean; 787 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-09-10T22:22:35.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:35 smithi171 ceph-mon[34138]: pgmap v1004: 129 pgs: 129 active+clean; 787 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 135 op/s 2024-09-10T22:22:35.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.83 MB/sec execute 206 sec latency 6454.837 ms 2024-09-10T22:22:36.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.81 MB/sec execute 207 sec latency 7454.992 ms 2024-09-10T22:22:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:37 smithi090 ceph-mon[30350]: pgmap v1005: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 150 op/s 2024-09-10T22:22:37.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:37 smithi154 ceph-mon[34495]: pgmap v1005: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 150 op/s 2024-09-10T22:22:37.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:37 smithi171 ceph-mon[34138]: pgmap v1005: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 150 op/s 2024-09-10T22:22:37.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.78 MB/sec execute 208 sec latency 8455.124 ms 2024-09-10T22:22:38.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.76 MB/sec execute 209 sec latency 9455.269 ms 2024-09-10T22:22:39.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:39 smithi090 ceph-mon[30350]: pgmap v1006: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 136 op/s 2024-09-10T22:22:39.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:39 smithi154 ceph-mon[34495]: pgmap v1006: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 136 op/s 2024-09-10T22:22:39.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:39 smithi171 ceph-mon[34138]: pgmap v1006: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 136 op/s 2024-09-10T22:22:39.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.74 MB/sec execute 210 sec latency 10455.411 ms 2024-09-10T22:22:40.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.71 MB/sec execute 211 sec latency 11455.553 ms 2024-09-10T22:22:41.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:41 smithi090 ceph-mon[30350]: pgmap v1007: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 136 op/s 2024-09-10T22:22:41.502 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:41 smithi171 ceph-mon[34138]: pgmap v1007: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 136 op/s 2024-09-10T22:22:41.631 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:41 smithi154 ceph-mon[34495]: pgmap v1007: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 136 op/s 2024-09-10T22:22:41.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.69 MB/sec execute 212 sec latency 12455.646 ms 2024-09-10T22:22:42.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.67 MB/sec execute 213 sec latency 13455.758 ms 2024-09-10T22:22:43.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:43 smithi090 ceph-mon[30350]: pgmap v1008: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 429 KiB/s wr, 42 op/s 2024-09-10T22:22:43.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:43 smithi154 ceph-mon[34495]: pgmap v1008: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 429 KiB/s wr, 42 op/s 2024-09-10T22:22:43.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:43 smithi171 ceph-mon[34138]: pgmap v1008: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 429 KiB/s wr, 42 op/s 2024-09-10T22:22:43.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.65 MB/sec execute 214 sec latency 14455.897 ms 2024-09-10T22:22:44.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.63 MB/sec execute 215 sec latency 15456.028 ms 2024-09-10T22:22:45.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:45 smithi090 ceph-mon[30350]: pgmap v1009: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 335 KiB/s wr, 40 op/s 2024-09-10T22:22:45.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:45 smithi154 ceph-mon[34495]: pgmap v1009: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 335 KiB/s wr, 40 op/s 2024-09-10T22:22:45.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:45 smithi171 ceph-mon[34138]: pgmap v1009: 129 pgs: 129 active+clean; 787 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 335 KiB/s wr, 40 op/s 2024-09-10T22:22:45.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.61 MB/sec execute 216 sec latency 16456.158 ms 2024-09-10T22:22:46.039 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:22:46.424 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:22:46.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.58 MB/sec execute 217 sec latency 17456.270 ms 2024-09-10T22:22:47.096 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:47 smithi090 ceph-mon[30350]: pgmap v1010: 129 pgs: 129 active+clean; 788 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 120 KiB/s wr, 28 op/s 2024-09-10T22:22:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.094+0000 7f78df200640 1 -- 172.21.15.90:0/449268849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0113960 msgr2=0x7f78e0115d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.094+0000 7f78df200640 1 --2- 172.21.15.90:0/449268849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0113960 0x7f78e0115d50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f78d8009ca0 tx=0x7f78d802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.094+0000 7f78df200640 1 -- 172.21.15.90:0/449268849 shutdown_connections 2024-09-10T22:22:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.094+0000 7f78df200640 1 --2- 172.21.15.90:0/449268849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0113960 0x7f78e0115d50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.094+0000 7f78df200640 1 --2- 172.21.15.90:0/449268849 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e0075e60 0x7f78e0113420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.094+0000 7f78df200640 1 --2- 172.21.15.90:0/449268849 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e0075540 0x7f78e0075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.094+0000 7f78df200640 1 -- 172.21.15.90:0/449268849 >> 172.21.15.90:0/449268849 conn(0x7f78e00fe6d0 msgr2=0x7f78e0100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.094+0000 7f78df200640 1 -- 172.21.15.90:0/449268849 shutdown_connections 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.094+0000 7f78df200640 1 -- 172.21.15.90:0/449268849 wait complete. 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.095+0000 7f78df200640 1 Processor -- start 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.095+0000 7f78df200640 1 -- start start 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78df200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0075540 0x7f78e019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78df200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e0075e60 0x7f78e019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78df200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e0113960 0x7f78e01a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78df200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01190d0 con 0x7f78e0075540 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78df200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0118f50 con 0x7f78e0113960 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78df200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0119250 con 0x7f78e0075e60 2024-09-10T22:22:47.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78dde00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0075540 0x7f78e019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78dde00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0075540 0x7f78e019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38810/0 (socket says 172.21.15.90:38810) 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78dde00640 1 -- 172.21.15.90:0/3961642430 learned_addr learned my addr 172.21.15.90:0/3961642430 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78de800640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e0113960 0x7f78e01a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.096+0000 7f78dd400640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e0075e60 0x7f78e019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78dde00640 1 -- 172.21.15.90:0/3961642430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e0075e60 msgr2=0x7f78e019fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78dde00640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e0075e60 0x7f78e019fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78dde00640 1 -- 172.21.15.90:0/3961642430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e0113960 msgr2=0x7f78e01a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78dde00640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e0113960 0x7f78e01a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78dde00640 1 -- 172.21.15.90:0/3961642430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e01a55f0 con 0x7f78e0075540 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78dde00640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0075540 0x7f78e019f540 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f78d000d770 tx=0x7f78d000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78d6a00640 1 -- 172.21.15.90:0/3961642430 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0017070 con 0x7f78e0075540 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78d6a00640 1 -- 172.21.15.90:0/3961642430 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f78d00054b0 con 0x7f78e0075540 2024-09-10T22:22:47.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e01a58e0 con 0x7f78e0075540 2024-09-10T22:22:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.097+0000 7f78d6a00640 1 -- 172.21.15.90:0/3961642430 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0004c60 con 0x7f78e0075540 2024-09-10T22:22:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.098+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e01abfa0 con 0x7f78e0075540 2024-09-10T22:22:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.099+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e0075920 con 0x7f78e0075540 2024-09-10T22:22:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.101+0000 7f78d6a00640 1 -- 172.21.15.90:0/3961642430 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f78d0004e00 con 0x7f78e0075540 2024-09-10T22:22:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.102+0000 7f78d6a00640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78c40b5500 0x7f78c40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.102+0000 7f78d6a00640 1 -- 172.21.15.90:0/3961642430 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7f78d00e5e80 con 0x7f78e0075540 2024-09-10T22:22:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.103+0000 7f78dd400640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78c40b5500 0x7f78c40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.103+0000 7f78dd400640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78c40b5500 0x7f78c40b79c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005fd0 tx=0x7f78cc005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.106+0000 7f78d6a00640 1 -- 172.21.15.90:0/3961642430 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f78d00ac5d0 con 0x7f78e0075540 2024-09-10T22:22:47.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.325+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78e01a0670 con 0x7f78e0075540 2024-09-10T22:22:47.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.325+0000 7f78d6a00640 1 -- 172.21.15.90:0/3961642430 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+11986 (secure 0 0 0) 0x7f78d00abed0 con 0x7f78e0075540 2024-09-10T22:22:47.327 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:22:47.327 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":68,"btime":"2024-09-10T22:22:28:045670+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":68,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:22:27.677721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:22:47.328 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 68 2024-09-10T22:22:47.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78c40b5500 msgr2=0x7f78c40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:47.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78c40b5500 0x7f78c40b79c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f78cc005fd0 tx=0x7f78cc005850 comp rx=0 tx=0).stop 2024-09-10T22:22:47.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0075540 msgr2=0x7f78e019f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:47.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0075540 0x7f78e019f540 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f78d000d770 tx=0x7f78d000dc30 comp rx=0 tx=0).stop 2024-09-10T22:22:47.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 shutdown_connections 2024-09-10T22:22:47.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f78c40b5500 0x7f78c40b79c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f78e0113960 0x7f78e01a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f78e0075e60 0x7f78e019fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 --2- 172.21.15.90:0/3961642430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78e0075540 0x7f78e019f540 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:47.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 >> 172.21.15.90:0/3961642430 conn(0x7f78e00fe6d0 msgr2=0x7f78e00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:47.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 shutdown_connections 2024-09-10T22:22:47.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:47.328+0000 7f78df200640 1 -- 172.21.15.90:0/3961642430 wait complete. 2024-09-10T22:22:47.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:47 smithi154 ceph-mon[34495]: pgmap v1010: 129 pgs: 129 active+clean; 788 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 120 KiB/s wr, 28 op/s 2024-09-10T22:22:47.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:47 smithi171 ceph-mon[34138]: pgmap v1010: 129 pgs: 129 active+clean; 788 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 120 KiB/s wr, 28 op/s 2024-09-10T22:22:47.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.56 MB/sec execute 218 sec latency 18456.345 ms 2024-09-10T22:22:48.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 9c41c224 --release --await' 2024-09-10T22:22:48.231 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 9c41c224 --release --await 2024-09-10T22:22:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:48 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3961642430' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:48 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3961642430' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:48.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:48 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3961642430' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:22:48.604 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:22:48.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28592 4.54 MB/sec execute 219 sec latency 19456.418 ms 2024-09-10T22:22:49.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:49 smithi090 ceph-mon[30350]: pgmap v1011: 129 pgs: 129 active+clean; 788 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2024-09-10T22:22:49.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.333+0000 7fca7f200640 1 -- 172.21.15.90:0/2938677114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010a820 msgr2=0x7fca8010ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:49.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.333+0000 7fca7f200640 1 --2- 172.21.15.90:0/2938677114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010a820 0x7fca8010ac60 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fca68009900 tx=0x7fca6802a410 comp rx=0 tx=0).stop 2024-09-10T22:22:49.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.333+0000 7fca7f200640 1 -- 172.21.15.90:0/2938677114 shutdown_connections 2024-09-10T22:22:49.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.333+0000 7fca7f200640 1 --2- 172.21.15.90:0/2938677114 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca8010b1a0 0x7fca801119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.333+0000 7fca7f200640 1 --2- 172.21.15.90:0/2938677114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010a820 0x7fca8010ac60 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.333+0000 7fca7f200640 1 --2- 172.21.15.90:0/2938677114 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca80104e10 0x7fca801051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.333+0000 7fca7f200640 1 -- 172.21.15.90:0/2938677114 >> 172.21.15.90:0/2938677114 conn(0x7fca80100840 msgr2=0x7fca80102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:49.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.333+0000 7fca7f200640 1 -- 172.21.15.90:0/2938677114 shutdown_connections 2024-09-10T22:22:49.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.333+0000 7fca7f200640 1 -- 172.21.15.90:0/2938677114 wait complete. 2024-09-10T22:22:49.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.334+0000 7fca7f200640 1 Processor -- start 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.334+0000 7fca7f200640 1 -- start start 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.334+0000 7fca7f200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca80104e10 0x7fca8019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.334+0000 7fca7f200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca8010a820 0x7fca8019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.334+0000 7fca7f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010b1a0 0x7fca801a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.334+0000 7fca7f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca80114d50 con 0x7fca8010b1a0 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.334+0000 7fca7f200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca80114bd0 con 0x7fca80104e10 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.334+0000 7fca7f200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca80114ed0 con 0x7fca8010a820 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca7e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010b1a0 0x7fca801a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca7e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010b1a0 0x7fca801a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57662/0 (socket says 172.21.15.90:57662) 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca7e800640 1 -- 172.21.15.90:0/610582057 learned_addr learned my addr 172.21.15.90:0/610582057 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:49.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca75400640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca8010a820 0x7fca8019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca7de00640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca80104e10 0x7fca8019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca7e800640 1 -- 172.21.15.90:0/610582057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca8010a820 msgr2=0x7fca8019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca7e800640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca8010a820 0x7fca8019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca7e800640 1 -- 172.21.15.90:0/610582057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca80104e10 msgr2=0x7fca8019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca7e800640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca80104e10 0x7fca8019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.335+0000 7fca7e800640 1 -- 172.21.15.90:0/610582057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca801a1460 con 0x7fca8010b1a0 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.336+0000 7fca7e800640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010b1a0 0x7fca801a0c90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b970 tx=0x7fca7800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.336+0000 7fca77400640 1 -- 172.21.15.90:0/610582057 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78019070 con 0x7fca8010b1a0 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.336+0000 7fca7f200640 1 -- 172.21.15.90:0/610582057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca801a1750 con 0x7fca8010b1a0 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.336+0000 7fca77400640 1 -- 172.21.15.90:0/610582057 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca78014440 con 0x7fca8010b1a0 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.336+0000 7fca77400640 1 -- 172.21.15.90:0/610582057 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca780075d0 con 0x7fca8010b1a0 2024-09-10T22:22:49.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.336+0000 7fca7f200640 1 -- 172.21.15.90:0/610582057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca801a7e10 con 0x7fca8010b1a0 2024-09-10T22:22:49.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.340+0000 7fca87c40740 1 Processor -- start 2024-09-10T22:22:49.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.340+0000 7fca77400640 1 -- 172.21.15.90:0/610582057 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fca780145f0 con 0x7fca8010b1a0 2024-09-10T22:22:49.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca87c40740 1 -- start start 2024-09-10T22:22:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca87c40740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c0f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca87c40740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c1b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca87c40740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 0x55fb40c1de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca87c40740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fb40b86290 con 0x55fb40c0faf0 2024-09-10T22:22:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca87c40740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fb40b756c0 con 0x55fb40c0f1d0 2024-09-10T22:22:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca87c40740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fb40b794d0 con 0x55fb40c1ba80 2024-09-10T22:22:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca7de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c1b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca7de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c1b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57676/0 (socket says 172.21.15.90:57676) 2024-09-10T22:22:49.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.341+0000 7fca7de00640 1 -- 172.21.15.90:0/3321772547 learned_addr learned my addr 172.21.15.90:0/3321772547 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:49.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca7de00640 1 -- 172.21.15.90:0/3321772547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 msgr2=0x55fb40c1de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:49.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca7de00640 1 --2- 172.21.15.90:0/3321772547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 0x55fb40c1de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca7de00640 1 -- 172.21.15.90:0/3321772547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 msgr2=0x55fb40c0f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:49.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca7de00640 1 --2- 172.21.15.90:0/3321772547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c0f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca7de00640 1 -- 172.21.15.90:0/3321772547 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb40b02460 con 0x55fb40c0faf0 2024-09-10T22:22:49.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca77400640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca640b5500 0x7fca640b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca7de00640 1 --2- 172.21.15.90:0/3321772547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c1b540 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fca70009aa0 tx=0x7fca70009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8db73419b79624 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:49.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca77400640 1 -- 172.21.15.90:0/610582057 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7fca780e5a70 con 0x7fca8010b1a0 2024-09-10T22:22:49.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca6f400640 1 -- 172.21.15.90:0/3321772547 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70017070 con 0x55fb40c0faf0 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca75400640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca640b5500 0x7fca640b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca6f400640 1 -- 172.21.15.90:0/3321772547 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca70013470 con 0x55fb40c0faf0 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.342+0000 7fca6f400640 1 -- 172.21.15.90:0/3321772547 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70012630 con 0x55fb40c0faf0 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca87c40740 1 -- 172.21.15.90:0/3321772547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 msgr2=0x55fb40c1b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca75400640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca640b5500 0x7fca640b79c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca680057c0 tx=0x7fca68005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca87c40740 1 --2- 172.21.15.90:0/3321772547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c1b540 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fca70009aa0 tx=0x7fca70009f60 comp rx=0 tx=0).stop 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca87c40740 1 -- 172.21.15.90:0/3321772547 shutdown_connections 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca87c40740 1 --2- 172.21.15.90:0/3321772547 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 0x55fb40c1de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca87c40740 1 --2- 172.21.15.90:0/3321772547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c1b540 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca87c40740 1 --2- 172.21.15.90:0/3321772547 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c0f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca87c40740 1 -- 172.21.15.90:0/3321772547 >> 172.21.15.90:0/3321772547 conn(0x55fb40c0d000 msgr2=0x55fb40c0e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:49.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca87c40740 1 -- 172.21.15.90:0/3321772547 shutdown_connections 2024-09-10T22:22:49.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.343+0000 7fca87c40740 1 -- 172.21.15.90:0/3321772547 wait complete. 2024-09-10T22:22:49.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.344+0000 7fca87c40740 1 -- start start 2024-09-10T22:22:49.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.344+0000 7fca87c40740 1 Processor -- start 2024-09-10T22:22:49.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca87c40740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0f1d0 0x55fb40c189b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca87c40740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c18ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca87c40740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c1ba80 0x55fb40c19430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca87c40740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fb409d6bb0 con 0x55fb40c0faf0 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca87c40740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fb40b106c0 con 0x55fb40c1ba80 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca87c40740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fb40c1e730 con 0x55fb40c0f1d0 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca87c40740 10 client.0 _ll_register_callbacks cb 0x55fb40b2b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca75400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c18ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca75400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c18ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57680/0 (socket says 172.21.15.90:57680) 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca75400640 1 -- 172.21.15.90:0/2245636741 learned_addr learned my addr 172.21.15.90:0/2245636741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca7de00640 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0f1d0 0x55fb40c189b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca7e800640 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c1ba80 0x55fb40c19430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca75400640 1 -- 172.21.15.90:0/2245636741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0f1d0 msgr2=0x55fb40c189b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca75400640 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0f1d0 0x55fb40c189b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca75400640 1 -- 172.21.15.90:0/2245636741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c1ba80 msgr2=0x55fb40c19430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca75400640 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c1ba80 0x55fb40c19430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.345+0000 7fca75400640 1 -- 172.21.15.90:0/2245636741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb408d3b00 con 0x55fb40c0faf0 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.346+0000 7fca75400640 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c18ef0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fca68009900 tx=0x7fca68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19201feb339f1800 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.346+0000 7fca63e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.346+0000 7fca63e00640 1 -- 172.21.15.90:0/2245636741 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6804d070 con 0x55fb40c0faf0 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.346+0000 7fca87c40740 1 -- 172.21.15.90:0/2245636741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca70013470 con 0x55fb40c0faf0 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.346+0000 7fca63e00640 1 -- 172.21.15.90:0/2245636741 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7fca680035f0 con 0x55fb40c0faf0 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.346+0000 7fca63e00640 1 -- 172.21.15.90:0/2245636741 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fca68059050 con 0x55fb40c0faf0 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.346+0000 7fca87c40740 10 client.16407 fetch_fsmap learned FSMap version 68 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.346+0000 7fca87c40740 1 -- 172.21.15.90:0/2245636741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55fb4094a960 con 0x55fb40c0faf0 2024-09-10T22:22:49.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.347+0000 7fca63e00640 1 -- 172.21.15.90:0/2245636741 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca70013470 con 0x55fb40c0faf0 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.347+0000 7fca63e00640 1 -- 172.21.15.90:0/2245636741 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 68) ==== 8370+0+0 (secure 0 0 0) 0x7fca680497c0 con 0x55fb40c0faf0 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.347+0000 7fca87c40740 10 client.16407 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.347+0000 7fca87c40740 10 client.16407 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.347+0000 7fca87c40740 20 client.16407 populate_metadata read hostname 'smithi090' 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.347+0000 7fca87c40740 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fb40c2b530 0x55fb40c2d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.347+0000 7fca87c40740 4 client.16407 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.347+0000 7fca87c40740 1 -- 172.21.15.90:0/2245636741 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb40b881e0 con 0x55fb40c2b530 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.348+0000 7fca7de00640 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fb40c2b530 0x55fb40c2d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.348+0000 7fca7de00640 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fb40c2b530 0x55fb40c2d990 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=febad910bf1a4957 server_cookie=63805922f0b819c7 in_seq=0 out_seq=0 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.348+0000 7fca63e00640 10 client.16407 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca63e00640 1 -- 172.21.15.90:0/2245636741 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb40b881e0 con 0x55fb40c2b530 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca63e00640 10 client.16407 handle_command_reply: tid=0 2024-09-10T22:22:49.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 client.16407 shutdown 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 -- 172.21.15.90:0/2245636741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 msgr2=0x55fb40c18ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c18ef0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fca68009900 tx=0x7fca68000bc0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 -- 172.21.15.90:0/2245636741 shutdown_connections 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fb40c2b530 0x55fb40c2d990 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c1ba80 0x55fb40c19430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c18ef0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 --2- 172.21.15.90:0/2245636741 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0f1d0 0x55fb40c189b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 -- 172.21.15.90:0/2245636741 >> 172.21.15.90:0/2245636741 conn(0x55fb40c0d000 msgr2=0x55fb40c12ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca63e00640 0 client.16407 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 -- 172.21.15.90:0/2245636741 shutdown_connections 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.349+0000 7fca87c40740 1 -- 172.21.15.90:0/2245636741 wait complete. 2024-09-10T22:22:49.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.350+0000 7fca87c40740 20 client.16407 trim_cache size 0 max 16384 2024-09-10T22:22:49.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.367+0000 7fca87c40740 1 Processor -- start 2024-09-10T22:22:49.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.367+0000 7fca87c40740 1 -- start start 2024-09-10T22:22:49.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca87c40740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c15bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca87c40740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c16100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca87c40740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 0x55fb40c33260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca87c40740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca801b1fc0 con 0x55fb40c0faf0 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca87c40740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca801b3b00 con 0x55fb40c0f1d0 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca87c40740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca801b2690 con 0x55fb40c1ba80 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca75400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca75400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c16100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57682/0 (socket says 172.21.15.90:57682) 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca75400640 1 -- 172.21.15.90:0/1942722884 learned_addr learned my addr 172.21.15.90:0/1942722884 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca7de00640 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c15bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca75400640 1 -- 172.21.15.90:0/1942722884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 msgr2=0x55fb40c33260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:49.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca7e800640 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 0x55fb40c33260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca75400640 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 0x55fb40c33260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca75400640 1 -- 172.21.15.90:0/1942722884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 msgr2=0x55fb40c15bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:49.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.368+0000 7fca75400640 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c15bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.369+0000 7fca75400640 1 -- 172.21.15.90:0/1942722884 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb40c0da80 con 0x55fb40c0faf0 2024-09-10T22:22:49.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.369+0000 7fca7e800640 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 0x55fb40c33260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:22:49.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.369+0000 7fca75400640 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c16100 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fca68009690 tx=0x7fca68031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515047cc95f3c09e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:49.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.369+0000 7fca6ea00640 1 -- 172.21.15.90:0/1942722884 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68061020 con 0x55fb40c0faf0 2024-09-10T22:22:49.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.369+0000 7fca6ea00640 1 -- 172.21.15.90:0/1942722884 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca68031830 con 0x55fb40c0faf0 2024-09-10T22:22:49.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.369+0000 7fca6ea00640 1 -- 172.21.15.90:0/1942722884 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6804d070 con 0x55fb40c0faf0 2024-09-10T22:22:49.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 -- 172.21.15.90:0/1942722884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 msgr2=0x55fb40c16100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:49.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c16100 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fca68009690 tx=0x7fca68031400 comp rx=0 tx=0).stop 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 -- 172.21.15.90:0/1942722884 shutdown_connections 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c1ba80 0x55fb40c33260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c0faf0 0x55fb40c16100 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 --2- 172.21.15.90:0/1942722884 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c15bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 -- 172.21.15.90:0/1942722884 >> 172.21.15.90:0/1942722884 conn(0x55fb40c0d000 msgr2=0x55fb40c1a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 -- 172.21.15.90:0/1942722884 shutdown_connections 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 -- 172.21.15.90:0/1942722884 wait complete. 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.370+0000 7fca87c40740 1 -- start start 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca87c40740 1 Processor -- start 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca87c40740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c20390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca87c40740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0faf0 0x55fb40c20910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca87c40740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c1ba80 0x55fb40c99cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca87c40740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fca801b3430 con 0x55fb40c1ba80 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca87c40740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fca801b2d60 con 0x55fb40c0f1d0 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca87c40740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fca801affc0 con 0x55fb40c0faf0 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca87c40740 10 client.0 _ll_register_callbacks cb 0x7fca68005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca7e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c1ba80 0x55fb40c99cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca7e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c1ba80 0x55fb40c99cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57686/0 (socket says 172.21.15.90:57686) 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.371+0000 7fca7e800640 1 -- 172.21.15.90:0/406461784 learned_addr learned my addr 172.21.15.90:0/406461784 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca75400640 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0faf0 0x55fb40c20910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca7de00640 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c20390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca7e800640 1 -- 172.21.15.90:0/406461784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0faf0 msgr2=0x55fb40c20910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca7e800640 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0faf0 0x55fb40c20910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca7e800640 1 -- 172.21.15.90:0/406461784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 msgr2=0x55fb40c20390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca7e800640 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c20390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca7e800640 1 -- 172.21.15.90:0/406461784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb40c9a3c0 con 0x55fb40c1ba80 2024-09-10T22:22:49.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca7e800640 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c1ba80 0x55fb40c99cb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fca780abb00 tx=0x7fca780a8240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f2b0585e66b7d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca63e00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca63e00640 1 -- 172.21.15.90:0/406461784 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca780b05a0 con 0x55fb40c1ba80 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca63e00640 1 -- 172.21.15.90:0/406461784 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9490+0+0 (secure 0 0 0) 0x7fca780b4bc0 con 0x55fb40c1ba80 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.372+0000 7fca87c40740 1 -- 172.21.15.90:0/406461784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca68031830 con 0x55fb40c1ba80 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca63e00640 1 -- 172.21.15.90:0/406461784 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fca780b5050 con 0x55fb40c1ba80 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca87c40740 10 client.16413 fetch_fsmap learned FSMap version 68 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca87c40740 1 -- 172.21.15.90:0/406461784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55fb40ca1860 con 0x55fb40c1ba80 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca63e00640 1 -- 172.21.15.90:0/406461784 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fca68031830 con 0x55fb40c1ba80 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca63e00640 1 -- 172.21.15.90:0/406461784 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 68) ==== 8370+0+0 (secure 0 0 0) 0x7fca780a76c0 con 0x55fb40c1ba80 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca87c40740 10 client.16413 fetch_fsmap finished waiting for FSMap version 68 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca87c40740 10 client.16413 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca87c40740 20 client.16413 populate_metadata read hostname 'smithi090' 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca87c40740 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fb40ca1fd0 0x55fb40ca43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:22:49.378 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca87c40740 4 client.16413 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9c41c224", "release": true, "await": true}] 2024-09-10T22:22:49.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.373+0000 7fca87c40740 1 -- 172.21.15.90:0/406461784 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9c41c224", "release": true, "await": true}) -- 0x55fb40b83c90 con 0x55fb40ca1fd0 2024-09-10T22:22:49.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.374+0000 7fca7de00640 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fb40ca1fd0 0x55fb40ca43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:22:49.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.374+0000 7fca7de00640 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fb40ca1fd0 0x55fb40ca43f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1306d49a01c6ae server_cookie=b85277107ac6a687 in_seq=0 out_seq=0 2024-09-10T22:22:49.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.374+0000 7fca63e00640 10 client.16413 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:49.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca63e00640 1 -- 172.21.15.90:0/406461784 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fb40b83c90 con 0x55fb40ca1fd0 2024-09-10T22:22:49.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca63e00640 10 client.16413 handle_command_reply: tid=0 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca87c40740 1 client.16413 shutdown 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca87c40740 1 -- 172.21.15.90:0/406461784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c1ba80 msgr2=0x55fb40c99cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca87c40740 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c1ba80 0x55fb40c99cb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fca780abb00 tx=0x7fca780a8240 comp rx=0 tx=0).stop 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca87c40740 1 -- 172.21.15.90:0/406461784 shutdown_connections 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca87c40740 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fb40ca1fd0 0x55fb40ca43f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca87c40740 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fb40c1ba80 0x55fb40c99cb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca63e00640 0 client.16413 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca87c40740 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fb40c0faf0 0x55fb40c20910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca87c40740 1 --2- 172.21.15.90:0/406461784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fb40c0f1d0 0x55fb40c20390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.410+0000 7fca87c40740 1 -- 172.21.15.90:0/406461784 >> 172.21.15.90:0/406461784 conn(0x55fb40c0d000 msgr2=0x55fb40c12380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.411+0000 7fca87c40740 1 -- 172.21.15.90:0/406461784 shutdown_connections 2024-09-10T22:22:49.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.411+0000 7fca87c40740 1 -- 172.21.15.90:0/406461784 wait complete. 2024-09-10T22:22:49.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.411+0000 7fca87c40740 20 client.16413 trim_cache size 0 max 16384 2024-09-10T22:22:49.414 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:22:49.414 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 68, 2024-09-10T22:22:49.414 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:22:49.414 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 60, 2024-09-10T22:22:49.414 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:22:49.414 INFO:teuthology.orchestra.run.smithi090.stdout: "9c41c224": { 2024-09-10T22:22:49.414 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 60, 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 141.0, 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:22:49.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:22:49.416 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:22:49.416 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:49.416 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:49.416 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:49.416 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:49.416 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:22:49.416 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:22:49.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 -- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca640b5500 msgr2=0x7fca640b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca640b5500 0x7fca640b79c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca680057c0 tx=0x7fca68005fd0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 -- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010b1a0 msgr2=0x7fca801a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010b1a0 0x7fca801a0c90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fca7800b970 tx=0x7fca7800bd70 comp rx=0 tx=0).stop 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 -- 172.21.15.90:0/610582057 shutdown_connections 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fca640b5500 0x7fca640b79c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca8010b1a0 0x7fca801a0c90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fca8010a820 0x7fca8019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 --2- 172.21.15.90:0/610582057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fca80104e10 0x7fca8019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 -- 172.21.15.90:0/610582057 >> 172.21.15.90:0/610582057 conn(0x7fca80100840 msgr2=0x7fca80102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 -- 172.21.15.90:0/610582057 shutdown_connections 2024-09-10T22:22:49.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:22:49.412+0000 7fca7f200640 1 -- 172.21.15.90:0/610582057 wait complete. 2024-09-10T22:22:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:49 smithi154 ceph-mon[34495]: pgmap v1011: 129 pgs: 129 active+clean; 788 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2024-09-10T22:22:49.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:49 smithi171 ceph-mon[34138]: pgmap v1011: 129 pgs: 129 active+clean; 788 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 48 KiB/s wr, 11 op/s 2024-09-10T22:22:49.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28649 4.53 MB/sec execute 220 sec latency 20006.246 ms 2024-09-10T22:22:50.340 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9c41c224', total seconds elapsed: 25.3 2024-09-10T22:22:50.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28786 4.56 MB/sec execute 221 sec latency 1457.700 ms 2024-09-10T22:22:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:51 smithi090 ceph-mon[30350]: pgmap v1012: 129 pgs: 129 active+clean; 789 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 125 KiB/s wr, 22 op/s 2024-09-10T22:22:51.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:51 smithi154 ceph-mon[34495]: pgmap v1012: 129 pgs: 129 active+clean; 789 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 125 KiB/s wr, 22 op/s 2024-09-10T22:22:51.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:51 smithi171 ceph-mon[34138]: pgmap v1012: 129 pgs: 129 active+clean; 789 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 125 KiB/s wr, 22 op/s 2024-09-10T22:22:51.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28950 4.59 MB/sec execute 222 sec latency 2457.748 ms 2024-09-10T22:22:52.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29080 4.61 MB/sec execute 223 sec latency 3358.642 ms 2024-09-10T22:22:53.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:53 smithi154 ceph-mon[34495]: pgmap v1013: 129 pgs: 129 active+clean; 789 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 118 KiB/s wr, 16 op/s 2024-09-10T22:22:53.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:53 smithi171 ceph-mon[34138]: pgmap v1013: 129 pgs: 129 active+clean; 789 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 118 KiB/s wr, 16 op/s 2024-09-10T22:22:53.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:53 smithi090 ceph-mon[30350]: pgmap v1013: 129 pgs: 129 active+clean; 789 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 118 KiB/s wr, 16 op/s 2024-09-10T22:22:53.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29205 4.61 MB/sec execute 224 sec latency 179.485 ms 2024-09-10T22:22:54.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29329 4.61 MB/sec execute 225 sec latency 79.038 ms 2024-09-10T22:22:55.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:55 smithi154 ceph-mon[34495]: pgmap v1014: 129 pgs: 129 active+clean; 793 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 853 KiB/s wr, 40 op/s 2024-09-10T22:22:55.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:55 smithi171 ceph-mon[34138]: pgmap v1014: 129 pgs: 129 active+clean; 793 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 853 KiB/s wr, 40 op/s 2024-09-10T22:22:55.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:55 smithi090 ceph-mon[30350]: pgmap v1014: 129 pgs: 129 active+clean; 793 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 853 KiB/s wr, 40 op/s 2024-09-10T22:22:55.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29436 4.60 MB/sec execute 226 sec latency 79.229 ms 2024-09-10T22:22:56.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29568 4.60 MB/sec execute 227 sec latency 71.731 ms 2024-09-10T22:22:57.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:57 smithi154 ceph-mon[34495]: pgmap v1015: 129 pgs: 129 active+clean; 801 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-09-10T22:22:57.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:57 smithi171 ceph-mon[34138]: pgmap v1015: 129 pgs: 129 active+clean; 801 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-09-10T22:22:57.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:57 smithi090 ceph-mon[30350]: pgmap v1015: 129 pgs: 129 active+clean; 801 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-09-10T22:22:57.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29692 4.59 MB/sec execute 228 sec latency 68.328 ms 2024-09-10T22:22:58.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29825 4.61 MB/sec execute 229 sec latency 80.637 ms 2024-09-10T22:22:59.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:22:59 smithi090 ceph-mon[30350]: pgmap v1016: 129 pgs: 129 active+clean; 801 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-10T22:22:59.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:22:59 smithi154 ceph-mon[34495]: pgmap v1016: 129 pgs: 129 active+clean; 801 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-10T22:22:59.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:22:59 smithi171 ceph-mon[34138]: pgmap v1016: 129 pgs: 129 active+clean; 801 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-10T22:22:59.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29949 4.62 MB/sec execute 230 sec latency 89.590 ms 2024-09-10T22:23:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:00 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:00.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:00 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:00.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:00 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:00.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30070 4.62 MB/sec execute 231 sec latency 122.161 ms 2024-09-10T22:23:01.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:01 smithi154 ceph-mon[34495]: pgmap v1017: 129 pgs: 129 active+clean; 809 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-09-10T22:23:01.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:01 smithi171 ceph-mon[34138]: pgmap v1017: 129 pgs: 129 active+clean; 809 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-09-10T22:23:01.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:01 smithi090 ceph-mon[30350]: pgmap v1017: 129 pgs: 129 active+clean; 809 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 206 op/s 2024-09-10T22:23:01.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30070 4.60 MB/sec execute 232 sec latency 1122.368 ms 2024-09-10T22:23:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:02 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:02.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:02 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:02.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:02 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:02.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30070 4.58 MB/sec execute 233 sec latency 2122.425 ms 2024-09-10T22:23:03.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:03 smithi154 ceph-mon[34495]: pgmap v1018: 129 pgs: 129 active+clean; 809 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 204 op/s 2024-09-10T22:23:03.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:03 smithi171 ceph-mon[34138]: pgmap v1018: 129 pgs: 129 active+clean; 809 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 204 op/s 2024-09-10T22:23:03.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:03 smithi090 ceph-mon[30350]: pgmap v1018: 129 pgs: 129 active+clean; 809 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 204 op/s 2024-09-10T22:23:03.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30073 4.56 MB/sec execute 234 sec latency 3083.180 ms 2024-09-10T22:23:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:04 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:04.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:04 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:04.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:04 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:04.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30256 4.60 MB/sec execute 235 sec latency 3083.126 ms 2024-09-10T22:23:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:05 smithi154 ceph-mon[34495]: pgmap v1019: 129 pgs: 129 active+clean; 807 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2024-09-10T22:23:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:05 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:05.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:05 smithi171 ceph-mon[34138]: pgmap v1019: 129 pgs: 129 active+clean; 807 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2024-09-10T22:23:05.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:05 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:05.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:05 smithi090 ceph-mon[30350]: pgmap v1019: 129 pgs: 129 active+clean; 807 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.9 MiB/s wr, 241 op/s 2024-09-10T22:23:05.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:05 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:05.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30404 4.62 MB/sec execute 236 sec latency 69.713 ms 2024-09-10T22:23:06.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30539 4.63 MB/sec execute 237 sec latency 59.689 ms 2024-09-10T22:23:07.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:07 smithi171 ceph-mon[34138]: pgmap v1020: 129 pgs: 129 active+clean; 809 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 252 op/s 2024-09-10T22:23:07.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:07 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:07.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:07 smithi090 ceph-mon[30350]: pgmap v1020: 129 pgs: 129 active+clean; 809 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 252 op/s 2024-09-10T22:23:07.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:07 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:07.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:07 smithi154 ceph-mon[34495]: pgmap v1020: 129 pgs: 129 active+clean; 809 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.6 MiB/s wr, 252 op/s 2024-09-10T22:23:07.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:07 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:07.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30691 4.66 MB/sec execute 238 sec latency 74.019 ms 2024-09-10T22:23:08.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30792 4.65 MB/sec execute 239 sec latency 67.601 ms 2024-09-10T22:23:09.501 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:09 smithi171 ceph-mon[34138]: pgmap v1021: 129 pgs: 129 active+clean; 815 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-09-10T22:23:09.501 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:09 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:09.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:09 smithi090 ceph-mon[30350]: pgmap v1021: 129 pgs: 129 active+clean; 815 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-09-10T22:23:09.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:09 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:09.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:09 smithi154 ceph-mon[34495]: pgmap v1021: 129 pgs: 129 active+clean; 815 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 198 op/s 2024-09-10T22:23:09.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:09 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:09.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30874 4.66 MB/sec execute 240 sec latency 544.923 ms 2024-09-10T22:23:10.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:10 smithi171 ceph-mon[34138]: osdmap e117: 12 total, 12 up, 12 in 2024-09-10T22:23:10.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:10 smithi090 ceph-mon[30350]: osdmap e117: 12 total, 12 up, 12 in 2024-09-10T22:23:10.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:10 smithi154 ceph-mon[34495]: osdmap e117: 12 total, 12 up, 12 in 2024-09-10T22:23:10.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30982 4.65 MB/sec execute 241 sec latency 74.619 ms 2024-09-10T22:23:11.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:11 smithi171 ceph-mon[34138]: pgmap v1023: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 820 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-10T22:23:11.508 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:11 smithi090 ceph-mon[30350]: pgmap v1023: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 820 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-10T22:23:11.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:11 smithi154 ceph-mon[34495]: pgmap v1023: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 820 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.2 MiB/s wr, 210 op/s 2024-09-10T22:23:11.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31113 4.66 MB/sec execute 242 sec latency 63.834 ms 2024-09-10T22:23:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:23:12.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:23:12.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:23:12.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31208 4.67 MB/sec execute 243 sec latency 89.256 ms 2024-09-10T22:23:13.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:13 smithi154 ceph-mon[34495]: pgmap v1024: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 820 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 218 op/s 2024-09-10T22:23:13.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:13 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:13 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:13 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:13 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:13 smithi090 ceph-mon[30350]: pgmap v1024: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 820 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 218 op/s 2024-09-10T22:23:13.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:13 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:13 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:13 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:13 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:13 smithi171 ceph-mon[34138]: pgmap v1024: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 820 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 218 op/s 2024-09-10T22:23:13.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:13 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:13 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:13 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:13 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:13.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31279 4.66 MB/sec execute 244 sec latency 478.230 ms 2024-09-10T22:23:14.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31353 4.66 MB/sec execute 245 sec latency 444.957 ms 2024-09-10T22:23:15.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:15 smithi154 ceph-mon[34495]: pgmap v1025: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 820 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 244 op/s 2024-09-10T22:23:15.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:15 smithi171 ceph-mon[34138]: pgmap v1025: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 820 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 244 op/s 2024-09-10T22:23:15.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:15 smithi090 ceph-mon[30350]: pgmap v1025: 129 pgs: 3 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 119 active+clean; 820 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 244 op/s 2024-09-10T22:23:15.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31465 4.66 MB/sec execute 246 sec latency 217.901 ms 2024-09-10T22:23:16.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31607 4.67 MB/sec execute 247 sec latency 105.354 ms 2024-09-10T22:23:16.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:16.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:16 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:16 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:17.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:17.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:17.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:16 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:17.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31715 4.67 MB/sec execute 248 sec latency 87.642 ms 2024-09-10T22:23:18.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:17 smithi171 ceph-mon[34138]: pgmap v1026: 129 pgs: 129 active+clean; 822 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-09-10T22:23:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:17 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:17 smithi090 ceph-mon[30350]: pgmap v1026: 129 pgs: 129 active+clean; 822 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-09-10T22:23:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:17 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:17 smithi154 ceph-mon[34495]: pgmap v1026: 129 pgs: 129 active+clean; 822 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 244 op/s 2024-09-10T22:23:18.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:17 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:18.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31855 4.68 MB/sec execute 249 sec latency 93.678 ms 2024-09-10T22:23:19.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31999 4.70 MB/sec execute 250 sec latency 72.457 ms 2024-09-10T22:23:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:19 smithi090 ceph-mon[30350]: pgmap v1027: 129 pgs: 129 active+clean; 825 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 234 op/s 2024-09-10T22:23:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:23:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:23:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:19 smithi154 ceph-mon[34495]: pgmap v1027: 129 pgs: 129 active+clean; 825 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 234 op/s 2024-09-10T22:23:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:23:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:23:20.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:19 smithi171 ceph-mon[34138]: pgmap v1027: 129 pgs: 129 active+clean; 825 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 234 op/s 2024-09-10T22:23:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:20.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:20.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:23:20.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:23:20.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:23:20.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32118 4.71 MB/sec execute 251 sec latency 275.469 ms 2024-09-10T22:23:21.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32285 4.74 MB/sec execute 252 sec latency 347.134 ms 2024-09-10T22:23:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:21 smithi090 ceph-mon[30350]: pgmap v1028: 129 pgs: 129 active+clean; 833 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-09-10T22:23:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:21 smithi090 ceph-mon[30350]: osdmap e118: 12 total, 12 up, 12 in 2024-09-10T22:23:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:21 smithi154 ceph-mon[34495]: pgmap v1028: 129 pgs: 129 active+clean; 833 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-09-10T22:23:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:21 smithi154 ceph-mon[34495]: osdmap e118: 12 total, 12 up, 12 in 2024-09-10T22:23:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:21 smithi171 ceph-mon[34138]: pgmap v1028: 129 pgs: 129 active+clean; 833 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 227 op/s 2024-09-10T22:23:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:21 smithi171 ceph-mon[34138]: osdmap e118: 12 total, 12 up, 12 in 2024-09-10T22:23:22.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32411 4.77 MB/sec execute 253 sec latency 149.635 ms 2024-09-10T22:23:23.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32516 4.77 MB/sec execute 254 sec latency 694.298 ms 2024-09-10T22:23:24.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:23 smithi090 ceph-mon[30350]: pgmap v1030: 129 pgs: 129 active+clean; 835 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-09-10T22:23:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:23 smithi154 ceph-mon[34495]: pgmap v1030: 129 pgs: 129 active+clean; 835 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-09-10T22:23:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:23 smithi171 ceph-mon[34138]: pgmap v1030: 129 pgs: 129 active+clean; 835 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 235 op/s 2024-09-10T22:23:24.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32653 4.78 MB/sec execute 255 sec latency 65.624 ms 2024-09-10T22:23:25.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32783 4.79 MB/sec execute 256 sec latency 73.336 ms 2024-09-10T22:23:26.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:25 smithi090 ceph-mon[30350]: pgmap v1031: 129 pgs: 129 active+clean; 840 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-09-10T22:23:26.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:25 smithi154 ceph-mon[34495]: pgmap v1031: 129 pgs: 129 active+clean; 840 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-09-10T22:23:26.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:25 smithi171 ceph-mon[34138]: pgmap v1031: 129 pgs: 129 active+clean; 840 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.4 MiB/s wr, 241 op/s 2024-09-10T22:23:26.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32914 4.78 MB/sec execute 257 sec latency 69.810 ms 2024-09-10T22:23:27.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33034 4.77 MB/sec execute 258 sec latency 62.935 ms 2024-09-10T22:23:28.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:27 smithi090 ceph-mon[30350]: pgmap v1032: 129 pgs: 129 active+clean; 845 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 254 op/s 2024-09-10T22:23:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:27 smithi154 ceph-mon[34495]: pgmap v1032: 129 pgs: 129 active+clean; 845 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 254 op/s 2024-09-10T22:23:28.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:27 smithi171 ceph-mon[34138]: pgmap v1032: 129 pgs: 129 active+clean; 845 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 254 op/s 2024-09-10T22:23:28.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33145 4.77 MB/sec execute 259 sec latency 131.859 ms 2024-09-10T22:23:29.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33260 4.76 MB/sec execute 260 sec latency 61.242 ms 2024-09-10T22:23:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:29 smithi090 ceph-mon[30350]: pgmap v1033: 129 pgs: 129 active+clean; 846 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-09-10T22:23:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:29 smithi154 ceph-mon[34495]: pgmap v1033: 129 pgs: 129 active+clean; 846 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-09-10T22:23:30.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:29 smithi171 ceph-mon[34138]: pgmap v1033: 129 pgs: 129 active+clean; 846 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-09-10T22:23:30.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33391 4.77 MB/sec execute 261 sec latency 90.423 ms 2024-09-10T22:23:31.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33536 4.79 MB/sec execute 262 sec latency 156.206 ms 2024-09-10T22:23:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:31 smithi090 ceph-mon[30350]: pgmap v1034: 129 pgs: 129 active+clean; 852 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 264 op/s 2024-09-10T22:23:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:31 smithi154 ceph-mon[34495]: pgmap v1034: 129 pgs: 129 active+clean; 852 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 264 op/s 2024-09-10T22:23:32.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:31 smithi171 ceph-mon[34138]: pgmap v1034: 129 pgs: 129 active+clean; 852 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 264 op/s 2024-09-10T22:23:32.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33691 4.82 MB/sec execute 263 sec latency 75.409 ms 2024-09-10T22:23:33.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33844 4.84 MB/sec execute 264 sec latency 74.941 ms 2024-09-10T22:23:34.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:33 smithi090 ceph-mon[30350]: pgmap v1035: 129 pgs: 129 active+clean; 856 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-09-10T22:23:34.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:33 smithi154 ceph-mon[34495]: pgmap v1035: 129 pgs: 129 active+clean; 856 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-09-10T22:23:34.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:33 smithi171 ceph-mon[34138]: pgmap v1035: 129 pgs: 129 active+clean; 856 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-09-10T22:23:34.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33992 4.86 MB/sec execute 265 sec latency 74.625 ms 2024-09-10T22:23:35.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34098 4.86 MB/sec execute 266 sec latency 96.892 ms 2024-09-10T22:23:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:35 smithi090 ceph-mon[30350]: pgmap v1036: 129 pgs: 129 active+clean; 857 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-09-10T22:23:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:35 smithi154 ceph-mon[34495]: pgmap v1036: 129 pgs: 129 active+clean; 857 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-09-10T22:23:36.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:35 smithi171 ceph-mon[34138]: pgmap v1036: 129 pgs: 129 active+clean; 857 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-09-10T22:23:36.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34238 4.88 MB/sec execute 267 sec latency 253.091 ms 2024-09-10T22:23:37.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34363 4.88 MB/sec execute 268 sec latency 72.531 ms 2024-09-10T22:23:38.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:37 smithi090 ceph-mon[30350]: pgmap v1037: 129 pgs: 129 active+clean; 863 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-09-10T22:23:38.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:37 smithi154 ceph-mon[34495]: pgmap v1037: 129 pgs: 129 active+clean; 863 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-09-10T22:23:38.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:37 smithi171 ceph-mon[34138]: pgmap v1037: 129 pgs: 129 active+clean; 863 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-09-10T22:23:38.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34492 4.88 MB/sec execute 269 sec latency 93.770 ms 2024-09-10T22:23:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:38 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:39.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:38 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:39.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:38 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:39.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34610 4.88 MB/sec execute 270 sec latency 99.958 ms 2024-09-10T22:23:40.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:39 smithi090 ceph-mon[30350]: pgmap v1038: 129 pgs: 129 active+clean; 867 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 253 op/s 2024-09-10T22:23:40.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:39 smithi154 ceph-mon[34495]: pgmap v1038: 129 pgs: 129 active+clean; 867 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 253 op/s 2024-09-10T22:23:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:39 smithi171 ceph-mon[34138]: pgmap v1038: 129 pgs: 129 active+clean; 867 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 253 op/s 2024-09-10T22:23:40.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34721 4.88 MB/sec execute 271 sec latency 70.191 ms 2024-09-10T22:23:41.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34827 4.87 MB/sec execute 272 sec latency 157.114 ms 2024-09-10T22:23:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:41 smithi090 ceph-mon[30350]: pgmap v1039: 129 pgs: 129 active+clean; 871 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 297 op/s 2024-09-10T22:23:42.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:41 smithi154 ceph-mon[34495]: pgmap v1039: 129 pgs: 129 active+clean; 871 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 297 op/s 2024-09-10T22:23:42.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:41 smithi171 ceph-mon[34138]: pgmap v1039: 129 pgs: 129 active+clean; 871 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 297 op/s 2024-09-10T22:23:42.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34970 4.89 MB/sec execute 273 sec latency 141.520 ms 2024-09-10T22:23:43.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35101 4.89 MB/sec execute 274 sec latency 78.239 ms 2024-09-10T22:23:44.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:43 smithi090 ceph-mon[30350]: pgmap v1040: 129 pgs: 129 active+clean; 873 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2024-09-10T22:23:44.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:43 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:44.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:43 smithi154 ceph-mon[34495]: pgmap v1040: 129 pgs: 129 active+clean; 873 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2024-09-10T22:23:44.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:43 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:44.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:43 smithi171 ceph-mon[34138]: pgmap v1040: 129 pgs: 129 active+clean; 873 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2024-09-10T22:23:44.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:43 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:23:44.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35226 4.91 MB/sec execute 275 sec latency 82.319 ms 2024-09-10T22:23:45.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35339 4.91 MB/sec execute 276 sec latency 64.531 ms 2024-09-10T22:23:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:45 smithi090 ceph-mon[30350]: pgmap v1041: 129 pgs: 129 active+clean; 881 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 275 op/s 2024-09-10T22:23:46.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:45 smithi154 ceph-mon[34495]: pgmap v1041: 129 pgs: 129 active+clean; 881 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 275 op/s 2024-09-10T22:23:46.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:45 smithi171 ceph-mon[34138]: pgmap v1041: 129 pgs: 129 active+clean; 881 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.1 MiB/s wr, 275 op/s 2024-09-10T22:23:46.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35476 4.92 MB/sec execute 277 sec latency 79.494 ms 2024-09-10T22:23:47.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35628 4.95 MB/sec execute 278 sec latency 92.243 ms 2024-09-10T22:23:48.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:47 smithi090 ceph-mon[30350]: pgmap v1042: 129 pgs: 129 active+clean; 884 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-09-10T22:23:48.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:47 smithi154 ceph-mon[34495]: pgmap v1042: 129 pgs: 129 active+clean; 884 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-09-10T22:23:48.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:47 smithi171 ceph-mon[34138]: pgmap v1042: 129 pgs: 129 active+clean; 884 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-09-10T22:23:48.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35704 4.95 MB/sec execute 279 sec latency 498.251 ms 2024-09-10T22:23:49.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35832 4.95 MB/sec execute 280 sec latency 64.064 ms 2024-09-10T22:23:50.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:49 smithi154 ceph-mon[34495]: pgmap v1043: 129 pgs: 129 active+clean; 885 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 245 op/s 2024-09-10T22:23:50.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:49 smithi171 ceph-mon[34138]: pgmap v1043: 129 pgs: 129 active+clean; 885 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 245 op/s 2024-09-10T22:23:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:49 smithi090 ceph-mon[30350]: pgmap v1043: 129 pgs: 129 active+clean; 885 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 245 op/s 2024-09-10T22:23:50.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35974 4.96 MB/sec execute 281 sec latency 69.781 ms 2024-09-10T22:23:51.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36051 4.96 MB/sec execute 282 sec latency 416.691 ms 2024-09-10T22:23:52.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:51 smithi154 ceph-mon[34495]: pgmap v1044: 129 pgs: 129 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-09-10T22:23:52.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:51 smithi171 ceph-mon[34138]: pgmap v1044: 129 pgs: 129 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-09-10T22:23:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:51 smithi090 ceph-mon[30350]: pgmap v1044: 129 pgs: 129 active+clean; 888 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-09-10T22:23:52.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36157 4.96 MB/sec execute 283 sec latency 102.920 ms 2024-09-10T22:23:53.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36215 4.94 MB/sec execute 284 sec latency 457.301 ms 2024-09-10T22:23:54.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:53 smithi154 ceph-mon[34495]: pgmap v1045: 129 pgs: 129 active+clean; 889 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-09-10T22:23:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:53 smithi171 ceph-mon[34138]: pgmap v1045: 129 pgs: 129 active+clean; 889 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-09-10T22:23:54.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:53 smithi090 ceph-mon[30350]: pgmap v1045: 129 pgs: 129 active+clean; 889 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 240 op/s 2024-09-10T22:23:54.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36318 4.95 MB/sec execute 285 sec latency 609.018 ms 2024-09-10T22:23:55.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36452 4.95 MB/sec execute 286 sec latency 397.765 ms 2024-09-10T22:23:56.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:55 smithi154 ceph-mon[34495]: pgmap v1046: 129 pgs: 129 active+clean; 896 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 274 op/s 2024-09-10T22:23:56.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:55 smithi171 ceph-mon[34138]: pgmap v1046: 129 pgs: 129 active+clean; 896 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 274 op/s 2024-09-10T22:23:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:55 smithi090 ceph-mon[30350]: pgmap v1046: 129 pgs: 129 active+clean; 896 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 274 op/s 2024-09-10T22:23:56.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36566 4.95 MB/sec execute 287 sec latency 105.104 ms 2024-09-10T22:23:57.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36698 4.95 MB/sec execute 288 sec latency 60.793 ms 2024-09-10T22:23:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:57 smithi154 ceph-mon[34495]: pgmap v1047: 129 pgs: 129 active+clean; 899 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-09-10T22:23:58.226 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:57 smithi171 ceph-mon[34138]: pgmap v1047: 129 pgs: 129 active+clean; 899 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-09-10T22:23:58.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:57 smithi090 ceph-mon[30350]: pgmap v1047: 129 pgs: 129 active+clean; 899 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 249 op/s 2024-09-10T22:23:58.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36827 4.96 MB/sec execute 289 sec latency 221.948 ms 2024-09-10T22:23:59.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36921 4.97 MB/sec execute 290 sec latency 586.950 ms 2024-09-10T22:24:00.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:23:59 smithi154 ceph-mon[34495]: pgmap v1048: 129 pgs: 129 active+clean; 904 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-09-10T22:24:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:23:59 smithi171 ceph-mon[34138]: pgmap v1048: 129 pgs: 129 active+clean; 904 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-09-10T22:24:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:23:59 smithi090 ceph-mon[30350]: pgmap v1048: 129 pgs: 129 active+clean; 904 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-09-10T22:24:00.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37068 4.99 MB/sec execute 291 sec latency 73.343 ms 2024-09-10T22:24:01.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37079 4.97 MB/sec execute 292 sec latency 873.610 ms 2024-09-10T22:24:02.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:01 smithi154 ceph-mon[34495]: pgmap v1049: 129 pgs: 129 active+clean; 905 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 259 op/s 2024-09-10T22:24:02.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:01 smithi171 ceph-mon[34138]: pgmap v1049: 129 pgs: 129 active+clean; 905 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 259 op/s 2024-09-10T22:24:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:01 smithi090 ceph-mon[30350]: pgmap v1049: 129 pgs: 129 active+clean; 905 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 259 op/s 2024-09-10T22:24:02.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37161 4.97 MB/sec execute 293 sec latency 1165.228 ms 2024-09-10T22:24:03.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37281 4.98 MB/sec execute 294 sec latency 70.877 ms 2024-09-10T22:24:04.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:03 smithi154 ceph-mon[34495]: pgmap v1050: 129 pgs: 129 active+clean; 905 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-09-10T22:24:04.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:03 smithi171 ceph-mon[34138]: pgmap v1050: 129 pgs: 129 active+clean; 905 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-09-10T22:24:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:03 smithi090 ceph-mon[30350]: pgmap v1050: 129 pgs: 129 active+clean; 905 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 201 op/s 2024-09-10T22:24:04.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37386 4.99 MB/sec execute 295 sec latency 159.651 ms 2024-09-10T22:24:05.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:04 smithi154 ceph-mon[34495]: osdmap e119: 12 total, 12 up, 12 in 2024-09-10T22:24:05.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:04 smithi171 ceph-mon[34138]: osdmap e119: 12 total, 12 up, 12 in 2024-09-10T22:24:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:04 smithi090 ceph-mon[30350]: osdmap e119: 12 total, 12 up, 12 in 2024-09-10T22:24:05.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37543 5.00 MB/sec execute 296 sec latency 73.793 ms 2024-09-10T22:24:06.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:05 smithi154 ceph-mon[34495]: pgmap v1052: 129 pgs: 129 active+clean; 910 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 228 op/s 2024-09-10T22:24:06.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:05 smithi171 ceph-mon[34138]: pgmap v1052: 129 pgs: 129 active+clean; 910 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 228 op/s 2024-09-10T22:24:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:05 smithi090 ceph-mon[30350]: pgmap v1052: 129 pgs: 129 active+clean; 910 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 228 op/s 2024-09-10T22:24:06.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37674 5.00 MB/sec execute 297 sec latency 269.572 ms 2024-09-10T22:24:07.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37788 5.00 MB/sec execute 298 sec latency 68.941 ms 2024-09-10T22:24:08.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:07 smithi154 ceph-mon[34495]: pgmap v1053: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 905 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 246 op/s 2024-09-10T22:24:08.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:07 smithi171 ceph-mon[34138]: pgmap v1053: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 905 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 246 op/s 2024-09-10T22:24:08.341 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T22:24:08.341 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T22:24:08.341 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:24:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:07 smithi090 ceph-mon[30350]: pgmap v1053: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 905 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 246 op/s 2024-09-10T22:24:08.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37897 5.01 MB/sec execute 299 sec latency 73.872 ms 2024-09-10T22:24:08.964 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:24:09.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.833+0000 7f1110400640 1 -- 172.21.15.90:0/3925012140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810b340 msgr2=0x7f1108111b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:09.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.833+0000 7f1110400640 1 --2- 172.21.15.90:0/3925012140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810b340 0x7f1108111b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1100009be0 tx=0x7f110002a6c0 comp rx=0 tx=0).stop 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.833+0000 7f1110400640 1 -- 172.21.15.90:0/3925012140 shutdown_connections 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.833+0000 7f1110400640 1 --2- 172.21.15.90:0/3925012140 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810b340 0x7f1108111b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.833+0000 7f1110400640 1 --2- 172.21.15.90:0/3925012140 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f110810a9c0 0x7f110810ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.833+0000 7f1110400640 1 --2- 172.21.15.90:0/3925012140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1108104fb0 0x7f1108105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.833+0000 7f1110400640 1 -- 172.21.15.90:0/3925012140 >> 172.21.15.90:0/3925012140 conn(0x7f1108100a20 msgr2=0x7f1108102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.833+0000 7f1110400640 1 -- 172.21.15.90:0/3925012140 shutdown_connections 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.834+0000 7f1110400640 1 -- 172.21.15.90:0/3925012140 wait complete. 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.835+0000 7f1110400640 1 Processor -- start 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.835+0000 7f1110400640 1 -- start start 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.836+0000 7f1110400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1108104fb0 0x7f110819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:09.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.836+0000 7f1110400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810a9c0 0x7f110819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.836+0000 7f1110400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f110810b340 0x7f11081a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.836+0000 7f1110400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1108114e20 con 0x7f1108104fb0 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.836+0000 7f1110400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f1108114ca0 con 0x7f110810b340 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.836+0000 7f1110400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f1108114fa0 con 0x7f110810a9c0 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.836+0000 7f110d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810a9c0 0x7f110819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.836+0000 7f110d000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810a9c0 0x7f110819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:35184/0 (socket says 172.21.15.90:35184) 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.836+0000 7f110d000640 1 -- 172.21.15.90:0/2904382133 learned_addr learned my addr 172.21.15.90:0/2904382133 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.837+0000 7f110e400640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f110810b340 0x7f11081a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.837+0000 7f110da00640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1108104fb0 0x7f110819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.837+0000 7f110d000640 1 -- 172.21.15.90:0/2904382133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f110810b340 msgr2=0x7f11081a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:09.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.837+0000 7f110d000640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f110810b340 0x7f11081a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.837+0000 7f110d000640 1 -- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1108104fb0 msgr2=0x7f110819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.837+0000 7f110d000640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1108104fb0 0x7f110819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.837+0000 7f110d000640 1 -- 172.21.15.90:0/2904382133 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11081a13c0 con 0x7f110810a9c0 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.838+0000 7f110d000640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810a9c0 0x7f110819b850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f10f400e9c0 tx=0x7f10f400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.838+0000 7f1106000640 1 -- 172.21.15.90:0/2904382133 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f400ccc0 con 0x7f110810a9c0 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.838+0000 7f1106000640 1 -- 172.21.15.90:0/2904382133 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f10f40054b0 con 0x7f110810a9c0 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.838+0000 7f1106000640 1 -- 172.21.15.90:0/2904382133 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f4010430 con 0x7f110810a9c0 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.839+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11081a16b0 con 0x7f110810a9c0 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.839+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11081a7d70 con 0x7f110810a9c0 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.840+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1108105390 con 0x7f110810a9c0 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.845+0000 7f1106000640 1 -- 172.21.15.90:0/2904382133 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f10f4004eb0 con 0x7f110810a9c0 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.849+0000 7f1106000640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f10f00b55a0 0x7f10f00b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:09.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.849+0000 7f1106000640 1 -- 172.21.15.90:0/2904382133 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9578+0+0 (secure 0 0 0) 0x7f10f40e6d40 con 0x7f110810a9c0 2024-09-10T22:24:09.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.849+0000 7f1106000640 1 -- 172.21.15.90:0/2904382133 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f10f40e7260 con 0x7f110810a9c0 2024-09-10T22:24:09.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.849+0000 7f110da00640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f10f00b55a0 0x7f10f00b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:09.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:09.850+0000 7f110da00640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f10f00b55a0 0x7f10f00b7a60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0095f0 tx=0x7f10fc005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:09.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37996 5.00 MB/sec execute 300 sec latency 72.630 ms 2024-09-10T22:24:10.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:09 smithi090 ceph-mon[30350]: pgmap v1054: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 895 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 212 op/s 2024-09-10T22:24:10.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:09 smithi154 ceph-mon[34495]: pgmap v1054: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 895 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 212 op/s 2024-09-10T22:24:10.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:09 smithi171 ceph-mon[34138]: pgmap v1054: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 895 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 212 op/s 2024-09-10T22:24:10.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.264+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f110819c440 con 0x7f110810a9c0 2024-09-10T22:24:10.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.265+0000 7f1106000640 1 -- 172.21.15.90:0/2904382133 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+12012 (secure 0 0 0) 0x7f110819c440 con 0x7f110810a9c0 2024-09-10T22:24:10.267 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:24:10.267 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":76,"btime":"2024-09-10T22:23:42:837927+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:23:42.763306+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:24:10.268 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 76 2024-09-10T22:24:10.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.268+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f10f00b55a0 msgr2=0x7f10f00b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:10.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.268+0000 7f1110400640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f10f00b55a0 0x7f10f00b7a60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0095f0 tx=0x7f10fc005f90 comp rx=0 tx=0).stop 2024-09-10T22:24:10.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.268+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810a9c0 msgr2=0x7f110819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:10.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.268+0000 7f1110400640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810a9c0 0x7f110819b850 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f10f400e9c0 tx=0x7f10f400ee80 comp rx=0 tx=0).stop 2024-09-10T22:24:10.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.269+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 shutdown_connections 2024-09-10T22:24:10.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.269+0000 7f1110400640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f10f00b55a0 0x7f10f00b7a60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:10.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.269+0000 7f1110400640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f110810b340 0x7f11081a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:10.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.269+0000 7f1110400640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f110810a9c0 0x7f110819b850 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:10.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.269+0000 7f1110400640 1 --2- 172.21.15.90:0/2904382133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1108104fb0 0x7f110819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:10.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.269+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 >> 172.21.15.90:0/2904382133 conn(0x7f1108100a20 msgr2=0x7f1108102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:10.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.269+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 shutdown_connections 2024-09-10T22:24:10.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:10.269+0000 7f1110400640 1 -- 172.21.15.90:0/2904382133 wait complete. 2024-09-10T22:24:10.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38111 5.01 MB/sec execute 301 sec latency 136.569 ms 2024-09-10T22:24:11.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:10 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2904382133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:24:11.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:10 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2904382133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:24:11.279 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T22:24:11.280 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T22:24:11.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:10 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2904382133' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:24:11.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38224 5.01 MB/sec execute 302 sec latency 78.789 ms 2024-09-10T22:24:11.909 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:24:12.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:11 smithi171 ceph-mon[34138]: pgmap v1055: 129 pgs: 129 active+clean; 906 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 242 op/s 2024-09-10T22:24:12.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:11 smithi090 ceph-mon[30350]: pgmap v1055: 129 pgs: 129 active+clean; 906 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 242 op/s 2024-09-10T22:24:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:11 smithi154 ceph-mon[34495]: pgmap v1055: 129 pgs: 129 active+clean; 906 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 242 op/s 2024-09-10T22:24:12.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.667+0000 7f622ea00640 1 -- 172.21.15.90:0/1395685234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6228108020 msgr2=0x7f622810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:12.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.667+0000 7f622ea00640 1 --2- 172.21.15.90:0/1395685234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6228108020 0x7f622810e820 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6220009ca0 tx=0x7f622002a8d0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.668+0000 7f622ea00640 1 -- 172.21.15.90:0/1395685234 shutdown_connections 2024-09-10T22:24:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.668+0000 7f622ea00640 1 --2- 172.21.15.90:0/1395685234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6228108020 0x7f622810e820 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.668+0000 7f622ea00640 1 --2- 172.21.15.90:0/1395685234 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f62281076a0 0x7f6228107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.668+0000 7f622ea00640 1 --2- 172.21.15.90:0/1395685234 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6228101c90 0x7f6228102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.668+0000 7f622ea00640 1 -- 172.21.15.90:0/1395685234 >> 172.21.15.90:0/1395685234 conn(0x7f62280fd700 msgr2=0x7f62280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.668+0000 7f622ea00640 1 -- 172.21.15.90:0/1395685234 shutdown_connections 2024-09-10T22:24:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.668+0000 7f622ea00640 1 -- 172.21.15.90:0/1395685234 wait complete. 2024-09-10T22:24:12.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.670+0000 7f622ea00640 1 Processor -- start 2024-09-10T22:24:12.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.670+0000 7f622ea00640 1 -- start start 2024-09-10T22:24:12.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.671+0000 7f622ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6228101c90 0x7f622819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.671+0000 7f622ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62281076a0 0x7f622819d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.671+0000 7f622ea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6228108020 0x7f62281a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.671+0000 7f622ea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6228110a30 con 0x7f6228101c90 2024-09-10T22:24:12.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.671+0000 7f622ea00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f62281108b0 con 0x7f6228108020 2024-09-10T22:24:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.671+0000 7f622ea00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6228110bb0 con 0x7f62281076a0 2024-09-10T22:24:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.672+0000 7f6227400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62281076a0 0x7f622819d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.672+0000 7f6227400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62281076a0 0x7f622819d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:35194/0 (socket says 172.21.15.90:35194) 2024-09-10T22:24:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.672+0000 7f6227400640 1 -- 172.21.15.90:0/3452642183 learned_addr learned my addr 172.21.15.90:0/3452642183 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.672+0000 7f622ca00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6228108020 0x7f62281a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.673+0000 7f6227e00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6228101c90 0x7f622819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.673+0000 7f6227400640 1 -- 172.21.15.90:0/3452642183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6228108020 msgr2=0x7f62281a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.673+0000 7f6227400640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6228108020 0x7f62281a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.673+0000 7f6227400640 1 -- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6228101c90 msgr2=0x7f622819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:12.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.673+0000 7f6227400640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6228101c90 0x7f622819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.673+0000 7f6227400640 1 -- 172.21.15.90:0/3452642183 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62281a1260 con 0x7f62281076a0 2024-09-10T22:24:12.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.673+0000 7f6227e00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6228101c90 0x7f622819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:24:12.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.674+0000 7f6227400640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62281076a0 0x7f622819d1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f621400b3d0 tx=0x7f621400b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:12.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.675+0000 7f6224c00640 1 -- 172.21.15.90:0/3452642183 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214015020 con 0x7f62281076a0 2024-09-10T22:24:12.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.675+0000 7f6224c00640 1 -- 172.21.15.90:0/3452642183 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6214011cc0 con 0x7f62281076a0 2024-09-10T22:24:12.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.675+0000 7f622ea00640 1 -- 172.21.15.90:0/3452642183 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62281a1550 con 0x7f62281076a0 2024-09-10T22:24:12.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.675+0000 7f622ea00640 1 -- 172.21.15.90:0/3452642183 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62281a1b60 con 0x7f62281076a0 2024-09-10T22:24:12.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.676+0000 7f6224c00640 1 -- 172.21.15.90:0/3452642183 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6214010d60 con 0x7f62281076a0 2024-09-10T22:24:12.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.679+0000 7f6231459740 1 Processor -- start 2024-09-10T22:24:12.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.680+0000 7f6231459740 1 -- start start 2024-09-10T22:24:12.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.680+0000 7f6231459740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f7c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.680+0000 7f6231459740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f884f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.680+0000 7f6231459740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 0x55fda0f8ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.680+0000 7f6231459740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fda0ef1ca0 con 0x55fda0f88a30 2024-09-10T22:24:12.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.680+0000 7f6231459740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fda0ee10d0 con 0x55fda0f7c180 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.680+0000 7f6231459740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fda0ee4ee0 con 0x55fda0f7caa0 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.681+0000 7f622ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f7c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.681+0000 7f622ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f7c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:39146/0 (socket says 172.21.15.90:39146) 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.681+0000 7f622ca00640 1 -- 172.21.15.90:0/3942358551 learned_addr learned my addr 172.21.15.90:0/3942358551 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.681+0000 7f6227e00640 1 --2- 172.21.15.90:0/3942358551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f884f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.681+0000 7f622ca00640 1 -- 172.21.15.90:0/3942358551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 msgr2=0x55fda0f884f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.681+0000 7f622ca00640 1 --2- 172.21.15.90:0/3942358551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f884f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.681+0000 7f622ca00640 1 -- 172.21.15.90:0/3942358551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 msgr2=0x55fda0f8ae20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.681+0000 7f622ca00640 1 --2- 172.21.15.90:0/3942358551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 0x55fda0f8ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.681+0000 7f622ca00640 1 -- 172.21.15.90:0/3942358551 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda0e93400 con 0x55fda0f7c180 2024-09-10T22:24:12.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.682+0000 7f622ca00640 1 --2- 172.21.15.90:0/3942358551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f7c560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6220035000 tx=0x7f6220031510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f02c7bb232652e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.683+0000 7f621d200640 1 -- 172.21.15.90:0/3942358551 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220031990 con 0x55fda0f7c180 2024-09-10T22:24:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.684+0000 7f6224c00640 1 -- 172.21.15.90:0/3452642183 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f621401a400 con 0x7f62281076a0 2024-09-10T22:24:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.684+0000 7f621d200640 1 -- 172.21.15.90:0/3942358551 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6220036030 con 0x55fda0f7c180 2024-09-10T22:24:12.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.684+0000 7f621d200640 1 -- 172.21.15.90:0/3942358551 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62200043c0 con 0x55fda0f7c180 2024-09-10T22:24:12.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.687+0000 7f6224c00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f620c0b5580 0x7f620c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.687+0000 7f6231459740 1 -- 172.21.15.90:0/3942358551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 msgr2=0x55fda0f7c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:12.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.687+0000 7f6231459740 1 --2- 172.21.15.90:0/3942358551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f7c560 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6220035000 tx=0x7f6220031510 comp rx=0 tx=0).stop 2024-09-10T22:24:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.688+0000 7f6224c00640 1 -- 172.21.15.90:0/3452642183 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9578+0+0 (secure 0 0 0) 0x7f62140e5c20 con 0x7f62281076a0 2024-09-10T22:24:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.688+0000 7f6231459740 1 -- 172.21.15.90:0/3942358551 shutdown_connections 2024-09-10T22:24:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.688+0000 7f6231459740 1 --2- 172.21.15.90:0/3942358551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 0x55fda0f8ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.688+0000 7f6231459740 1 --2- 172.21.15.90:0/3942358551 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f884f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.689+0000 7f6231459740 1 --2- 172.21.15.90:0/3942358551 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f7c560 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.689+0000 7f6231459740 1 -- 172.21.15.90:0/3942358551 >> 172.21.15.90:0/3942358551 conn(0x55fda0f79fb0 msgr2=0x55fda0f7b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:12.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.690+0000 7f6227e00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f620c0b5580 0x7f620c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.690+0000 7f6231459740 1 -- 172.21.15.90:0/3942358551 shutdown_connections 2024-09-10T22:24:12.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.690+0000 7f6231459740 1 -- 172.21.15.90:0/3942358551 wait complete. 2024-09-10T22:24:12.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.691+0000 7f6231459740 1 -- start start 2024-09-10T22:24:12.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.691+0000 7f6231459740 1 Processor -- start 2024-09-10T22:24:12.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.693+0000 7f6231459740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f7c180 0x55fda0f88320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.693+0000 7f6231459740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f85770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.693+0000 7f6231459740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f88a30 0x55fda0f85da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.694+0000 7f6231459740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55fda0d427a0 con 0x55fda0f7c180 2024-09-10T22:24:12.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.694+0000 7f6231459740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55fda0e86a10 con 0x55fda0f88a30 2024-09-10T22:24:12.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.694+0000 7f6231459740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55fda0f8b4e0 con 0x55fda0f7caa0 2024-09-10T22:24:12.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.694+0000 7f6231459740 10 client.0 _ll_register_callbacks cb 0x55fda0e96be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:24:12.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.694+0000 7f622ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f88a30 0x55fda0f85da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.694+0000 7f622ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f88a30 0x55fda0f85da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:39150/0 (socket says 172.21.15.90:39150) 2024-09-10T22:24:12.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.694+0000 7f622ca00640 1 -- 172.21.15.90:0/3871095910 learned_addr learned my addr 172.21.15.90:0/3871095910 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:12.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.694+0000 7f6227400640 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f85770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.695+0000 7f6227400640 1 -- 172.21.15.90:0/3871095910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f88a30 msgr2=0x55fda0f85da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:12.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.695+0000 7f6227400640 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f88a30 0x55fda0f85da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.695+0000 7f6227400640 1 -- 172.21.15.90:0/3871095910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f7c180 msgr2=0x55fda0f88320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:12.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.695+0000 7f6227400640 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f7c180 0x55fda0f88320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.695+0000 7f6227400640 1 -- 172.21.15.90:0/3871095910 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda0c4eb90 con 0x55fda0f7caa0 2024-09-10T22:24:12.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.696+0000 7f6227400640 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f85770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f62140e2ef0 tx=0x7f62140ab570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dccee01ab07e0bca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:12.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.696+0000 7f6211600640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:24:12.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.697+0000 7f6211600640 1 -- 172.21.15.90:0/3871095910 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140e3370 con 0x55fda0f7caa0 2024-09-10T22:24:12.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.697+0000 7f6211600640 1 -- 172.21.15.90:0/3871095910 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9578+0+0 (secure 0 0 0) 0x7f6214011830 con 0x55fda0f7caa0 2024-09-10T22:24:12.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.697+0000 7f6231459740 1 -- 172.21.15.90:0/3871095910 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6220036030 con 0x55fda0f7caa0 2024-09-10T22:24:12.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.698+0000 7f6211600640 1 -- 172.21.15.90:0/3871095910 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f62140ab730 con 0x55fda0f7caa0 2024-09-10T22:24:12.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.698+0000 7f6231459740 10 client.26035 fetch_fsmap learned FSMap version 76 2024-09-10T22:24:12.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.698+0000 7f6231459740 1 -- 172.21.15.90:0/3871095910 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55fda0f96e80 con 0x55fda0f7caa0 2024-09-10T22:24:12.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.698+0000 7f6227e00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f620c0b5580 0x7f620c0b7a40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6218005fd0 tx=0x7f621800bf50 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:12.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.698+0000 7f6211600640 1 -- 172.21.15.90:0/3871095910 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6220036030 con 0x55fda0f7caa0 2024-09-10T22:24:12.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.699+0000 7f6211600640 1 -- 172.21.15.90:0/3871095910 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 76) ==== 8430+0+0 (secure 0 0 0) 0x7f62140b0180 con 0x55fda0f7caa0 2024-09-10T22:24:12.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.699+0000 7f6231459740 10 client.26035 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T22:24:12.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.699+0000 7f6231459740 10 client.26035 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:24:12.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.699+0000 7f6231459740 20 client.26035 populate_metadata read hostname 'smithi090' 2024-09-10T22:24:12.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.699+0000 7f6231459740 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fda0f986e0 0x55fda0f9ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.699+0000 7f6231459740 4 client.26035 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:24:12.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.699+0000 7f6231459740 1 -- 172.21.15.90:0/3871095910 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fda0ef3bf0 con 0x55fda0f986e0 2024-09-10T22:24:12.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.700+0000 7f6227e00640 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fda0f986e0 0x55fda0f9ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.701+0000 7f6211600640 10 client.26035 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:12.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.702+0000 7f6227e00640 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fda0f986e0 0x55fda0f9ab00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199c959487ce9659 server_cookie=80d68e7a08efecf7 in_seq=0 out_seq=0 2024-09-10T22:24:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.703+0000 7f6211600640 1 -- 172.21.15.90:0/3871095910 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fda0ef3bf0 con 0x55fda0f986e0 2024-09-10T22:24:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.703+0000 7f6211600640 10 client.26035 handle_command_reply: tid=0 2024-09-10T22:24:12.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.703+0000 7f6231459740 1 client.26035 shutdown 2024-09-10T22:24:12.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 -- 172.21.15.90:0/3871095910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 msgr2=0x55fda0f85770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:12.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f85770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f62140e2ef0 tx=0x7f62140ab570 comp rx=0 tx=0).stop 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 -- 172.21.15.90:0/3871095910 shutdown_connections 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fda0f986e0 0x55fda0f9ab00 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f88a30 0x55fda0f85da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f85770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 --2- 172.21.15.90:0/3871095910 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f7c180 0x55fda0f88320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 -- 172.21.15.90:0/3871095910 >> 172.21.15.90:0/3871095910 conn(0x55fda0f79fb0 msgr2=0x55fda0f790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6211600640 0 client.26035 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 -- 172.21.15.90:0/3871095910 shutdown_connections 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.705+0000 7f6231459740 1 -- 172.21.15.90:0/3871095910 wait complete. 2024-09-10T22:24:12.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.706+0000 7f6231459740 20 client.26035 trim_cache size 0 max 16384 2024-09-10T22:24:12.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.744+0000 7f6231459740 1 Processor -- start 2024-09-10T22:24:12.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.744+0000 7f6231459740 1 -- start start 2024-09-10T22:24:12.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.745+0000 7f6231459740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f7c180 0x55fda0f82980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.745+0000 7f6231459740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7caa0 0x55fda0f82ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.745+0000 7f6231459740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f88a30 0x55fda0f78890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.745+0000 7f6231459740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f62080102c0 con 0x55fda0f7c180 2024-09-10T22:24:12.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.745+0000 7f6231459740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6208012010 con 0x55fda0f7caa0 2024-09-10T22:24:12.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.745+0000 7f6231459740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6208010a20 con 0x55fda0f88a30 2024-09-10T22:24:12.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.745+0000 7f6227400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7caa0 0x55fda0f82ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.745+0000 7f6227400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7caa0 0x55fda0f82ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:39156/0 (socket says 172.21.15.90:39156) 2024-09-10T22:24:12.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.745+0000 7f6227400640 1 -- 172.21.15.90:0/2697586242 learned_addr learned my addr 172.21.15.90:0/2697586242 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:12.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.746+0000 7f622ca00640 1 --2- 172.21.15.90:0/2697586242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f88a30 0x55fda0f78890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.747+0000 7f6227400640 1 -- 172.21.15.90:0/2697586242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f88a30 msgr2=0x55fda0f78890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:12.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.747+0000 7f6227400640 1 --2- 172.21.15.90:0/2697586242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f88a30 0x55fda0f78890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.747+0000 7f6227400640 1 -- 172.21.15.90:0/2697586242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f7c180 msgr2=0x55fda0f82980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:12.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.747+0000 7f6227400640 1 --2- 172.21.15.90:0/2697586242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f7c180 0x55fda0f82980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.748+0000 7f6227400640 1 -- 172.21.15.90:0/2697586242 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fda0f86110 con 0x55fda0f7caa0 2024-09-10T22:24:12.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.749+0000 7f6227400640 1 --2- 172.21.15.90:0/2697586242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7caa0 0x55fda0f82ec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62140abbb0 tx=0x7f62140ab6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89d2e74ac37c552 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:12.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.751+0000 7f6213e00640 1 -- 172.21.15.90:0/2697586242 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140c6080 con 0x55fda0f7caa0 2024-09-10T22:24:12.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.751+0000 7f6213e00640 1 -- 172.21.15.90:0/2697586242 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62140ac320 con 0x55fda0f7caa0 2024-09-10T22:24:12.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.751+0000 7f6213e00640 1 -- 172.21.15.90:0/2697586242 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140ab730 con 0x55fda0f7caa0 2024-09-10T22:24:12.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.753+0000 7f6231459740 1 -- 172.21.15.90:0/2697586242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7caa0 msgr2=0x55fda0f82ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:12.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.753+0000 7f6231459740 1 --2- 172.21.15.90:0/2697586242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7caa0 0x55fda0f82ec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62140abbb0 tx=0x7f62140ab6a0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.754+0000 7f6231459740 1 -- 172.21.15.90:0/2697586242 shutdown_connections 2024-09-10T22:24:12.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.754+0000 7f6231459740 1 --2- 172.21.15.90:0/2697586242 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f88a30 0x55fda0f78890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.754+0000 7f6231459740 1 --2- 172.21.15.90:0/2697586242 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7caa0 0x55fda0f82ec0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.754+0000 7f6231459740 1 --2- 172.21.15.90:0/2697586242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f7c180 0x55fda0f82980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.754+0000 7f6231459740 1 -- 172.21.15.90:0/2697586242 >> 172.21.15.90:0/2697586242 conn(0x55fda0f79fb0 msgr2=0x55fda0f7d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:12.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.757+0000 7f6231459740 1 -- 172.21.15.90:0/2697586242 shutdown_connections 2024-09-10T22:24:12.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.757+0000 7f6231459740 1 -- 172.21.15.90:0/2697586242 wait complete. 2024-09-10T22:24:12.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.758+0000 7f6231459740 1 -- start start 2024-09-10T22:24:12.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.759+0000 7f6231459740 1 Processor -- start 2024-09-10T22:24:12.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.760+0000 7f6231459740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f9d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.760+0000 7f6231459740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f9da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.761+0000 7f6231459740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 0x55fda0f9df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.761+0000 7f6231459740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6208011940 con 0x55fda0f88a30 2024-09-10T22:24:12.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.761+0000 7f6231459740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f62080111b0 con 0x55fda0f7c180 2024-09-10T22:24:12.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.761+0000 7f6231459740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f620800e280 con 0x55fda0f7caa0 2024-09-10T22:24:12.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.761+0000 7f6231459740 10 client.0 _ll_register_callbacks cb 0x7f62140a7b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:24:12.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.762+0000 7f6227e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f9d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.762+0000 7f6227400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f9da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.762+0000 7f6227e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f9d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:39166/0 (socket says 172.21.15.90:39166) 2024-09-10T22:24:12.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.762+0000 7f6227e00640 1 -- 172.21.15.90:0/2199212730 learned_addr learned my addr 172.21.15.90:0/2199212730 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:12.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.762+0000 7f622ca00640 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 0x55fda0f9df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.763+0000 7f6227e00640 1 -- 172.21.15.90:0/2199212730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 msgr2=0x55fda0f9da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:12.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.763+0000 7f6227e00640 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f9da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.763+0000 7f6227e00640 1 -- 172.21.15.90:0/2199212730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 msgr2=0x55fda0f9df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:12.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.763+0000 7f6227e00640 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 0x55fda0f9df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:12.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.763+0000 7f6227e00640 1 -- 172.21.15.90:0/2199212730 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda1005780 con 0x55fda0f7c180 2024-09-10T22:24:12.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.763+0000 7f622ca00640 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 0x55fda0f9df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:24:12.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.763+0000 7f6227e00640 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f9d510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55fda0f9b340 tx=0x7f6218005410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=984aaf74bd61fe94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:12.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.763+0000 7f6211600640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:24:12.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.764+0000 7f6211600640 1 -- 172.21.15.90:0/2199212730 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218004c10 con 0x55fda0f7c180 2024-09-10T22:24:12.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.764+0000 7f6211600640 1 -- 172.21.15.90:0/2199212730 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9578+0+0 (secure 0 0 0) 0x7f6218004db0 con 0x55fda0f7c180 2024-09-10T22:24:12.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.764+0000 7f6231459740 1 -- 172.21.15.90:0/2199212730 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62140ac320 con 0x55fda0f7c180 2024-09-10T22:24:12.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.765+0000 7f6211600640 1 -- 172.21.15.90:0/2199212730 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f621801a480 con 0x55fda0f7c180 2024-09-10T22:24:12.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.765+0000 7f6231459740 10 client.36053 fetch_fsmap learned FSMap version 76 2024-09-10T22:24:12.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.765+0000 7f6231459740 1 -- 172.21.15.90:0/2199212730 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55fda1006ac0 con 0x55fda0f7c180 2024-09-10T22:24:12.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.765+0000 7f6211600640 1 -- 172.21.15.90:0/2199212730 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62140ac320 con 0x55fda0f7c180 2024-09-10T22:24:12.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.766+0000 7f6211600640 1 -- 172.21.15.90:0/2199212730 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 76) ==== 8430+0+0 (secure 0 0 0) 0x7f6218035020 con 0x55fda0f7c180 2024-09-10T22:24:12.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.766+0000 7f6231459740 10 client.36053 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T22:24:12.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.766+0000 7f6231459740 10 client.36053 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:24:12.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.766+0000 7f6231459740 20 client.36053 populate_metadata read hostname 'smithi090' 2024-09-10T22:24:12.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.766+0000 7f6231459740 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fda10072c0 0x55fda10096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:12.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.766+0000 7f6231459740 4 client.36053 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T22:24:12.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.766+0000 7f6231459740 1 -- 172.21.15.90:0/2199212730 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55fda0eef6a0 con 0x55fda10072c0 2024-09-10T22:24:12.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.766+0000 7f6227400640 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fda10072c0 0x55fda10096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:12.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.767+0000 7f6211600640 10 client.36053 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:12.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:12.767+0000 7f6227400640 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fda10072c0 0x55fda10096e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124d0c80b7abf4b7 server_cookie=eaab65f38abad5bc in_seq=0 out_seq=0 2024-09-10T22:24:12.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38350 5.02 MB/sec execute 303 sec latency 174.699 ms 2024-09-10T22:24:13.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.205+0000 7f6211600640 1 -- 172.21.15.90:0/2199212730 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fda0eef6a0 con 0x55fda10072c0 2024-09-10T22:24:13.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.205+0000 7f6211600640 10 client.36053 handle_command_reply: tid=0 2024-09-10T22:24:13.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.205+0000 7f6231459740 1 client.36053 shutdown 2024-09-10T22:24:13.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 -- 172.21.15.90:0/2199212730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 msgr2=0x55fda0f9d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:13.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f9d510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55fda0f9b340 tx=0x7f6218005410 comp rx=0 tx=0).stop 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 -- 172.21.15.90:0/2199212730 shutdown_connections 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55fda10072c0 0x55fda10096e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55fda0f88a30 0x55fda0f9df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55fda0f7caa0 0x55fda0f9da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 --2- 172.21.15.90:0/2199212730 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55fda0f7c180 0x55fda0f9d510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 -- 172.21.15.90:0/2199212730 >> 172.21.15.90:0/2199212730 conn(0x55fda0f79fb0 msgr2=0x55fda0f790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6211600640 0 client.36053 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 -- 172.21.15.90:0/2199212730 shutdown_connections 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 1 -- 172.21.15.90:0/2199212730 wait complete. 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.206+0000 7f6231459740 20 client.36053 trim_cache size 0 max 16384 2024-09-10T22:24:13.208 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 76, 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 62, 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "1af74f6a": { 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 62, 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:24:13.209 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:13.210 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.207+0000 7f622ea00640 1 -- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f620c0b5580 msgr2=0x7f620c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.207+0000 7f622ea00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f620c0b5580 0x7f620c0b7a40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6218005fd0 tx=0x7f621800bf50 comp rx=0 tx=0).stop 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.207+0000 7f622ea00640 1 -- 172.21.15.90:0/3452642183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62281076a0 msgr2=0x7f622819d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.207+0000 7f622ea00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62281076a0 0x7f622819d1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f621400b3d0 tx=0x7f621400b890 comp rx=0 tx=0).stop 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.208+0000 7f622ea00640 1 -- 172.21.15.90:0/3452642183 shutdown_connections 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.208+0000 7f622ea00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f620c0b5580 0x7f620c0b7a40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.208+0000 7f622ea00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6228108020 0x7f62281a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.208+0000 7f622ea00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f62281076a0 0x7f622819d1c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.208+0000 7f622ea00640 1 --2- 172.21.15.90:0/3452642183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6228101c90 0x7f622819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:13.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.208+0000 7f622ea00640 1 -- 172.21.15.90:0/3452642183 >> 172.21.15.90:0/3452642183 conn(0x7f62280fd700 msgr2=0x7f6228104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:13.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.208+0000 7f622ea00640 1 -- 172.21.15.90:0/3452642183 shutdown_connections 2024-09-10T22:24:13.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:13.208+0000 7f622ea00640 1 -- 172.21.15.90:0/3452642183 wait complete. 2024-09-10T22:24:13.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 5.01 MB/sec execute 304 sec latency 775.345 ms 2024-09-10T22:24:14.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:13 smithi090 ceph-mon[30350]: pgmap v1056: 129 pgs: 129 active+clean; 906 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 250 op/s 2024-09-10T22:24:14.154 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1af74f6a', quiesce took 5.8 seconds. Will release after: 24.2 2024-09-10T22:24:14.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:13 smithi171 ceph-mon[34138]: pgmap v1056: 129 pgs: 129 active+clean; 906 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 250 op/s 2024-09-10T22:24:14.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:13 smithi154 ceph-mon[34495]: pgmap v1056: 129 pgs: 129 active+clean; 906 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 250 op/s 2024-09-10T22:24:14.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.99 MB/sec execute 305 sec latency 1775.420 ms 2024-09-10T22:24:15.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.97 MB/sec execute 306 sec latency 2775.491 ms 2024-09-10T22:24:16.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:15 smithi171 ceph-mon[34138]: pgmap v1057: 129 pgs: 129 active+clean; 910 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 208 op/s 2024-09-10T22:24:16.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:15 smithi171 ceph-mon[34138]: osdmap e120: 12 total, 12 up, 12 in 2024-09-10T22:24:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:15 smithi090 ceph-mon[30350]: pgmap v1057: 129 pgs: 129 active+clean; 910 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 208 op/s 2024-09-10T22:24:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:15 smithi090 ceph-mon[30350]: osdmap e120: 12 total, 12 up, 12 in 2024-09-10T22:24:16.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:15 smithi154 ceph-mon[34495]: pgmap v1057: 129 pgs: 129 active+clean; 910 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 208 op/s 2024-09-10T22:24:16.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:15 smithi154 ceph-mon[34495]: osdmap e120: 12 total, 12 up, 12 in 2024-09-10T22:24:16.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.96 MB/sec execute 307 sec latency 3775.540 ms 2024-09-10T22:24:17.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.94 MB/sec execute 308 sec latency 4775.608 ms 2024-09-10T22:24:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:17 smithi171 ceph-mon[34138]: pgmap v1059: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 239 op/s 2024-09-10T22:24:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:17 smithi090 ceph-mon[30350]: pgmap v1059: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 239 op/s 2024-09-10T22:24:18.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:17 smithi154 ceph-mon[34495]: pgmap v1059: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.2 MiB/s wr, 239 op/s 2024-09-10T22:24:18.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.92 MB/sec execute 309 sec latency 5775.699 ms 2024-09-10T22:24:19.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.91 MB/sec execute 310 sec latency 6775.768 ms 2024-09-10T22:24:20.220 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:19 smithi154 ceph-mon[34495]: pgmap v1060: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-09-10T22:24:20.220 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:19 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:24:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:19 smithi171 ceph-mon[34138]: pgmap v1060: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-09-10T22:24:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:19 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:24:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:19 smithi090 ceph-mon[30350]: pgmap v1060: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.7 MiB/s wr, 222 op/s 2024-09-10T22:24:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:19 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:24:20.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.89 MB/sec execute 311 sec latency 7775.814 ms 2024-09-10T22:24:21.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:24:21.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:24:21.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:24:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:24:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:24:21.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:24:21.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:24:21.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:24:21.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:24:21.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.88 MB/sec execute 312 sec latency 8775.862 ms 2024-09-10T22:24:22.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:21 smithi171 ceph-mon[34138]: pgmap v1061: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-09-10T22:24:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:21 smithi090 ceph-mon[30350]: pgmap v1061: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-09-10T22:24:22.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:21 smithi154 ceph-mon[34495]: pgmap v1061: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 140 op/s 2024-09-10T22:24:22.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.86 MB/sec execute 313 sec latency 9775.973 ms 2024-09-10T22:24:23.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.85 MB/sec execute 314 sec latency 10776.097 ms 2024-09-10T22:24:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:23 smithi171 ceph-mon[34138]: pgmap v1062: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-10T22:24:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:23 smithi090 ceph-mon[30350]: pgmap v1062: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-10T22:24:24.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:23 smithi154 ceph-mon[34495]: pgmap v1062: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-10T22:24:24.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.83 MB/sec execute 315 sec latency 11776.147 ms 2024-09-10T22:24:25.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.82 MB/sec execute 316 sec latency 12776.217 ms 2024-09-10T22:24:26.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:25 smithi171 ceph-mon[34138]: pgmap v1063: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-10T22:24:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:25 smithi090 ceph-mon[30350]: pgmap v1063: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-10T22:24:26.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:25 smithi154 ceph-mon[34495]: pgmap v1063: 129 pgs: 129 active+clean; 920 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-09-10T22:24:26.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.80 MB/sec execute 317 sec latency 13776.342 ms 2024-09-10T22:24:27.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.79 MB/sec execute 318 sec latency 14776.489 ms 2024-09-10T22:24:28.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:27 smithi171 ceph-mon[34138]: pgmap v1064: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 273 KiB/s wr, 39 op/s 2024-09-10T22:24:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:27 smithi090 ceph-mon[30350]: pgmap v1064: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 273 KiB/s wr, 39 op/s 2024-09-10T22:24:28.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:27 smithi154 ceph-mon[34495]: pgmap v1064: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 273 KiB/s wr, 39 op/s 2024-09-10T22:24:28.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.77 MB/sec execute 319 sec latency 15776.633 ms 2024-09-10T22:24:29.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.76 MB/sec execute 320 sec latency 16776.745 ms 2024-09-10T22:24:30.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:29 smithi171 ceph-mon[34138]: pgmap v1065: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 79 KiB/s wr, 18 op/s 2024-09-10T22:24:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:29 smithi090 ceph-mon[30350]: pgmap v1065: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 79 KiB/s wr, 18 op/s 2024-09-10T22:24:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:29 smithi154 ceph-mon[34495]: pgmap v1065: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 79 KiB/s wr, 18 op/s 2024-09-10T22:24:30.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.74 MB/sec execute 321 sec latency 17776.853 ms 2024-09-10T22:24:31.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.73 MB/sec execute 322 sec latency 18776.989 ms 2024-09-10T22:24:32.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:31 smithi171 ceph-mon[34138]: pgmap v1066: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 21 op/s 2024-09-10T22:24:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:31 smithi090 ceph-mon[30350]: pgmap v1066: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 21 op/s 2024-09-10T22:24:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:31 smithi154 ceph-mon[34495]: pgmap v1066: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 114 KiB/s wr, 21 op/s 2024-09-10T22:24:32.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.71 MB/sec execute 323 sec latency 19777.144 ms 2024-09-10T22:24:33.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:32 smithi171 ceph-mon[34138]: pgmap v1067: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 14 op/s 2024-09-10T22:24:33.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:32 smithi090 ceph-mon[30350]: pgmap v1067: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 14 op/s 2024-09-10T22:24:33.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:32 smithi154 ceph-mon[34495]: pgmap v1067: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 14 op/s 2024-09-10T22:24:33.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.70 MB/sec execute 324 sec latency 20777.326 ms 2024-09-10T22:24:34.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.68 MB/sec execute 325 sec latency 21777.438 ms 2024-09-10T22:24:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:35 smithi090 ceph-mon[30350]: pgmap v1068: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-09-10T22:24:35.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:35 smithi154 ceph-mon[34495]: pgmap v1068: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-09-10T22:24:35.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:35 smithi171 ceph-mon[34138]: pgmap v1068: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 53 KiB/s wr, 12 op/s 2024-09-10T22:24:35.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.67 MB/sec execute 326 sec latency 22777.546 ms 2024-09-10T22:24:36.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.65 MB/sec execute 327 sec latency 23777.668 ms 2024-09-10T22:24:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:37 smithi090 ceph-mon[30350]: pgmap v1069: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 16 op/s 2024-09-10T22:24:37.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:37 smithi154 ceph-mon[34495]: pgmap v1069: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 16 op/s 2024-09-10T22:24:37.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:37 smithi171 ceph-mon[34138]: pgmap v1069: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 64 KiB/s wr, 16 op/s 2024-09-10T22:24:37.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.64 MB/sec execute 328 sec latency 24777.774 ms 2024-09-10T22:24:38.355 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:24:38.738 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:24:38.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.63 MB/sec execute 329 sec latency 25777.840 ms 2024-09-10T22:24:39.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:39 smithi090 ceph-mon[30350]: pgmap v1070: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 10 op/s 2024-09-10T22:24:39.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:39 smithi154 ceph-mon[34495]: pgmap v1070: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 10 op/s 2024-09-10T22:24:39.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.357+0000 7f8c49400640 1 -- 172.21.15.90:0/2290089936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c44104df0 msgr2=0x7f8c441051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.357+0000 7f8c49400640 1 --2- 172.21.15.90:0/2290089936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c44104df0 0x7f8c441051d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009900 tx=0x7f8c3402a410 comp rx=0 tx=0).stop 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.357+0000 7f8c49400640 1 -- 172.21.15.90:0/2290089936 shutdown_connections 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.357+0000 7f8c49400640 1 --2- 172.21.15.90:0/2290089936 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c4410b180 0x7f8c44111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.357+0000 7f8c49400640 1 --2- 172.21.15.90:0/2290089936 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c4410a800 0x7f8c4410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.357+0000 7f8c49400640 1 --2- 172.21.15.90:0/2290089936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c44104df0 0x7f8c441051d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.357+0000 7f8c49400640 1 -- 172.21.15.90:0/2290089936 >> 172.21.15.90:0/2290089936 conn(0x7f8c44100840 msgr2=0x7f8c44102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.357+0000 7f8c49400640 1 -- 172.21.15.90:0/2290089936 shutdown_connections 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.357+0000 7f8c49400640 1 -- 172.21.15.90:0/2290089936 wait complete. 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.358+0000 7f8c49400640 1 Processor -- start 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.358+0000 7f8c49400640 1 -- start start 2024-09-10T22:24:39.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c49400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c44104df0 0x7f8c4419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c49400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c4410a800 0x7f8c4419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c49400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4410b180 0x7f8c441a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c49400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8c44114cf0 con 0x7f8c4410b180 2024-09-10T22:24:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c49400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8c44114b70 con 0x7f8c44104df0 2024-09-10T22:24:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c49400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8c44114e70 con 0x7f8c4410a800 2024-09-10T22:24:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c48a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4410b180 0x7f8c441a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c48a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4410b180 0x7f8c441a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54288/0 (socket says 172.21.15.90:54288) 2024-09-10T22:24:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c48a00640 1 -- 172.21.15.90:0/3463468478 learned_addr learned my addr 172.21.15.90:0/3463468478 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:39.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c43400640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c4410a800 0x7f8c4419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c43e00640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c44104df0 0x7f8c4419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c48a00640 1 -- 172.21.15.90:0/3463468478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c4410a800 msgr2=0x7f8c4419b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c48a00640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c4410a800 0x7f8c4419b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c48a00640 1 -- 172.21.15.90:0/3463468478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c44104df0 msgr2=0x7f8c4419b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c48a00640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c44104df0 0x7f8c4419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.359+0000 7f8c48a00640 1 -- 172.21.15.90:0/3463468478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c441a1230 con 0x7f8c4410b180 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.360+0000 7f8c48a00640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4410b180 0x7f8c441a0a60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00eaa0 tx=0x7f8c3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.360+0000 7f8c40c00640 1 -- 172.21.15.90:0/3463468478 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00cdb0 con 0x7f8c4410b180 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.360+0000 7f8c40c00640 1 -- 172.21.15.90:0/3463468478 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8c3c005020 con 0x7f8c4410b180 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.360+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c441a1520 con 0x7f8c4410b180 2024-09-10T22:24:39.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.360+0000 7f8c40c00640 1 -- 172.21.15.90:0/3463468478 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0116d0 con 0x7f8c4410b180 2024-09-10T22:24:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.360+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c441a7c60 con 0x7f8c4410b180 2024-09-10T22:24:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.361+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c441051d0 con 0x7f8c4410b180 2024-09-10T22:24:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.363+0000 7f8c40c00640 1 -- 172.21.15.90:0/3463468478 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8c3c00f040 con 0x7f8c4410b180 2024-09-10T22:24:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.365+0000 7f8c40c00640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c280b5500 0x7f8c280b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.365+0000 7f8c40c00640 1 -- 172.21.15.90:0/3463468478 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9490+0+0 (secure 0 0 0) 0x7f8c3c0e63b0 con 0x7f8c4410b180 2024-09-10T22:24:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.365+0000 7f8c43e00640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c280b5500 0x7f8c280b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.366+0000 7f8c43e00640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c280b5500 0x7f8c280b79c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8c34006010 tx=0x7f8c34000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:39.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.369+0000 7f8c40c00640 1 -- 172.21.15.90:0/3463468478 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8c3c016030 con 0x7f8c4410b180 2024-09-10T22:24:39.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:39 smithi171 ceph-mon[34138]: pgmap v1070: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 47 KiB/s wr, 10 op/s 2024-09-10T22:24:39.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.593+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c4402e5c0 con 0x7f8c4410b180 2024-09-10T22:24:39.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.593+0000 7f8c40c00640 1 -- 172.21.15.90:0/3463468478 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+12012 (secure 0 0 0) 0x7f8c3c0acb00 con 0x7f8c4410b180 2024-09-10T22:24:39.595 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:24:39.596 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":76,"btime":"2024-09-10T22:23:42:837927+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:23:42.763306+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:24:39.596 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 76 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c280b5500 msgr2=0x7f8c280b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c280b5500 0x7f8c280b79c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8c34006010 tx=0x7f8c34000bc0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4410b180 msgr2=0x7f8c441a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4410b180 0x7f8c441a0a60 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00eaa0 tx=0x7f8c3c00ef60 comp rx=0 tx=0).stop 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 shutdown_connections 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f8c280b5500 0x7f8c280b79c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c4410b180 0x7f8c441a0a60 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f8c4410a800 0x7f8c4419b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 --2- 172.21.15.90:0/3463468478 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f8c44104df0 0x7f8c4419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 >> 172.21.15.90:0/3463468478 conn(0x7f8c44100840 msgr2=0x7f8c44102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:39.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 shutdown_connections 2024-09-10T22:24:39.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:39.596+0000 7f8c49400640 1 -- 172.21.15.90:0/3463468478 wait complete. 2024-09-10T22:24:39.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.61 MB/sec execute 330 sec latency 26777.944 ms 2024-09-10T22:24:40.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:40 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3463468478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:24:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:40 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3463468478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:24:40.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:40 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3463468478' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:24:40.512 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 1af74f6a --release --await' 2024-09-10T22:24:40.512 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 1af74f6a --release --await 2024-09-10T22:24:40.884 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:24:40.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38356 4.60 MB/sec execute 331 sec latency 27778.039 ms 2024-09-10T22:24:41.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:41 smithi090 ceph-mon[30350]: pgmap v1071: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-09-10T22:24:41.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:41 smithi154 ceph-mon[34495]: pgmap v1071: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-09-10T22:24:41.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:41 smithi171 ceph-mon[34138]: pgmap v1071: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-09-10T22:24:41.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.619+0000 7ff904000640 1 -- 172.21.15.90:0/3780864587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc075540 msgr2=0x7ff8fc075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:41.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.619+0000 7ff904000640 1 --2- 172.21.15.90:0/3780864587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc075540 0x7ff8fc075920 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f002a410 comp rx=0 tx=0).stop 2024-09-10T22:24:41.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.619+0000 7ff904000640 1 -- 172.21.15.90:0/3780864587 shutdown_connections 2024-09-10T22:24:41.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.619+0000 7ff904000640 1 --2- 172.21.15.90:0/3780864587 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc10f3c0 0x7ff8fc1117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.619+0000 7ff904000640 1 --2- 172.21.15.90:0/3780864587 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8fc075f40 0x7ff8fc10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.619+0000 7ff904000640 1 --2- 172.21.15.90:0/3780864587 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc075540 0x7ff8fc075920 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.619+0000 7ff904000640 1 -- 172.21.15.90:0/3780864587 >> 172.21.15.90:0/3780864587 conn(0x7ff8fc0fe850 msgr2=0x7ff8fc100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:41.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.619+0000 7ff904000640 1 -- 172.21.15.90:0/3780864587 shutdown_connections 2024-09-10T22:24:41.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.619+0000 7ff904000640 1 -- 172.21.15.90:0/3780864587 wait complete. 2024-09-10T22:24:41.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.620+0000 7ff904000640 1 Processor -- start 2024-09-10T22:24:41.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.620+0000 7ff904000640 1 -- start start 2024-09-10T22:24:41.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.620+0000 7ff904000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc075540 0x7ff8fc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.620+0000 7ff904000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8fc075f40 0x7ff8fc19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.620+0000 7ff904000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc10f3c0 0x7ff8fc1a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.620+0000 7ff904000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc114e10 con 0x7ff8fc10f3c0 2024-09-10T22:24:41.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.620+0000 7ff904000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc114c90 con 0x7ff8fc075540 2024-09-10T22:24:41.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.620+0000 7ff904000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc114f90 con 0x7ff8fc075f40 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff902000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc10f3c0 0x7ff8fc1a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff901600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc075540 0x7ff8fc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff902000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc10f3c0 0x7ff8fc1a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54296/0 (socket says 172.21.15.90:54296) 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff901600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc075540 0x7ff8fc19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:41490/0 (socket says 172.21.15.90:41490) 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff902000640 1 -- 172.21.15.90:0/4036996563 learned_addr learned my addr 172.21.15.90:0/4036996563 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff900c00640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8fc075f40 0x7ff8fc19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff902000640 1 -- 172.21.15.90:0/4036996563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8fc075f40 msgr2=0x7ff8fc19b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff902000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8fc075f40 0x7ff8fc19b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff902000640 1 -- 172.21.15.90:0/4036996563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc075540 msgr2=0x7ff8fc19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff902000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc075540 0x7ff8fc19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.621+0000 7ff902000640 1 -- 172.21.15.90:0/4036996563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc1a14f0 con 0x7ff8fc10f3c0 2024-09-10T22:24:41.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.622+0000 7ff902000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc10f3c0 0x7ff8fc1a0d20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e9d0 tx=0x7ff8f400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:41.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.622+0000 7ff8fa000640 1 -- 172.21.15.90:0/4036996563 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400cd80 con 0x7ff8fc10f3c0 2024-09-10T22:24:41.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.622+0000 7ff8fa000640 1 -- 172.21.15.90:0/4036996563 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8f4005020 con 0x7ff8fc10f3c0 2024-09-10T22:24:41.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.622+0000 7ff8fa000640 1 -- 172.21.15.90:0/4036996563 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f40116d0 con 0x7ff8fc10f3c0 2024-09-10T22:24:41.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.622+0000 7ff904000640 1 -- 172.21.15.90:0/4036996563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc1a17e0 con 0x7ff8fc10f3c0 2024-09-10T22:24:41.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.622+0000 7ff904000640 1 -- 172.21.15.90:0/4036996563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1a7ea0 con 0x7ff8fc10f3c0 2024-09-10T22:24:41.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.625+0000 7ff906a95740 1 Processor -- start 2024-09-10T22:24:41.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.627+0000 7ff8fa000640 1 -- 172.21.15.90:0/4036996563 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff8f400f040 con 0x7ff8fc10f3c0 2024-09-10T22:24:41.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff906a95740 1 -- start start 2024-09-10T22:24:41.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff906a95740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x56394971f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff906a95740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x56394972b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff906a95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 0x56394972dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff906a95740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563949696290 con 0x56394971eff0 2024-09-10T22:24:41.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff906a95740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5639496856c0 con 0x56394971f910 2024-09-10T22:24:41.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff906a95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x5639496894d0 con 0x56394972b8a0 2024-09-10T22:24:41.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff900c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x56394971f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff900c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x56394971f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54302/0 (socket says 172.21.15.90:54302) 2024-09-10T22:24:41.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.628+0000 7ff900c00640 1 -- 172.21.15.90:0/1993634230 learned_addr learned my addr 172.21.15.90:0/1993634230 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.629+0000 7ff901600640 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x56394972b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.629+0000 7ff900c00640 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 0x56394972dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.629+0000 7ff900c00640 1 -- 172.21.15.90:0/1993634230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 msgr2=0x56394972dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.629+0000 7ff900c00640 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 0x56394972dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.629+0000 7ff900c00640 1 -- 172.21.15.90:0/1993634230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 msgr2=0x56394972b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.629+0000 7ff900c00640 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x56394972b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.629+0000 7ff900c00640 1 -- 172.21.15.90:0/1993634230 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563949612460 con 0x56394971eff0 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.629+0000 7ff901600640 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x56394972b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.629+0000 7ff900c00640 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x56394971f3d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80135d0 tx=0x7ff8e8009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143bb552e70c3966 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.630+0000 7ff8ee600640 1 -- 172.21.15.90:0/1993634230 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8004050 con 0x56394971eff0 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.630+0000 7ff8ee600640 1 -- 172.21.15.90:0/1993634230 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8e80035f0 con 0x56394971eff0 2024-09-10T22:24:41.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.630+0000 7ff8ee600640 1 -- 172.21.15.90:0/1993634230 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e800f780 con 0x56394971eff0 2024-09-10T22:24:41.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.630+0000 7ff8fa000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff8e40b54d0 0x7ff8e40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.630+0000 7ff8fa000640 1 -- 172.21.15.90:0/4036996563 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9490+0+0 (secure 0 0 0) 0x7ff8f40e6490 con 0x7ff8fc10f3c0 2024-09-10T22:24:41.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.630+0000 7ff901600640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff8e40b54d0 0x7ff8e40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 -- 172.21.15.90:0/1993634230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 msgr2=0x56394971f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x56394971f3d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80135d0 tx=0x7ff8e8009850 comp rx=0 tx=0).stop 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 -- 172.21.15.90:0/1993634230 shutdown_connections 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 0x56394972dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x56394972b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 --2- 172.21.15.90:0/1993634230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x56394971f3d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 -- 172.21.15.90:0/1993634230 >> 172.21.15.90:0/1993634230 conn(0x56394971ce20 msgr2=0x56394971dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 -- 172.21.15.90:0/1993634230 shutdown_connections 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 -- 172.21.15.90:0/1993634230 wait complete. 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff901600640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff8e40b54d0 0x7ff8e40b7990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00057c0 tx=0x7ff8f00058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.631+0000 7ff906a95740 1 -- start start 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.632+0000 7ff906a95740 1 Processor -- start 2024-09-10T22:24:41.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.632+0000 7ff906a95740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x563949728830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.632+0000 7ff906a95740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x563949728d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.632+0000 7ff906a95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 0x5639497292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.632+0000 7ff906a95740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5639494e6bb0 con 0x56394971eff0 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.632+0000 7ff906a95740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5639496206c0 con 0x56394971f910 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.632+0000 7ff906a95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x56394972e550 con 0x56394972b8a0 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.632+0000 7ff906a95740 10 client.0 _ll_register_callbacks cb 0x56394963b490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.632+0000 7ff901600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x563949728830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.633+0000 7ff901600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x563949728830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54306/0 (socket says 172.21.15.90:54306) 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.633+0000 7ff901600640 1 -- 172.21.15.90:0/353400755 learned_addr learned my addr 172.21.15.90:0/353400755 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.633+0000 7ff900c00640 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x563949728d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.633+0000 7ff902000640 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 0x5639497292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.633+0000 7ff901600640 1 -- 172.21.15.90:0/353400755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 msgr2=0x5639497292b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.633+0000 7ff901600640 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 0x5639497292b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.633+0000 7ff901600640 1 -- 172.21.15.90:0/353400755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 msgr2=0x563949728d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:41.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.633+0000 7ff901600640 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x563949728d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.633+0000 7ff901600640 1 -- 172.21.15.90:0/353400755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639493e3b00 con 0x56394971eff0 2024-09-10T22:24:41.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.634+0000 7ff901600640 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x563949728830 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b31d634bd3d808 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:41.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.634+0000 7ff8e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:24:41.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.634+0000 7ff8e2a00640 1 -- 172.21.15.90:0/353400755 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0035070 con 0x56394971eff0 2024-09-10T22:24:41.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.634+0000 7ff906a95740 1 -- 172.21.15.90:0/353400755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e80035f0 con 0x56394971eff0 2024-09-10T22:24:41.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.634+0000 7ff8e2a00640 1 -- 172.21.15.90:0/353400755 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9490+0+0 (secure 0 0 0) 0x7ff8f0004240 con 0x56394971eff0 2024-09-10T22:24:41.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.634+0000 7ff8e2a00640 1 -- 172.21.15.90:0/353400755 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff8f0004440 con 0x56394971eff0 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.634+0000 7ff906a95740 10 client.16443 fetch_fsmap learned FSMap version 76 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.634+0000 7ff906a95740 1 -- 172.21.15.90:0/353400755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56394945a960 con 0x56394971eff0 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.635+0000 7ff8e2a00640 1 -- 172.21.15.90:0/353400755 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8e80035f0 con 0x56394971eff0 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.635+0000 7ff8e2a00640 1 -- 172.21.15.90:0/353400755 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 76) ==== 8430+0+0 (secure 0 0 0) 0x7ff8f0043070 con 0x56394971eff0 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.635+0000 7ff906a95740 10 client.16443 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.635+0000 7ff906a95740 10 client.16443 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.635+0000 7ff906a95740 20 client.16443 populate_metadata read hostname 'smithi090' 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.635+0000 7ff906a95740 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56394973b3e0 0x56394973d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.635+0000 7ff906a95740 4 client.16443 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.635+0000 7ff906a95740 1 -- 172.21.15.90:0/353400755 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639496981e0 con 0x56394973b3e0 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.635+0000 7ff900c00640 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56394973b3e0 0x56394973d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.636+0000 7ff900c00640 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56394973b3e0 0x56394973d870 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d71e943ad4d54a4 server_cookie=ca1d0676c6b5e18f in_seq=0 out_seq=0 2024-09-10T22:24:41.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.636+0000 7ff8e2a00640 10 client.16443 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:41.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff8e2a00640 1 -- 172.21.15.90:0/353400755 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5639496981e0 con 0x56394973b3e0 2024-09-10T22:24:41.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff8e2a00640 10 client.16443 handle_command_reply: tid=0 2024-09-10T22:24:41.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff906a95740 1 client.16443 shutdown 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff906a95740 1 -- 172.21.15.90:0/353400755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 msgr2=0x563949728830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff906a95740 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x563949728830 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f0009260 comp rx=0 tx=0).stop 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff906a95740 1 -- 172.21.15.90:0/353400755 shutdown_connections 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff906a95740 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x56394973b3e0 0x56394973d870 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff906a95740 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394972b8a0 0x5639497292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff906a95740 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x563949728d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff906a95740 1 --2- 172.21.15.90:0/353400755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x563949728830 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff906a95740 1 -- 172.21.15.90:0/353400755 >> 172.21.15.90:0/353400755 conn(0x56394971ce20 msgr2=0x563949722990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.637+0000 7ff8e2a00640 0 client.16443 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.638+0000 7ff906a95740 1 -- 172.21.15.90:0/353400755 shutdown_connections 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.638+0000 7ff906a95740 1 -- 172.21.15.90:0/353400755 wait complete. 2024-09-10T22:24:41.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.638+0000 7ff906a95740 20 client.16443 trim_cache size 0 max 16384 2024-09-10T22:24:41.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.655+0000 7ff906a95740 1 Processor -- start 2024-09-10T22:24:41.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.655+0000 7ff906a95740 1 -- start start 2024-09-10T22:24:41.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff906a95740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x5639497230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff906a95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971f910 0x56394971bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff906a95740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394972b8a0 0x563949725a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff906a95740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b20f0 con 0x56394971eff0 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff906a95740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b3e40 con 0x56394972b8a0 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff906a95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b2850 con 0x56394971f910 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff901600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x5639497230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff901600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x5639497230f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54316/0 (socket says 172.21.15.90:54316) 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff901600640 1 -- 172.21.15.90:0/2179160682 learned_addr learned my addr 172.21.15.90:0/2179160682 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.656+0000 7ff900c00640 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971f910 0x56394971bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff902000640 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394972b8a0 0x563949725a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff901600640 1 -- 172.21.15.90:0/2179160682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971f910 msgr2=0x56394971bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff901600640 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971f910 0x56394971bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff901600640 1 -- 172.21.15.90:0/2179160682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394972b8a0 msgr2=0x563949725a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff901600640 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394972b8a0 0x563949725a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff901600640 1 -- 172.21.15.90:0/2179160682 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56394971e9d0 con 0x56394971eff0 2024-09-10T22:24:41.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff902000640 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394972b8a0 0x563949725a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:24:41.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff901600640 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x5639497230f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002a930 tx=0x7ff8f0009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dfbfd4ffdb2b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff8edc00640 1 -- 172.21.15.90:0/2179160682 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f003ca40 con 0x56394971eff0 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff8edc00640 1 -- 172.21.15.90:0/2179160682 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8f0031c50 con 0x56394971eff0 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.657+0000 7ff8edc00640 1 -- 172.21.15.90:0/2179160682 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f004f430 con 0x56394971eff0 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.658+0000 7ff906a95740 1 -- 172.21.15.90:0/2179160682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 msgr2=0x5639497230f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.658+0000 7ff906a95740 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x5639497230f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff8f002a930 tx=0x7ff8f0009520 comp rx=0 tx=0).stop 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.658+0000 7ff906a95740 1 -- 172.21.15.90:0/2179160682 shutdown_connections 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.658+0000 7ff906a95740 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394972b8a0 0x563949725a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.658+0000 7ff906a95740 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971f910 0x56394971bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.658+0000 7ff906a95740 1 --2- 172.21.15.90:0/2179160682 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394971eff0 0x5639497230f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.658+0000 7ff906a95740 1 -- 172.21.15.90:0/2179160682 >> 172.21.15.90:0/2179160682 conn(0x56394971ce20 msgr2=0x56394972a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.658+0000 7ff906a95740 1 -- 172.21.15.90:0/2179160682 shutdown_connections 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.658+0000 7ff906a95740 1 -- 172.21.15.90:0/2179160682 wait complete. 2024-09-10T22:24:41.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.659+0000 7ff906a95740 1 -- start start 2024-09-10T22:24:41.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.659+0000 7ff906a95740 1 Processor -- start 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff906a95740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971eff0 0x5639497307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff906a95740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x563949730cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff906a95740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394972b8a0 0x563949731230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff906a95740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b3770 con 0x56394972b8a0 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff906a95740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b2fe0 con 0x56394971f910 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff906a95740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b00f0 con 0x56394971eff0 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff906a95740 10 client.0 _ll_register_callbacks cb 0x7ff8f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff902000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394972b8a0 0x563949731230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff902000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394972b8a0 0x563949731230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54326/0 (socket says 172.21.15.90:54326) 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff902000640 1 -- 172.21.15.90:0/1414942267 learned_addr learned my addr 172.21.15.90:0/1414942267 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:41.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff901600640 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971eff0 0x5639497307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.660+0000 7ff900c00640 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x563949730cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff902000640 1 -- 172.21.15.90:0/1414942267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971eff0 msgr2=0x5639497307b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff902000640 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971eff0 0x5639497307b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff902000640 1 -- 172.21.15.90:0/1414942267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 msgr2=0x563949730cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff902000640 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x563949730cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff902000640 1 -- 172.21.15.90:0/1414942267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639497a8400 con 0x56394972b8a0 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff901600640 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971eff0 0x5639497307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff902000640 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394972b8a0 0x563949731230 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40e3810 tx=0x7ff8f40abd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5829233188e7e1d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff8e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff8e2a00640 1 -- 172.21.15.90:0/1414942267 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f401f040 con 0x56394972b8a0 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff8e2a00640 1 -- 172.21.15.90:0/1414942267 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9490+0+0 (secure 0 0 0) 0x7ff8f40b55e0 con 0x56394972b8a0 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff906a95740 1 -- 172.21.15.90:0/1414942267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8f0031c50 con 0x56394972b8a0 2024-09-10T22:24:41.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff8e2a00640 1 -- 172.21.15.90:0/1414942267 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff8f40ac590 con 0x56394972b8a0 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.661+0000 7ff906a95740 10 client.16449 fetch_fsmap learned FSMap version 76 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.662+0000 7ff906a95740 1 -- 172.21.15.90:0/1414942267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5639497a9760 con 0x56394972b8a0 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.662+0000 7ff8e2a00640 1 -- 172.21.15.90:0/1414942267 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8f0031c50 con 0x56394972b8a0 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.662+0000 7ff8e2a00640 1 -- 172.21.15.90:0/1414942267 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 76) ==== 8430+0+0 (secure 0 0 0) 0x7ff8f40ab7f0 con 0x56394972b8a0 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.662+0000 7ff906a95740 10 client.16449 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.662+0000 7ff906a95740 10 client.16449 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.662+0000 7ff906a95740 20 client.16449 populate_metadata read hostname 'smithi090' 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.662+0000 7ff906a95740 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639497a9eb0 0x5639497ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.662+0000 7ff906a95740 4 client.16449 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1af74f6a", "release": true, "await": true}] 2024-09-10T22:24:41.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.662+0000 7ff906a95740 1 -- 172.21.15.90:0/1414942267 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1af74f6a", "release": true, "await": true}) -- 0x563949693c90 con 0x5639497a9eb0 2024-09-10T22:24:41.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.663+0000 7ff901600640 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639497a9eb0 0x5639497ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:41.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.663+0000 7ff901600640 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639497a9eb0 0x5639497ac2d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e89f080c9f3e60b server_cookie=5df5c3f76c53b305 in_seq=0 out_seq=0 2024-09-10T22:24:41.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.663+0000 7ff8e2a00640 10 client.16449 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:41.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.707+0000 7ff8e2a00640 1 -- 172.21.15.90:0/1414942267 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563949693c90 con 0x5639497a9eb0 2024-09-10T22:24:41.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.707+0000 7ff8e2a00640 10 client.16449 handle_command_reply: tid=0 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.707+0000 7ff906a95740 1 client.16449 shutdown 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.707+0000 7ff906a95740 1 -- 172.21.15.90:0/1414942267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394972b8a0 msgr2=0x563949731230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.707+0000 7ff906a95740 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394972b8a0 0x563949731230 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40e3810 tx=0x7ff8f40abd00 comp rx=0 tx=0).stop 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff906a95740 1 -- 172.21.15.90:0/1414942267 shutdown_connections 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff906a95740 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x5639497a9eb0 0x5639497ac2d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff906a95740 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56394972b8a0 0x563949731230 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff906a95740 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x56394971f910 0x563949730cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff8e2a00640 0 client.16449 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff906a95740 1 --2- 172.21.15.90:0/1414942267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x56394971eff0 0x5639497307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff906a95740 1 -- 172.21.15.90:0/1414942267 >> 172.21.15.90:0/1414942267 conn(0x56394971ce20 msgr2=0x563949736770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff906a95740 1 -- 172.21.15.90:0/1414942267 shutdown_connections 2024-09-10T22:24:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff906a95740 1 -- 172.21.15.90:0/1414942267 wait complete. 2024-09-10T22:24:41.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.708+0000 7ff906a95740 20 client.16449 trim_cache size 0 max 16384 2024-09-10T22:24:41.711 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 76, 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 64, 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "1af74f6a": { 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 64, 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:24:41.712 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:41.713 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:24:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.709+0000 7ff904000640 1 -- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff8e40b54d0 msgr2=0x7ff8e40b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.709+0000 7ff904000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff8e40b54d0 0x7ff8e40b7990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00057c0 tx=0x7ff8f00058d0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.709+0000 7ff904000640 1 -- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc10f3c0 msgr2=0x7ff8fc1a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.709+0000 7ff904000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc10f3c0 0x7ff8fc1a0d20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400e9d0 tx=0x7ff8f400ee90 comp rx=0 tx=0).stop 2024-09-10T22:24:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.710+0000 7ff904000640 1 -- 172.21.15.90:0/4036996563 shutdown_connections 2024-09-10T22:24:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.710+0000 7ff904000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff8e40b54d0 0x7ff8e40b7990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.710+0000 7ff904000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc10f3c0 0x7ff8fc1a0d20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.710+0000 7ff904000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff8fc075f40 0x7ff8fc19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.710+0000 7ff904000640 1 --2- 172.21.15.90:0/4036996563 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff8fc075540 0x7ff8fc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:41.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.710+0000 7ff904000640 1 -- 172.21.15.90:0/4036996563 >> 172.21.15.90:0/4036996563 conn(0x7ff8fc0fe850 msgr2=0x7ff8fc0ff4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:41.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.710+0000 7ff904000640 1 -- 172.21.15.90:0/4036996563 shutdown_connections 2024-09-10T22:24:41.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:41.710+0000 7ff904000640 1 -- 172.21.15.90:0/4036996563 wait complete. 2024-09-10T22:24:41.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38374 4.59 MB/sec execute 332 sec latency 28598.225 ms 2024-09-10T22:24:42.774 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1af74f6a', total seconds elapsed: 34.4 2024-09-10T22:24:42.774 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T22:24:42.774 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:24:42.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38488 4.59 MB/sec execute 333 sec latency 1069.451 ms 2024-09-10T22:24:43.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:43 smithi090 ceph-mon[30350]: pgmap v1072: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-10T22:24:43.411 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:24:43.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:43 smithi154 ceph-mon[34495]: pgmap v1072: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-10T22:24:43.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:43 smithi171 ceph-mon[34138]: pgmap v1072: 129 pgs: 129 active+clean; 921 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 10 op/s 2024-09-10T22:24:43.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38618 4.59 MB/sec execute 334 sec latency 82.740 ms 2024-09-10T22:24:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.456+0000 7f0817e00640 1 -- 172.21.15.90:0/13852675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818102220 msgr2=0x7f08181066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.456+0000 7f0817e00640 1 --2- 172.21.15.90:0/13852675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818102220 0x7f08181066f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0804009860 tx=0x7f080402a340 comp rx=0 tx=0).stop 2024-09-10T22:24:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.456+0000 7f0817e00640 1 -- 172.21.15.90:0/13852675 shutdown_connections 2024-09-10T22:24:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.456+0000 7f0817e00640 1 --2- 172.21.15.90:0/13852675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0818106d40 0x7f0818107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.456+0000 7f0817e00640 1 --2- 172.21.15.90:0/13852675 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818102220 0x7f08181066f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.456+0000 7f0817e00640 1 --2- 172.21.15.90:0/13852675 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0818101900 0x7f0818101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.456+0000 7f0817e00640 1 -- 172.21.15.90:0/13852675 >> 172.21.15.90:0/13852675 conn(0x7f08180fd5b0 msgr2=0x7f08180ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.457+0000 7f0817e00640 1 -- 172.21.15.90:0/13852675 shutdown_connections 2024-09-10T22:24:44.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.457+0000 7f0817e00640 1 -- 172.21.15.90:0/13852675 wait complete. 2024-09-10T22:24:44.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.458+0000 7f0817e00640 1 Processor -- start 2024-09-10T22:24:44.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.459+0000 7f0817e00640 1 -- start start 2024-09-10T22:24:44.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.459+0000 7f0817e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0818101900 0x7f0818074780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:44.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.459+0000 7f0817e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0818102220 0x7f0818074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:44.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.459+0000 7f0817e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818106d40 0x7f081806dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:44.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.459+0000 7f0817e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0818119060 con 0x7f0818101900 2024-09-10T22:24:44.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.459+0000 7f0817e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f0818118ee0 con 0x7f0818102220 2024-09-10T22:24:44.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.459+0000 7f0817e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f08181191e0 con 0x7f0818106d40 2024-09-10T22:24:44.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.459+0000 7f0817400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818106d40 0x7f081806dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:44.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0817400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818106d40 0x7f081806dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:47366/0 (socket says 172.21.15.90:47366) 2024-09-10T22:24:44.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0817400640 1 -- 172.21.15.90:0/2531048863 learned_addr learned my addr 172.21.15.90:0/2531048863 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:44.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0816a00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0818101900 0x7f0818074780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:44.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0817400640 1 -- 172.21.15.90:0/2531048863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0818102220 msgr2=0x7f0818074cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T22:24:44.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0816000640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0818102220 0x7f0818074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:44.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0817400640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0818102220 0x7f0818074cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:44.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0817400640 1 -- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0818101900 msgr2=0x7f0818074780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:44.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0817400640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0818101900 0x7f0818074780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:44.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0817400640 1 -- 172.21.15.90:0/2531048863 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f081806e3a0 con 0x7f0818106d40 2024-09-10T22:24:44.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.460+0000 7f0816a00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0818101900 0x7f0818074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:24:44.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.461+0000 7f0816000640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0818102220 0x7f0818074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:24:44.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.461+0000 7f0817400640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818106d40 0x7f081806dae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f080800e970 tx=0x7f080800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:44.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.462+0000 7f080f400640 1 -- 172.21.15.90:0/2531048863 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080800ccc0 con 0x7f0818106d40 2024-09-10T22:24:44.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.462+0000 7f080f400640 1 -- 172.21.15.90:0/2531048863 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f08080054b0 con 0x7f0818106d40 2024-09-10T22:24:44.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.462+0000 7f080f400640 1 -- 172.21.15.90:0/2531048863 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0808004df0 con 0x7f0818106d40 2024-09-10T22:24:44.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.462+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f081806e690 con 0x7f0818106d40 2024-09-10T22:24:44.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.462+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08180735c0 con 0x7f0818106d40 2024-09-10T22:24:44.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.464+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f081806b1b0 con 0x7f0818106d40 2024-09-10T22:24:44.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.467+0000 7f080f400640 1 -- 172.21.15.90:0/2531048863 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0808004ff0 con 0x7f0818106d40 2024-09-10T22:24:44.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.469+0000 7f080f400640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07fc0b5580 0x7f07fc0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:44.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.469+0000 7f080f400640 1 -- 172.21.15.90:0/2531048863 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9490+0+0 (secure 0 0 0) 0x7f08080e6770 con 0x7f0818106d40 2024-09-10T22:24:44.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.469+0000 7f0816a00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07fc0b5580 0x7f07fc0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:44.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.470+0000 7f0816a00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07fc0b5580 0x7f07fc0b7a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0810005fd0 tx=0x7f0810009500 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:44.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.472+0000 7f080f400640 1 -- 172.21.15.90:0/2531048863 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f08080acf20 con 0x7f0818106d40 2024-09-10T22:24:44.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.881+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0818073880 con 0x7f0818106d40 2024-09-10T22:24:44.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.881+0000 7f080f400640 1 -- 172.21.15.90:0/2531048863 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+12012 (secure 0 0 0) 0x7f08080ac820 con 0x7f0818106d40 2024-09-10T22:24:44.883 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:24:44.884 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":76,"btime":"2024-09-10T22:23:42:837927+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:23:42.763306+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:24:44.884 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 76 2024-09-10T22:24:44.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07fc0b5580 msgr2=0x7f07fc0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07fc0b5580 0x7f07fc0b7a40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0810005fd0 tx=0x7f0810009500 comp rx=0 tx=0).stop 2024-09-10T22:24:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818106d40 msgr2=0x7f081806dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818106d40 0x7f081806dae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f080800e970 tx=0x7f080800ee30 comp rx=0 tx=0).stop 2024-09-10T22:24:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 shutdown_connections 2024-09-10T22:24:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f07fc0b5580 0x7f07fc0b7a40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f0818106d40 0x7f081806dae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f0818102220 0x7f0818074cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 --2- 172.21.15.90:0/2531048863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0818101900 0x7f0818074780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:44.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.885+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 >> 172.21.15.90:0/2531048863 conn(0x7f08180fd5b0 msgr2=0x7f0818104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:44.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.886+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 shutdown_connections 2024-09-10T22:24:44.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:44.886+0000 7f0817e00640 1 -- 172.21.15.90:0/2531048863 wait complete. 2024-09-10T22:24:44.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38760 4.60 MB/sec execute 335 sec latency 80.626 ms 2024-09-10T22:24:45.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:45 smithi090 ceph-mon[30350]: pgmap v1073: 129 pgs: 129 active+clean; 922 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 85 KiB/s wr, 13 op/s 2024-09-10T22:24:45.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:45 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2531048863' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:24:45.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:45 smithi154 ceph-mon[34495]: pgmap v1073: 129 pgs: 129 active+clean; 922 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 85 KiB/s wr, 13 op/s 2024-09-10T22:24:45.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:45 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2531048863' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:24:45.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:45 smithi171 ceph-mon[34138]: pgmap v1073: 129 pgs: 129 active+clean; 922 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 85 KiB/s wr, 13 op/s 2024-09-10T22:24:45.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:45 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2531048863' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:24:45.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38831 4.60 MB/sec execute 336 sec latency 456.723 ms 2024-09-10T22:24:46.759 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T22:24:46.759 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T22:24:46.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38916 4.60 MB/sec execute 337 sec latency 609.044 ms 2024-09-10T22:24:47.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:47 smithi154 ceph-mon[34495]: pgmap v1074: 129 pgs: 129 active+clean; 930 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 883 KiB/s wr, 69 op/s 2024-09-10T22:24:47.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:47 smithi171 ceph-mon[34138]: pgmap v1074: 129 pgs: 129 active+clean; 930 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 883 KiB/s wr, 69 op/s 2024-09-10T22:24:47.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:47 smithi090 ceph-mon[30350]: pgmap v1074: 129 pgs: 129 active+clean; 930 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 883 KiB/s wr, 69 op/s 2024-09-10T22:24:47.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39045 4.62 MB/sec execute 338 sec latency 82.453 ms 2024-09-10T22:24:48.247 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:24:48.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39161 4.62 MB/sec execute 339 sec latency 81.095 ms 2024-09-10T22:24:49.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:49 smithi090 ceph-mon[30350]: pgmap v1075: 129 pgs: 129 active+clean; 932 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-09-10T22:24:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:49 smithi154 ceph-mon[34495]: pgmap v1075: 129 pgs: 129 active+clean; 932 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-09-10T22:24:49.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:49 smithi171 ceph-mon[34138]: pgmap v1075: 129 pgs: 129 active+clean; 932 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-09-10T22:24:49.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39273 4.62 MB/sec execute 340 sec latency 260.859 ms 2024-09-10T22:24:50.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.153+0000 7fe621000640 1 -- 172.21.15.90:0/1750740057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c108fa0 msgr2=0x7fe61c10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:50.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.153+0000 7fe621000640 1 --2- 172.21.15.90:0/1750740057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c108fa0 0x7fe61c10f7a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe614009ca0 tx=0x7fe61402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.153+0000 7fe621000640 1 -- 172.21.15.90:0/1750740057 shutdown_connections 2024-09-10T22:24:50.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.153+0000 7fe621000640 1 --2- 172.21.15.90:0/1750740057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c108fa0 0x7fe61c10f7a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.153+0000 7fe621000640 1 --2- 172.21.15.90:0/1750740057 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe61c108620 0x7fe61c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.153+0000 7fe621000640 1 --2- 172.21.15.90:0/1750740057 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c102c10 0x7fe61c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.153+0000 7fe621000640 1 -- 172.21.15.90:0/1750740057 >> 172.21.15.90:0/1750740057 conn(0x7fe61c0fe6c0 msgr2=0x7fe61c100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:50.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.153+0000 7fe621000640 1 -- 172.21.15.90:0/1750740057 shutdown_connections 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.153+0000 7fe621000640 1 -- 172.21.15.90:0/1750740057 wait complete. 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.154+0000 7fe621000640 1 Processor -- start 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.154+0000 7fe621000640 1 -- start start 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.154+0000 7fe621000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c102c10 0x7fe61c19b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.154+0000 7fe621000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c108620 0x7fe61c19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.154+0000 7fe621000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe61c108fa0 0x7fe61c1a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe621000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c112b20 con 0x7fe61c102c10 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe621000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1129a0 con 0x7fe61c108620 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe621000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c112ca0 con 0x7fe61c108fa0 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c102c10 0x7fe61c19b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c102c10 0x7fe61c19b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50590/0 (socket says 172.21.15.90:50590) 2024-09-10T22:24:50.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61b400640 1 -- 172.21.15.90:0/2873387224 learned_addr learned my addr 172.21.15.90:0/2873387224 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61aa00640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c108620 0x7fe61c19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61be00640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe61c108fa0 0x7fe61c1a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61b400640 1 -- 172.21.15.90:0/2873387224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe61c108fa0 msgr2=0x7fe61c1a09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61b400640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe61c108fa0 0x7fe61c1a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61b400640 1 -- 172.21.15.90:0/2873387224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c108620 msgr2=0x7fe61c19b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61b400640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c108620 0x7fe61c19b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61b400640 1 -- 172.21.15.90:0/2873387224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c1a11a0 con 0x7fe61c102c10 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.155+0000 7fe61b400640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c102c10 0x7fe61c19b0f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00d950 tx=0x7fe60c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.156+0000 7fe613e00640 1 -- 172.21.15.90:0/2873387224 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c017070 con 0x7fe61c102c10 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.156+0000 7fe613e00640 1 -- 172.21.15.90:0/2873387224 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe60c009c30 con 0x7fe61c102c10 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.156+0000 7fe613e00640 1 -- 172.21.15.90:0/2873387224 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00f620 con 0x7fe61c102c10 2024-09-10T22:24:50.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.156+0000 7fe621000640 1 -- 172.21.15.90:0/2873387224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c1a1490 con 0x7fe61c102c10 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.156+0000 7fe621000640 1 -- 172.21.15.90:0/2873387224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c1a7b50 con 0x7fe61c102c10 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.159+0000 7fe6250cd740 1 Processor -- start 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.159+0000 7fe6250cd740 1 -- start start 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.159+0000 7fe613e00640 1 -- 172.21.15.90:0/2873387224 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe60c010460 con 0x7fe61c102c10 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4ba2f0 0x55706e4ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4bac10 0x55706e4c6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe6250cd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55706e42fca0 con 0x55706e4c6ba0 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe6250cd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55706e41f0d0 con 0x55706e4ba2f0 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe6250cd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55706e422ee0 con 0x55706e4bac10 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4ba2f0 0x55706e4ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4ba2f0 0x55706e4ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:38484/0 (socket says 172.21.15.90:38484) 2024-09-10T22:24:50.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61be00640 1 -- 172.21.15.90:0/4266824461 learned_addr learned my addr 172.21.15.90:0/4266824461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:50.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61b400640 1 --2- 172.21.15.90:0/4266824461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4bac10 0x55706e4c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61aa00640 1 --2- 172.21.15.90:0/4266824461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61be00640 1 -- 172.21.15.90:0/4266824461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4bac10 msgr2=0x55706e4c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:50.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61be00640 1 --2- 172.21.15.90:0/4266824461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4bac10 0x55706e4c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61be00640 1 -- 172.21.15.90:0/4266824461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 msgr2=0x55706e4c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:50.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61be00640 1 --2- 172.21.15.90:0/4266824461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.160+0000 7fe61be00640 1 -- 172.21.15.90:0/4266824461 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706e3d1400 con 0x55706e4ba2f0 2024-09-10T22:24:50.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.161+0000 7fe61be00640 1 --2- 172.21.15.90:0/4266824461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4ba2f0 0x55706e4ba6d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe61402ade0 tx=0x7fe614009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e286b8766b6d2d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:50.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.161+0000 7fe610c00640 1 -- 172.21.15.90:0/4266824461 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614031980 con 0x55706e4ba2f0 2024-09-10T22:24:50.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.161+0000 7fe610c00640 1 -- 172.21.15.90:0/4266824461 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe61403f070 con 0x55706e4ba2f0 2024-09-10T22:24:50.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.161+0000 7fe610c00640 1 -- 172.21.15.90:0/4266824461 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614004330 con 0x55706e4ba2f0 2024-09-10T22:24:50.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.161+0000 7fe613e00640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe6000b5580 0x7fe6000b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe613e00640 1 -- 172.21.15.90:0/2873387224 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9490+0+0 (secure 0 0 0) 0x7fe60c0e5fd0 con 0x7fe61c102c10 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe61b400640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe6000b5580 0x7fe6000b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe6250cd740 1 -- 172.21.15.90:0/4266824461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4ba2f0 msgr2=0x55706e4ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe6250cd740 1 --2- 172.21.15.90:0/4266824461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4ba2f0 0x55706e4ba6d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe61402ade0 tx=0x7fe614009a20 comp rx=0 tx=0).stop 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe6250cd740 1 -- 172.21.15.90:0/4266824461 shutdown_connections 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe6250cd740 1 --2- 172.21.15.90:0/4266824461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe6250cd740 1 --2- 172.21.15.90:0/4266824461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4bac10 0x55706e4c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe6250cd740 1 --2- 172.21.15.90:0/4266824461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4ba2f0 0x55706e4ba6d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe6250cd740 1 -- 172.21.15.90:0/4266824461 >> 172.21.15.90:0/4266824461 conn(0x55706e4b8120 msgr2=0x55706e4b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe6250cd740 1 -- 172.21.15.90:0/4266824461 shutdown_connections 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe6250cd740 1 -- 172.21.15.90:0/4266824461 wait complete. 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.162+0000 7fe61b400640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe6000b5580 0x7fe6000b7a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55706e4b9000 tx=0x7fe60c005880 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:50.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.163+0000 7fe6250cd740 1 -- start start 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.163+0000 7fe6250cd740 1 Processor -- start 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.163+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4c6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.163+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4c38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.163+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe6250cd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55706e2807a0 con 0x55706e4c6ba0 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe6250cd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55706e3c4a10 con 0x55706e4bac10 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe6250cd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55706e4c97c0 con 0x55706e4ba2f0 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe6250cd740 10 client.0 _ll_register_callbacks cb 0x55706e3d4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4c38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61aa00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4c38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:38486/0 (socket says 172.21.15.90:38486) 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61aa00640 1 -- 172.21.15.90:0/578691347 learned_addr learned my addr 172.21.15.90:0/578691347 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50606/0 (socket says 172.21.15.90:50606) 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61be00640 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61be00640 1 -- 172.21.15.90:0/578691347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 msgr2=0x55706e4c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61be00640 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61be00640 1 -- 172.21.15.90:0/578691347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 msgr2=0x55706e4c38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61be00640 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4c38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61be00640 1 -- 172.21.15.90:0/578691347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706e18cb90 con 0x55706e4c6ba0 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.164+0000 7fe61aa00640 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.165+0000 7fe61be00640 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c3e90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe614009620 tx=0x7fe614005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efdfcc66dc50652 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.165+0000 7fe605600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:24:50.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.165+0000 7fe605600640 1 -- 172.21.15.90:0/578691347 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614031d90 con 0x55706e4c6ba0 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.165+0000 7fe6250cd740 1 -- 172.21.15.90:0/578691347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe61403f070 con 0x55706e4c6ba0 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.165+0000 7fe605600640 1 -- 172.21.15.90:0/578691347 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9490+0+0 (secure 0 0 0) 0x7fe614003970 con 0x55706e4c6ba0 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.165+0000 7fe605600640 1 -- 172.21.15.90:0/578691347 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fe614003b70 con 0x55706e4c6ba0 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.165+0000 7fe6250cd740 10 client.16464 fetch_fsmap learned FSMap version 76 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.165+0000 7fe6250cd740 1 -- 172.21.15.90:0/578691347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55706e4d50c0 con 0x55706e4c6ba0 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.165+0000 7fe605600640 1 -- 172.21.15.90:0/578691347 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe61403f070 con 0x55706e4c6ba0 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.166+0000 7fe605600640 1 -- 172.21.15.90:0/578691347 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 76) ==== 8430+0+0 (secure 0 0 0) 0x7fe614044470 con 0x55706e4c6ba0 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.166+0000 7fe6250cd740 10 client.16464 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.166+0000 7fe6250cd740 10 client.16464 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.166+0000 7fe6250cd740 20 client.16464 populate_metadata read hostname 'smithi090' 2024-09-10T22:24:50.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.166+0000 7fe6250cd740 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55706e4d6410 0x55706e4d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.166+0000 7fe6250cd740 4 client.16464 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.166+0000 7fe6250cd740 1 -- 172.21.15.90:0/578691347 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55706e431bf0 con 0x55706e4d6410 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.166+0000 7fe61aa00640 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55706e4d6410 0x55706e4d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.167+0000 7fe61aa00640 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55706e4d6410 0x55706e4d8800 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a866df4d5589d5 server_cookie=b35867b394a1998a in_seq=0 out_seq=0 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.167+0000 7fe605600640 10 client.16464 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe605600640 1 -- 172.21.15.90:0/578691347 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55706e431bf0 con 0x55706e4d6410 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe605600640 10 client.16464 handle_command_reply: tid=0 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 client.16464 shutdown 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 -- 172.21.15.90:0/578691347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 msgr2=0x55706e4c3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c3e90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe614009620 tx=0x7fe614005c30 comp rx=0 tx=0).stop 2024-09-10T22:24:50.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 -- 172.21.15.90:0/578691347 shutdown_connections 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55706e4d6410 0x55706e4d8800 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4c3e90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4c38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe605600640 0 client.16464 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 --2- 172.21.15.90:0/578691347 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 -- 172.21.15.90:0/578691347 >> 172.21.15.90:0/578691347 conn(0x55706e4b8120 msgr2=0x55706e4bdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 -- 172.21.15.90:0/578691347 shutdown_connections 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 1 -- 172.21.15.90:0/578691347 wait complete. 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.168+0000 7fe6250cd740 20 client.16464 trim_cache size 0 max 16384 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.194+0000 7fe6250cd740 1 Processor -- start 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.194+0000 7fe6250cd740 1 -- start start 2024-09-10T22:24:50.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4bac10 0x55706e4d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4c6ba0 0x55706e4d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe6250cd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b1d40 con 0x55706e4bac10 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe6250cd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b3a90 con 0x55706e4c6ba0 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe6250cd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b24a0 con 0x55706e4ba2f0 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe61be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe61be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:52754/0 (socket says 172.21.15.90:52754) 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe61be00640 1 -- 172.21.15.90:0/1934807350 learned_addr learned my addr 172.21.15.90:0/1934807350 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.195+0000 7fe61be00640 1 --2- 172.21.15.90:0/1934807350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4c6ba0 0x55706e4d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.196+0000 7fe61be00640 1 -- 172.21.15.90:0/1934807350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4c6ba0 msgr2=0x55706e4d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.196+0000 7fe61be00640 1 --2- 172.21.15.90:0/1934807350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4c6ba0 0x55706e4d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.196+0000 7fe61be00640 1 -- 172.21.15.90:0/1934807350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4bac10 msgr2=0x55706e4d2160 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.196+0000 7fe61be00640 1 --2- 172.21.15.90:0/1934807350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4bac10 0x55706e4d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.196+0000 7fe61be00640 1 -- 172.21.15.90:0/1934807350 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706e4b8e00 con 0x55706e4ba2f0 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.196+0000 7fe61be00640 1 --2- 172.21.15.90:0/1934807350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4b7af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe61402c010 tx=0x7fe614009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e0f1837c9d7412 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.196+0000 7fe607e00640 1 -- 172.21.15.90:0/1934807350 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61405f070 con 0x55706e4ba2f0 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.196+0000 7fe607e00640 1 -- 172.21.15.90:0/1934807350 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe61403f040 con 0x55706e4ba2f0 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.196+0000 7fe607e00640 1 -- 172.21.15.90:0/1934807350 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614044a10 con 0x55706e4ba2f0 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.197+0000 7fe6250cd740 1 -- 172.21.15.90:0/1934807350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 msgr2=0x55706e4b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.197+0000 7fe6250cd740 1 --2- 172.21.15.90:0/1934807350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4b7af0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe61402c010 tx=0x7fe614009a20 comp rx=0 tx=0).stop 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.197+0000 7fe6250cd740 1 -- 172.21.15.90:0/1934807350 shutdown_connections 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.198+0000 7fe6250cd740 1 --2- 172.21.15.90:0/1934807350 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4c6ba0 0x55706e4d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.198+0000 7fe6250cd740 1 --2- 172.21.15.90:0/1934807350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4bac10 0x55706e4d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.198+0000 7fe6250cd740 1 --2- 172.21.15.90:0/1934807350 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4b7af0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.198+0000 7fe6250cd740 1 -- 172.21.15.90:0/1934807350 >> 172.21.15.90:0/1934807350 conn(0x55706e4b8120 msgr2=0x55706e4d1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.198+0000 7fe6250cd740 1 -- 172.21.15.90:0/1934807350 shutdown_connections 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.198+0000 7fe6250cd740 1 -- 172.21.15.90:0/1934807350 wait complete. 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.200+0000 7fe6250cd740 1 -- start start 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.200+0000 7fe6250cd740 1 Processor -- start 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4dbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe6250cd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4d8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe6250cd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b33c0 con 0x55706e4c6ba0 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe6250cd740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b2c30 con 0x55706e4bac10 2024-09-10T22:24:50.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe6250cd740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1afd40 con 0x55706e4ba2f0 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe6250cd740 10 client.0 _ll_register_callbacks cb 0x7fe608004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe61be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4dbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe61be00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4dbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:52760/0 (socket says 172.21.15.90:52760) 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.201+0000 7fe61be00640 1 -- 172.21.15.90:0/2846529669 learned_addr learned my addr 172.21.15.90:0/2846529669 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe61be00640 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4d8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe61aa00640 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe61be00640 1 -- 172.21.15.90:0/2846529669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 msgr2=0x55706e4d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe61be00640 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4d8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe61be00640 1 -- 172.21.15.90:0/2846529669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 msgr2=0x55706e4d8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:24:50.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe61be00640 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4d8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe61be00640 1 -- 172.21.15.90:0/2846529669 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706e4d92d0 con 0x55706e4ba2f0 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe61aa00640 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe61be00640 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4dbd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe61402ade0 tx=0x7fe614004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8400aef701bd4b58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.202+0000 7fe605600640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.203+0000 7fe605600640 1 -- 172.21.15.90:0/2846529669 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61405f040 con 0x55706e4ba2f0 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.203+0000 7fe605600640 1 -- 172.21.15.90:0/2846529669 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9490+0+0 (secure 0 0 0) 0x7fe614003970 con 0x55706e4ba2f0 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.203+0000 7fe6250cd740 1 -- 172.21.15.90:0/2846529669 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe61403f040 con 0x55706e4ba2f0 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.203+0000 7fe605600640 1 -- 172.21.15.90:0/2846529669 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fe614044470 con 0x55706e4ba2f0 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.203+0000 7fe6250cd740 10 client.26080 fetch_fsmap learned FSMap version 76 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.203+0000 7fe6250cd740 1 -- 172.21.15.90:0/2846529669 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55706e54c530 con 0x55706e4ba2f0 2024-09-10T22:24:50.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.204+0000 7fe605600640 1 -- 172.21.15.90:0/2846529669 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe61403f040 con 0x55706e4ba2f0 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.204+0000 7fe605600640 1 -- 172.21.15.90:0/2846529669 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 76) ==== 8430+0+0 (secure 0 0 0) 0x7fe614030a50 con 0x55706e4ba2f0 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.204+0000 7fe6250cd740 10 client.26080 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.204+0000 7fe6250cd740 10 client.26080 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.204+0000 7fe6250cd740 20 client.26080 populate_metadata read hostname 'smithi090' 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.204+0000 7fe6250cd740 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55706e54ca70 0x55706e54ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.204+0000 7fe6250cd740 4 client.26080 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.204+0000 7fe6250cd740 1 -- 172.21.15.90:0/2846529669 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55706e42d6a0 con 0x55706e54ca70 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.205+0000 7fe61aa00640 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55706e54ca70 0x55706e54ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.206+0000 7fe61aa00640 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55706e54ca70 0x55706e54ee60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ced70307d0bf0e7 server_cookie=a74f5ce3f97c3eb7 in_seq=0 out_seq=0 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.206+0000 7fe605600640 10 client.26080 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe605600640 1 -- 172.21.15.90:0/2846529669 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55706e42d6a0 con 0x55706e54ca70 2024-09-10T22:24:50.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe605600640 10 client.26080 handle_command_reply: tid=0 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe6250cd740 1 client.26080 shutdown 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe6250cd740 1 -- 172.21.15.90:0/2846529669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 msgr2=0x55706e4dbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe6250cd740 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4dbd60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe61402ade0 tx=0x7fe614004040 comp rx=0 tx=0).stop 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe6250cd740 1 -- 172.21.15.90:0/2846529669 shutdown_connections 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe6250cd740 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55706e54ca70 0x55706e54ee60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe6250cd740 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55706e4c6ba0 0x55706e4d8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe6250cd740 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55706e4bac10 0x55706e4d8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe6250cd740 1 --2- 172.21.15.90:0/2846529669 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55706e4ba2f0 0x55706e4dbd60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe605600640 0 client.26080 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.600+0000 7fe6250cd740 1 -- 172.21.15.90:0/2846529669 >> 172.21.15.90:0/2846529669 conn(0x55706e4b8120 msgr2=0x55706e4b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:50.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.601+0000 7fe6250cd740 1 -- 172.21.15.90:0/2846529669 shutdown_connections 2024-09-10T22:24:50.846 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:24:50.846 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 76, 2024-09-10T22:24:50.846 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:24:50.846 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 66, 2024-09-10T22:24:50.846 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:24:50.846 INFO:teuthology.orchestra.run.smithi090.stdout: "66d48a8d": { 2024-09-10T22:24:50.846 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 66, 2024-09-10T22:24:50.846 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:24:50.846 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:24:50.847 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:50.848 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:50.848 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:50.848 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:50.848 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:24:50.848 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:24:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.601+0000 7fe6250cd740 1 -- 172.21.15.90:0/2846529669 wait complete. 2024-09-10T22:24:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.601+0000 7fe6250cd740 20 client.26080 trim_cache size 0 max 16384 2024-09-10T22:24:50.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 -- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe6000b5580 msgr2=0x7fe6000b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe6000b5580 0x7fe6000b7a40 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x55706e4b9000 tx=0x7fe60c005880 comp rx=0 tx=0).stop 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 -- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c102c10 msgr2=0x7fe61c19b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c102c10 0x7fe61c19b0f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00d950 tx=0x7fe60c00de10 comp rx=0 tx=0).stop 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 -- 172.21.15.90:0/2873387224 shutdown_connections 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe6000b5580 0x7fe6000b7a40 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe61c108fa0 0x7fe61c1a09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe61c108620 0x7fe61c19b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 --2- 172.21.15.90:0/2873387224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe61c102c10 0x7fe61c19b0f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.602+0000 7fe621000640 1 -- 172.21.15.90:0/2873387224 >> 172.21.15.90:0/2873387224 conn(0x7fe61c0fe6c0 msgr2=0x7fe61c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.603+0000 7fe621000640 1 -- 172.21.15.90:0/2873387224 shutdown_connections 2024-09-10T22:24:50.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:24:50.603+0000 7fe621000640 1 -- 172.21.15.90:0/2873387224 wait complete. 2024-09-10T22:24:50.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.61 MB/sec execute 341 sec latency 888.663 ms 2024-09-10T22:24:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:51 smithi090 ceph-mon[30350]: pgmap v1076: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.2 MiB/s wr, 130 op/s 2024-09-10T22:24:51.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:51 smithi154 ceph-mon[34495]: pgmap v1076: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.2 MiB/s wr, 130 op/s 2024-09-10T22:24:51.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:51 smithi171 ceph-mon[34138]: pgmap v1076: 129 pgs: 129 active+clean; 934 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 857 KiB/s rd, 2.2 MiB/s wr, 130 op/s 2024-09-10T22:24:51.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.60 MB/sec execute 342 sec latency 1694.269 ms 2024-09-10T22:24:52.049 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '66d48a8d', quiesce took 9.3 seconds. Will release after: 20.7 2024-09-10T22:24:52.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.59 MB/sec execute 343 sec latency 2694.342 ms 2024-09-10T22:24:53.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:53 smithi090 ceph-mon[30350]: pgmap v1077: 129 pgs: 129 active+clean; 937 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 163 op/s 2024-09-10T22:24:53.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:53 smithi154 ceph-mon[34495]: pgmap v1077: 129 pgs: 129 active+clean; 937 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 163 op/s 2024-09-10T22:24:53.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:53 smithi171 ceph-mon[34138]: pgmap v1077: 129 pgs: 129 active+clean; 937 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 163 op/s 2024-09-10T22:24:53.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.57 MB/sec execute 344 sec latency 3694.450 ms 2024-09-10T22:24:54.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.56 MB/sec execute 345 sec latency 4694.593 ms 2024-09-10T22:24:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:55 smithi090 ceph-mon[30350]: pgmap v1078: 129 pgs: 129 active+clean; 936 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 176 op/s 2024-09-10T22:24:55.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:55 smithi154 ceph-mon[34495]: pgmap v1078: 129 pgs: 129 active+clean; 936 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 176 op/s 2024-09-10T22:24:55.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:55 smithi171 ceph-mon[34138]: pgmap v1078: 129 pgs: 129 active+clean; 936 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 176 op/s 2024-09-10T22:24:55.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.55 MB/sec execute 346 sec latency 5694.717 ms 2024-09-10T22:24:56.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.53 MB/sec execute 347 sec latency 6694.821 ms 2024-09-10T22:24:57.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:57 smithi154 ceph-mon[34495]: pgmap v1079: 129 pgs: 129 active+clean; 938 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 195 op/s 2024-09-10T22:24:57.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:57 smithi171 ceph-mon[34138]: pgmap v1079: 129 pgs: 129 active+clean; 938 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 195 op/s 2024-09-10T22:24:57.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:57 smithi090 ceph-mon[30350]: pgmap v1079: 129 pgs: 129 active+clean; 938 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 195 op/s 2024-09-10T22:24:57.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.52 MB/sec execute 348 sec latency 7694.954 ms 2024-09-10T22:24:58.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.51 MB/sec execute 349 sec latency 8695.091 ms 2024-09-10T22:24:59.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:24:59 smithi154 ceph-mon[34495]: pgmap v1080: 129 pgs: 129 active+clean; 938 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-09-10T22:24:59.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:24:59 smithi171 ceph-mon[34138]: pgmap v1080: 129 pgs: 129 active+clean; 938 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-09-10T22:24:59.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:24:59 smithi090 ceph-mon[30350]: pgmap v1080: 129 pgs: 129 active+clean; 938 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-09-10T22:24:59.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.49 MB/sec execute 350 sec latency 9695.196 ms 2024-09-10T22:25:00.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.48 MB/sec execute 351 sec latency 10695.282 ms 2024-09-10T22:25:01.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:01 smithi154 ceph-mon[34495]: pgmap v1081: 129 pgs: 129 active+clean; 938 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-10T22:25:01.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:01 smithi171 ceph-mon[34138]: pgmap v1081: 129 pgs: 129 active+clean; 938 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-10T22:25:01.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:01 smithi090 ceph-mon[30350]: pgmap v1081: 129 pgs: 129 active+clean; 938 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.7 MiB/s wr, 136 op/s 2024-09-10T22:25:01.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.47 MB/sec execute 352 sec latency 11695.368 ms 2024-09-10T22:25:02.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.46 MB/sec execute 353 sec latency 12695.454 ms 2024-09-10T22:25:03.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:03 smithi154 ceph-mon[34495]: pgmap v1082: 129 pgs: 129 active+clean; 938 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 808 KiB/s wr, 82 op/s 2024-09-10T22:25:03.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:03 smithi171 ceph-mon[34138]: pgmap v1082: 129 pgs: 129 active+clean; 938 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 808 KiB/s wr, 82 op/s 2024-09-10T22:25:03.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:03 smithi090 ceph-mon[30350]: pgmap v1082: 129 pgs: 129 active+clean; 938 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 808 KiB/s wr, 82 op/s 2024-09-10T22:25:03.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.44 MB/sec execute 354 sec latency 13695.577 ms 2024-09-10T22:25:04.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.43 MB/sec execute 355 sec latency 14695.755 ms 2024-09-10T22:25:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:05 smithi154 ceph-mon[34495]: pgmap v1083: 129 pgs: 129 active+clean; 938 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 550 KiB/s wr, 62 op/s 2024-09-10T22:25:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:05 smithi154 ceph-mon[34495]: osdmap e121: 12 total, 12 up, 12 in 2024-09-10T22:25:05.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:05 smithi171 ceph-mon[34138]: pgmap v1083: 129 pgs: 129 active+clean; 938 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 550 KiB/s wr, 62 op/s 2024-09-10T22:25:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:05 smithi171 ceph-mon[34138]: osdmap e121: 12 total, 12 up, 12 in 2024-09-10T22:25:05.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:05 smithi090 ceph-mon[30350]: pgmap v1083: 129 pgs: 129 active+clean; 938 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 550 KiB/s wr, 62 op/s 2024-09-10T22:25:05.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:05 smithi090 ceph-mon[30350]: osdmap e121: 12 total, 12 up, 12 in 2024-09-10T22:25:05.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.42 MB/sec execute 356 sec latency 15695.890 ms 2024-09-10T22:25:06.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.41 MB/sec execute 357 sec latency 16696.031 ms 2024-09-10T22:25:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:07 smithi154 ceph-mon[34495]: pgmap v1085: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 933 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-10T22:25:07.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:07 smithi171 ceph-mon[34138]: pgmap v1085: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 933 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-10T22:25:07.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:07 smithi090 ceph-mon[30350]: pgmap v1085: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 933 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-10T22:25:07.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.39 MB/sec execute 358 sec latency 17696.168 ms 2024-09-10T22:25:08.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.38 MB/sec execute 359 sec latency 18696.285 ms 2024-09-10T22:25:09.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:09 smithi154 ceph-mon[34495]: pgmap v1086: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 927 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-10T22:25:09.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:09 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:09.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:09 smithi171 ceph-mon[34138]: pgmap v1086: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 927 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-10T22:25:09.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:09 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:09.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:09 smithi090 ceph-mon[30350]: pgmap v1086: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 927 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 127 KiB/s wr, 36 op/s 2024-09-10T22:25:09.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:09 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:09.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.37 MB/sec execute 360 sec latency 19696.395 ms 2024-09-10T22:25:10.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.36 MB/sec execute 361 sec latency 20696.498 ms 2024-09-10T22:25:11.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.35 MB/sec execute 362 sec latency 21696.676 ms 2024-09-10T22:25:12.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:11 smithi090 ceph-mon[30350]: pgmap v1087: 129 pgs: 129 active+clean; 924 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 113 KiB/s wr, 34 op/s 2024-09-10T22:25:12.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:11 smithi090 ceph-mon[30350]: osdmap e122: 12 total, 12 up, 12 in 2024-09-10T22:25:12.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:11 smithi154 ceph-mon[34495]: pgmap v1087: 129 pgs: 129 active+clean; 924 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 113 KiB/s wr, 34 op/s 2024-09-10T22:25:12.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:11 smithi154 ceph-mon[34495]: osdmap e122: 12 total, 12 up, 12 in 2024-09-10T22:25:12.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:11 smithi171 ceph-mon[34138]: pgmap v1087: 129 pgs: 129 active+clean; 924 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 113 KiB/s wr, 34 op/s 2024-09-10T22:25:12.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:11 smithi171 ceph-mon[34138]: osdmap e122: 12 total, 12 up, 12 in 2024-09-10T22:25:12.751 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:25:12.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.33 MB/sec execute 363 sec latency 22696.752 ms 2024-09-10T22:25:13.212 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:25:13.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:13 smithi090 ceph-mon[30350]: pgmap v1089: 129 pgs: 129 active+clean; 924 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-09-10T22:25:13.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.32 MB/sec execute 364 sec latency 23696.797 ms 2024-09-10T22:25:14.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:13 smithi154 ceph-mon[34495]: pgmap v1089: 129 pgs: 129 active+clean; 924 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-09-10T22:25:14.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:13 smithi171 ceph-mon[34138]: pgmap v1089: 129 pgs: 129 active+clean; 924 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 70 KiB/s wr, 16 op/s 2024-09-10T22:25:14.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.656+0000 7f731b600640 1 -- 172.21.15.90:0/2276614194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 msgr2=0x7f7314103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:14.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.656+0000 7f731b600640 1 --2- 172.21.15.90:0/2276614194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 0x7f7314103180 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7308009ca0 tx=0x7f730802a8b0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.656+0000 7f731b600640 1 -- 172.21.15.90:0/2276614194 shutdown_connections 2024-09-10T22:25:14.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.656+0000 7f731b600640 1 --2- 172.21.15.90:0/2276614194 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f7314109130 0x7f731410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.656+0000 7f731b600640 1 --2- 172.21.15.90:0/2276614194 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f73141087b0 0x7f7314108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.656+0000 7f731b600640 1 --2- 172.21.15.90:0/2276614194 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 0x7f7314103180 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.656+0000 7f731b600640 1 -- 172.21.15.90:0/2276614194 >> 172.21.15.90:0/2276614194 conn(0x7f73140fe810 msgr2=0x7f7314100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:25:14.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.656+0000 7f731b600640 1 -- 172.21.15.90:0/2276614194 shutdown_connections 2024-09-10T22:25:14.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.656+0000 7f731b600640 1 -- 172.21.15.90:0/2276614194 wait complete. 2024-09-10T22:25:14.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.657+0000 7f731b600640 1 Processor -- start 2024-09-10T22:25:14.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.657+0000 7f731b600640 1 -- start start 2024-09-10T22:25:14.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f731b600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 0x7f731419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:14.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f731b600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73141087b0 0x7f731419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:14.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f731b600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7314109130 0x7f73141a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:14.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f731b600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7314112ca0 con 0x7f7314102da0 2024-09-10T22:25:14.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f731b600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7314112b20 con 0x7f7314109130 2024-09-10T22:25:14.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f731b600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7314112e20 con 0x7f73141087b0 2024-09-10T22:25:14.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f7318c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 0x7f731419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:14.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f7318c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 0x7f731419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58660/0 (socket says 172.21.15.90:58660) 2024-09-10T22:25:14.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f7318c00640 1 -- 172.21.15.90:0/32432720 learned_addr learned my addr 172.21.15.90:0/32432720 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f7313e00640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73141087b0 0x7f731419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.658+0000 7f7319600640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7314109130 0x7f73141a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f7318c00640 1 -- 172.21.15.90:0/32432720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73141087b0 msgr2=0x7f731419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f7318c00640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73141087b0 0x7f731419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f7318c00640 1 -- 172.21.15.90:0/32432720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7314109130 msgr2=0x7f73141a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f7318c00640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7314109130 0x7f73141a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f7318c00640 1 -- 172.21.15.90:0/32432720 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73141a1430 con 0x7f7314102da0 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f7318c00640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 0x7f731419b380 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7308006010 tx=0x7f7308003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f7311600640 1 -- 172.21.15.90:0/32432720 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308004240 con 0x7f7314102da0 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f7311600640 1 -- 172.21.15.90:0/32432720 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7308040070 con 0x7f7314102da0 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73141a16c0 con 0x7f7314102da0 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.659+0000 7f7311600640 1 -- 172.21.15.90:0/32432720 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7308045b70 con 0x7f7314102da0 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.660+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73141a7de0 con 0x7f7314102da0 2024-09-10T22:25:14.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.661+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7314103180 con 0x7f7314102da0 2024-09-10T22:25:14.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.663+0000 7f7311600640 1 -- 172.21.15.90:0/32432720 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f73080043e0 con 0x7f7314102da0 2024-09-10T22:25:14.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.664+0000 7f7311600640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72fc0b5450 0x7f72fc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:14.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.664+0000 7f7311600640 1 -- 172.21.15.90:0/32432720 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9490+0+0 (secure 0 0 0) 0x7f7308107840 con 0x7f7314102da0 2024-09-10T22:25:14.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.664+0000 7f7313e00640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72fc0b5450 0x7f72fc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:14.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.665+0000 7f7313e00640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72fc0b5450 0x7f72fc0b7910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f73140ffba0 tx=0x7f730000a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:25:14.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.666+0000 7f7311600640 1 -- 172.21.15.90:0/32432720 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7308004700 con 0x7f7314102da0 2024-09-10T22:25:14.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.889+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73141a8160 con 0x7f7314102da0 2024-09-10T22:25:14.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.890+0000 7f7311600640 1 -- 172.21.15.90:0/32432720 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+12006 (secure 0 0 0) 0x7f73080ce020 con 0x7f7314102da0 2024-09-10T22:25:14.892 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:25:14.892 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":78,"btime":"2024-09-10T22:25:13:815147+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":78,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:25:13.791305+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:25:14.892 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 78 2024-09-10T22:25:14.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72fc0b5450 msgr2=0x7f72fc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:14.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72fc0b5450 0x7f72fc0b7910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f73140ffba0 tx=0x7f730000a2d0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 msgr2=0x7f731419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:14.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 0x7f731419b380 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7308006010 tx=0x7f7308003ba0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 shutdown_connections 2024-09-10T22:25:14.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72fc0b5450 0x7f72fc0b7910 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.894 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7314109130 0x7f73141a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f73141087b0 0x7f731419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 --2- 172.21.15.90:0/32432720 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7314102da0 0x7f731419b380 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:14.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 >> 172.21.15.90:0/32432720 conn(0x7f73140fe810 msgr2=0x7f7314106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:25:14.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 shutdown_connections 2024-09-10T22:25:14.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:14.892+0000 7f731b600640 1 -- 172.21.15.90:0/32432720 wait complete. 2024-09-10T22:25:14.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.31 MB/sec execute 365 sec latency 24696.842 ms 2024-09-10T22:25:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:14 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:14 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:15.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:14 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:15.843 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 66d48a8d --release --await' 2024-09-10T22:25:15.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 66d48a8d --release --await 2024-09-10T22:25:15.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.30 MB/sec execute 366 sec latency 25696.883 ms 2024-09-10T22:25:16.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:15 smithi090 ceph-mon[30350]: pgmap v1090: 129 pgs: 129 active+clean; 925 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-09-10T22:25:16.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:15 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/32432720' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:25:16.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:15 smithi154 ceph-mon[34495]: pgmap v1090: 129 pgs: 129 active+clean; 925 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-09-10T22:25:16.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:15 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/32432720' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:25:16.228 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:25:16.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:15 smithi171 ceph-mon[34138]: pgmap v1090: 129 pgs: 129 active+clean; 925 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 60 KiB/s wr, 14 op/s 2024-09-10T22:25:16.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:15 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/32432720' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:25:16.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39292 4.29 MB/sec execute 367 sec latency 26696.970 ms 2024-09-10T22:25:16.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.942+0000 7f28e4e00640 1 -- 172.21.15.90:0/1865946592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 msgr2=0x7f28e010f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:16.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.942+0000 7f28e4e00640 1 --2- 172.21.15.90:0/1865946592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 0x7f28e010f790 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009ca0 tx=0x7f28d802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.942+0000 7f28e4e00640 1 -- 172.21.15.90:0/1865946592 shutdown_connections 2024-09-10T22:25:16.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.942+0000 7f28e4e00640 1 --2- 172.21.15.90:0/1865946592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 0x7f28e010f790 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.942+0000 7f28e4e00640 1 --2- 172.21.15.90:0/1865946592 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28e0108610 0x7f28e0108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.942+0000 7f28e4e00640 1 --2- 172.21.15.90:0/1865946592 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28e0102c00 0x7f28e0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.942+0000 7f28e4e00640 1 -- 172.21.15.90:0/1865946592 >> 172.21.15.90:0/1865946592 conn(0x7f28e00fe6d0 msgr2=0x7f28e0100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:25:16.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.942+0000 7f28e4e00640 1 -- 172.21.15.90:0/1865946592 shutdown_connections 2024-09-10T22:25:16.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.942+0000 7f28e4e00640 1 -- 172.21.15.90:0/1865946592 wait complete. 2024-09-10T22:25:16.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.943+0000 7f28e4e00640 1 Processor -- start 2024-09-10T22:25:16.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.943+0000 7f28e4e00640 1 -- start start 2024-09-10T22:25:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28e4e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28e0102c00 0x7f28e019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28e4e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28e0108610 0x7f28e019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28e4e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 0x7f28e01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28e4e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0112b20 con 0x7f28e0108f90 2024-09-10T22:25:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28e4e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01129a0 con 0x7f28e0102c00 2024-09-10T22:25:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28e4e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0112ca0 con 0x7f28e0108610 2024-09-10T22:25:16.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28dfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 0x7f28e01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28df400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28e0102c00 0x7f28e019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28dfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 0x7f28e01a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58686/0 (socket says 172.21.15.90:58686) 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28d7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28e0108610 0x7f28e019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.944+0000 7f28dfe00640 1 -- 172.21.15.90:0/2682467775 learned_addr learned my addr 172.21.15.90:0/2682467775 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28dfe00640 1 -- 172.21.15.90:0/2682467775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28e0108610 msgr2=0x7f28e019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28dfe00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28e0108610 0x7f28e019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28dfe00640 1 -- 172.21.15.90:0/2682467775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28e0102c00 msgr2=0x7f28e019b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28dfe00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28e0102c00 0x7f28e019b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28dfe00640 1 -- 172.21.15.90:0/2682467775 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e01a11e0 con 0x7f28e0108f90 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28dfe00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 0x7f28e01a0a10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f28d8031040 tx=0x7f28d8032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28dcc00640 1 -- 172.21.15.90:0/2682467775 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8004040 con 0x7f28e0108f90 2024-09-10T22:25:16.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28dcc00640 1 -- 172.21.15.90:0/2682467775 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28d8004670 con 0x7f28e0108f90 2024-09-10T22:25:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28e4e00640 1 -- 172.21.15.90:0/2682467775 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e01a1470 con 0x7f28e0108f90 2024-09-10T22:25:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28dcc00640 1 -- 172.21.15.90:0/2682467775 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d8030bb0 con 0x7f28e0108f90 2024-09-10T22:25:16.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.945+0000 7f28e4e00640 1 -- 172.21.15.90:0/2682467775 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e01a7b90 con 0x7f28e0108f90 2024-09-10T22:25:16.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.949+0000 7f28dcc00640 1 -- 172.21.15.90:0/2682467775 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f28d80041e0 con 0x7f28e0108f90 2024-09-10T22:25:16.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28e8e63740 1 Processor -- start 2024-09-10T22:25:16.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28e8e63740 1 -- start start 2024-09-10T22:25:16.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1b1d0 0x560a52f1b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 0x560a52f27540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 0x560a52f29e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28e8e63740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560a52e92290 con 0x560a52f1b1d0 2024-09-10T22:25:16.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28e8e63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560a52e816c0 con 0x560a52f1baf0 2024-09-10T22:25:16.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28e8e63740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560a52e854d0 con 0x560a52f27a80 2024-09-10T22:25:16.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28d7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1b1d0 0x560a52f1b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28d7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1b1d0 0x560a52f1b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58696/0 (socket says 172.21.15.90:58696) 2024-09-10T22:25:16.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28d7e00640 1 -- 172.21.15.90:0/4104667605 learned_addr learned my addr 172.21.15.90:0/4104667605 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:25:16.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.950+0000 7f28df400640 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 0x560a52f27540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28dcc00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f28c40b5500 0x7f28c40b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28d7e00640 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 0x560a52f29e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28d7e00640 1 -- 172.21.15.90:0/4104667605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 msgr2=0x560a52f29e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:25:16.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28d7e00640 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 0x560a52f29e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28d7e00640 1 -- 172.21.15.90:0/4104667605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 msgr2=0x560a52f27540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:25:16.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28d7e00640 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 0x560a52f27540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28d7e00640 1 -- 172.21.15.90:0/4104667605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a52e0e460 con 0x560a52f1b1d0 2024-09-10T22:25:16.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28df400640 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 0x560a52f27540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:25:16.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28dcc00640 1 -- 172.21.15.90:0/2682467775 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9490+0+0 (secure 0 0 0) 0x7f28d8107550 con 0x7f28e0108f90 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28dfe00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f28c40b5500 0x7f28c40b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.951+0000 7f28d7e00640 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1b1d0 0x560a52f1b5b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f28c80124f0 tx=0x7f28c800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f11464ff20f3f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.952+0000 7f28d4c00640 1 -- 172.21.15.90:0/4104667605 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8004030 con 0x560a52f1b1d0 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.952+0000 7f28d4c00640 1 -- 172.21.15.90:0/4104667605 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28c80035f0 con 0x560a52f1b1d0 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.952+0000 7f28d4c00640 1 -- 172.21.15.90:0/4104667605 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8010660 con 0x560a52f1b1d0 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.952+0000 7f28e8e63740 1 -- 172.21.15.90:0/4104667605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1b1d0 msgr2=0x560a52f1b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.952+0000 7f28e8e63740 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1b1d0 0x560a52f1b5b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f28c80124f0 tx=0x7f28c800b7c0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.952+0000 7f28dfe00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f28c40b5500 0x7f28c40b79c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28d80cd590 tx=0x7f28d80095c0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.953+0000 7f28e8e63740 1 -- 172.21.15.90:0/4104667605 shutdown_connections 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.953+0000 7f28e8e63740 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 0x560a52f29e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.953+0000 7f28e8e63740 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 0x560a52f27540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.953+0000 7f28e8e63740 1 --2- 172.21.15.90:0/4104667605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1b1d0 0x560a52f1b5b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.953+0000 7f28e8e63740 1 -- 172.21.15.90:0/4104667605 >> 172.21.15.90:0/4104667605 conn(0x560a52f19000 msgr2=0x560a52f1a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:25:16.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.953+0000 7f28e8e63740 1 -- 172.21.15.90:0/4104667605 shutdown_connections 2024-09-10T22:25:16.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.953+0000 7f28e8e63740 1 -- 172.21.15.90:0/4104667605 wait complete. 2024-09-10T22:25:16.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.953+0000 7f28e8e63740 1 -- start start 2024-09-10T22:25:16.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.953+0000 7f28e8e63740 1 Processor -- start 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 0x560a52f24ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 0x560a52f24ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f27a80 0x560a52f25530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28e8e63740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560a52ce2bb0 con 0x560a52f27a80 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28e8e63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x560a52e1c6c0 con 0x560a52f1baf0 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28e8e63740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x560a52f2a730 con 0x560a52f1b1d0 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28e8e63740 10 client.0 _ll_register_callbacks cb 0x560a52e37490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28d7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f27a80 0x560a52f25530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28d7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f27a80 0x560a52f25530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58698/0 (socket says 172.21.15.90:58698) 2024-09-10T22:25:16.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28d7e00640 1 -- 172.21.15.90:0/3963096360 learned_addr learned my addr 172.21.15.90:0/3963096360 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28d7e00640 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 0x560a52f24ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.954+0000 7f28df400640 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 0x560a52f24ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28d7e00640 1 -- 172.21.15.90:0/3963096360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 msgr2=0x560a52f24ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28d7e00640 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 0x560a52f24ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28d7e00640 1 -- 172.21.15.90:0/3963096360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 msgr2=0x560a52f24ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28d7e00640 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 0x560a52f24ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28d7e00640 1 -- 172.21.15.90:0/3963096360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a52bdfb00 con 0x560a52f27a80 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28d7e00640 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f27a80 0x560a52f25530 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f28c80124c0 tx=0x7f28c8003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ab676e037af1c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28cd600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28cd600640 1 -- 172.21.15.90:0/3963096360 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8028040 con 0x560a52f27a80 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28e8e63740 1 -- 172.21.15.90:0/3963096360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c80035f0 con 0x560a52f27a80 2024-09-10T22:25:16.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28cd600640 1 -- 172.21.15.90:0/3963096360 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9490+0+0 (secure 0 0 0) 0x7f28c80039a0 con 0x560a52f27a80 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28cd600640 1 -- 172.21.15.90:0/3963096360 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f28c8034050 con 0x560a52f27a80 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28e8e63740 10 client.16485 fetch_fsmap learned FSMap version 78 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.955+0000 7f28e8e63740 1 -- 172.21.15.90:0/3963096360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x560a52c56960 con 0x560a52f27a80 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.956+0000 7f28cd600640 1 -- 172.21.15.90:0/3963096360 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28c80035f0 con 0x560a52f27a80 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.956+0000 7f28cd600640 1 -- 172.21.15.90:0/3963096360 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 78) ==== 8418+0+0 (secure 0 0 0) 0x7f28c8004750 con 0x560a52f27a80 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.956+0000 7f28e8e63740 10 client.16485 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.956+0000 7f28e8e63740 10 client.16485 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.956+0000 7f28e8e63740 20 client.16485 populate_metadata read hostname 'smithi090' 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.956+0000 7f28e8e63740 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560a52f37560 0x560a52f39950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.956+0000 7f28e8e63740 4 client.16485 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.956+0000 7f28e8e63740 1 -- 172.21.15.90:0/3963096360 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a52e941e0 con 0x560a52f37560 2024-09-10T22:25:16.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.957+0000 7f28df400640 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560a52f37560 0x560a52f39950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.957+0000 7f28df400640 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560a52f37560 0x560a52f39950 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b4a240b53e1b210 server_cookie=2db319945bdd149a in_seq=0 out_seq=0 2024-09-10T22:25:16.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.957+0000 7f28cd600640 10 client.16485 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:25:16.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28cd600640 1 -- 172.21.15.90:0/3963096360 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560a52e941e0 con 0x560a52f37560 2024-09-10T22:25:16.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28cd600640 10 client.16485 handle_command_reply: tid=0 2024-09-10T22:25:16.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28e8e63740 1 client.16485 shutdown 2024-09-10T22:25:16.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28e8e63740 1 -- 172.21.15.90:0/3963096360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f27a80 msgr2=0x560a52f25530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:16.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28e8e63740 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f27a80 0x560a52f25530 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f28c80124c0 tx=0x7f28c8003ee0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28e8e63740 1 -- 172.21.15.90:0/3963096360 shutdown_connections 2024-09-10T22:25:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28e8e63740 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560a52f37560 0x560a52f39950 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28e8e63740 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f27a80 0x560a52f25530 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28e8e63740 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1baf0 0x560a52f24ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28e8e63740 1 --2- 172.21.15.90:0/3963096360 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 0x560a52f24ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28cd600640 0 client.16485 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:25:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.958+0000 7f28e8e63740 1 -- 172.21.15.90:0/3963096360 >> 172.21.15.90:0/3963096360 conn(0x560a52f19000 msgr2=0x560a52f1eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:25:16.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.959+0000 7f28e8e63740 1 -- 172.21.15.90:0/3963096360 shutdown_connections 2024-09-10T22:25:16.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.959+0000 7f28e8e63740 1 -- 172.21.15.90:0/3963096360 wait complete. 2024-09-10T22:25:16.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.959+0000 7f28e8e63740 20 client.16485 trim_cache size 0 max 16384 2024-09-10T22:25:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28e8e63740 1 Processor -- start 2024-09-10T22:25:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28e8e63740 1 -- start start 2024-09-10T22:25:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1b1d0 0x560a52f1f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 0x560a52f21cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28e8e63740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b1d30 con 0x560a52f1baf0 2024-09-10T22:25:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28e8e63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b39f0 con 0x560a52f1b1d0 2024-09-10T22:25:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28e8e63740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b2430 con 0x560a52f27a80 2024-09-10T22:25:16.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28df400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28df400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f267f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58700/0 (socket says 172.21.15.90:58700) 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28df400640 1 -- 172.21.15.90:0/1403774770 learned_addr learned my addr 172.21.15.90:0/1403774770 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28d7e00640 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1b1d0 0x560a52f1f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28d7e00640 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 0x560a52f21cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28df400640 1 -- 172.21.15.90:0/1403774770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 msgr2=0x560a52f21cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28df400640 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 0x560a52f21cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28df400640 1 -- 172.21.15.90:0/1403774770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1b1d0 msgr2=0x560a52f1f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28df400640 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1b1d0 0x560a52f1f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28df400640 1 -- 172.21.15.90:0/1403774770 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a52f22fb0 con 0x560a52f1baf0 2024-09-10T22:25:16.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.977+0000 7f28d7e00640 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1b1d0 0x560a52f1f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:25:16.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.978+0000 7f28df400640 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f267f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x560a52f1e350 tx=0x7f28d0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aedc4f0f4829df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:25:16.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.978+0000 7f28cfe00640 1 -- 172.21.15.90:0/1403774770 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0010900 con 0x560a52f1baf0 2024-09-10T22:25:16.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.978+0000 7f28cfe00640 1 -- 172.21.15.90:0/1403774770 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28d0005020 con 0x560a52f1baf0 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.978+0000 7f28cfe00640 1 -- 172.21.15.90:0/1403774770 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0004a00 con 0x560a52f1baf0 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 -- 172.21.15.90:0/1403774770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 msgr2=0x560a52f267f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f267f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x560a52f1e350 tx=0x7f28d0011ba0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 -- 172.21.15.90:0/1403774770 shutdown_connections 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f27a80 0x560a52f21cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f267f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 --2- 172.21.15.90:0/1403774770 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f1b1d0 0x560a52f1f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 -- 172.21.15.90:0/1403774770 >> 172.21.15.90:0/1403774770 conn(0x560a52f19000 msgr2=0x560a52f260e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 -- 172.21.15.90:0/1403774770 shutdown_connections 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 -- 172.21.15.90:0/1403774770 wait complete. 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.979+0000 7f28e8e63740 1 -- start start 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28e8e63740 1 Processor -- start 2024-09-10T22:25:16.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 0x560a52f2bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f2c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28e8e63740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f27a80 0x560a52fa33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28e8e63740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b3320 con 0x560a52f1baf0 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28e8e63740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01b2bc0 con 0x560a52f27a80 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28e8e63740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f28e01afd30 con 0x560a52f1b1d0 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28e8e63740 10 client.0 _ll_register_callbacks cb 0x7f28d000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28df400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f2c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28df400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f2c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58708/0 (socket says 172.21.15.90:58708) 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.980+0000 7f28df400640 1 -- 172.21.15.90:0/88108757 learned_addr learned my addr 172.21.15.90:0/88108757 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28d7e00640 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f27a80 0x560a52fa33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28d7e00640 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 0x560a52f2bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28df400640 1 -- 172.21.15.90:0/88108757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 msgr2=0x560a52f2bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:25:16.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28df400640 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 0x560a52f2bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28df400640 1 -- 172.21.15.90:0/88108757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f27a80 msgr2=0x560a52fa33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:25:16.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28df400640 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f27a80 0x560a52fa33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:16.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28df400640 1 -- 172.21.15.90:0/88108757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a52f2cac0 con 0x560a52f1baf0 2024-09-10T22:25:16.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28d7e00640 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 0x560a52f2bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:25:16.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28df400640 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f2c1f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f28d000be50 tx=0x7f28d0005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32554e6633f4314 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:25:16.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28cd600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:25:16.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.981+0000 7f28cd600640 1 -- 172.21.15.90:0/88108757 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0005660 con 0x560a52f1baf0 2024-09-10T22:25:16.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28e8e63740 1 -- 172.21.15.90:0/88108757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d0005020 con 0x560a52f1baf0 2024-09-10T22:25:16.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28cd600640 1 -- 172.21.15.90:0/88108757 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9490+0+0 (secure 0 0 0) 0x7f28d0004020 con 0x560a52f1baf0 2024-09-10T22:25:16.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28cd600640 1 -- 172.21.15.90:0/88108757 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f28d003a050 con 0x560a52f1baf0 2024-09-10T22:25:16.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28e8e63740 10 client.16491 fetch_fsmap learned FSMap version 78 2024-09-10T22:25:16.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28e8e63740 1 -- 172.21.15.90:0/88108757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x560a52fa4a20 con 0x560a52f1baf0 2024-09-10T22:25:16.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28cd600640 1 -- 172.21.15.90:0/88108757 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f28d0005020 con 0x560a52f1baf0 2024-09-10T22:25:16.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28cd600640 1 -- 172.21.15.90:0/88108757 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 78) ==== 8418+0+0 (secure 0 0 0) 0x7f28d0012ce0 con 0x560a52f1baf0 2024-09-10T22:25:16.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28e8e63740 10 client.16491 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T22:25:16.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28e8e63740 10 client.16491 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:25:16.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28e8e63740 20 client.16491 populate_metadata read hostname 'smithi090' 2024-09-10T22:25:16.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28e8e63740 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560a52fa5110 0x560a52fa7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:25:16.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28e8e63740 4 client.16491 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "66d48a8d", "release": true, "await": true}] 2024-09-10T22:25:16.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.982+0000 7f28e8e63740 1 -- 172.21.15.90:0/88108757 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "66d48a8d", "release": true, "await": true}) -- 0x560a52e8fc90 con 0x560a52fa5110 2024-09-10T22:25:16.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.983+0000 7f28d7e00640 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560a52fa5110 0x560a52fa7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:25:16.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.983+0000 7f28d7e00640 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560a52fa5110 0x560a52fa7530 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e5978484e95336 server_cookie=8e6788aa24dedcc1 in_seq=0 out_seq=0 2024-09-10T22:25:16.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:16.983+0000 7f28cd600640 10 client.16491 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:25:17.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.029+0000 7f28cd600640 1 -- 172.21.15.90:0/88108757 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560a52e8fc90 con 0x560a52fa5110 2024-09-10T22:25:17.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.029+0000 7f28cd600640 10 client.16491 handle_command_reply: tid=0 2024-09-10T22:25:17.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.029+0000 7f28e8e63740 1 client.16491 shutdown 2024-09-10T22:25:17.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 -- 172.21.15.90:0/88108757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 msgr2=0x560a52f2c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:17.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f2c1f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f28d000be50 tx=0x7f28d0005260 comp rx=0 tx=0).stop 2024-09-10T22:25:17.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 -- 172.21.15.90:0/88108757 shutdown_connections 2024-09-10T22:25:17.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x560a52fa5110 0x560a52fa7530 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x560a52f27a80 0x560a52fa33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560a52f1baf0 0x560a52f2c1f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 --2- 172.21.15.90:0/88108757 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x560a52f1b1d0 0x560a52f2bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 -- 172.21.15.90:0/88108757 >> 172.21.15.90:0/88108757 conn(0x560a52f19000 msgr2=0x560a52f26900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:25:17.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28cd600640 0 client.16491 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:25:17.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 -- 172.21.15.90:0/88108757 shutdown_connections 2024-09-10T22:25:17.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.030+0000 7f28e8e63740 1 -- 172.21.15.90:0/88108757 wait complete. 2024-09-10T22:25:17.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.031+0000 7f28e8e63740 20 client.16491 trim_cache size 0 max 16384 2024-09-10T22:25:17.034 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:25:17.034 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 78, 2024-09-10T22:25:17.034 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:25:17.034 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 68, 2024-09-10T22:25:17.034 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "66d48a8d": { 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 68, 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:25:17.035 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:25:17.036 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:25:17.036 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:25:17.036 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:25:17.036 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:25:17.036 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:25:17.036 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:25:17.036 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:25:17.036 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.031+0000 7f28e4e00640 1 -- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f28c40b5500 msgr2=0x7f28c40b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.031+0000 7f28e4e00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f28c40b5500 0x7f28c40b79c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28d80cd590 tx=0x7f28d80095c0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.031+0000 7f28e4e00640 1 -- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 msgr2=0x7f28e01a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.031+0000 7f28e4e00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 0x7f28e01a0a10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f28d8031040 tx=0x7f28d8032bc0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.032+0000 7f28e4e00640 1 -- 172.21.15.90:0/2682467775 shutdown_connections 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.032+0000 7f28e4e00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f28c40b5500 0x7f28c40b79c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.032+0000 7f28e4e00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f28e0108f90 0x7f28e01a0a10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.032+0000 7f28e4e00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f28e0108610 0x7f28e019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.032+0000 7f28e4e00640 1 --2- 172.21.15.90:0/2682467775 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f28e0102c00 0x7f28e019b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:25:17.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.032+0000 7f28e4e00640 1 -- 172.21.15.90:0/2682467775 >> 172.21.15.90:0/2682467775 conn(0x7f28e00fe6d0 msgr2=0x7f28e00753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:25:17.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.032+0000 7f28e4e00640 1 -- 172.21.15.90:0/2682467775 shutdown_connections 2024-09-10T22:25:17.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:25:17.032+0000 7f28e4e00640 1 -- 172.21.15.90:0/2682467775 wait complete. 2024-09-10T22:25:17.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39418 4.29 MB/sec execute 368 sec latency 26838.171 ms 2024-09-10T22:25:18.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:17 smithi090 ceph-mon[30350]: pgmap v1091: 129 pgs: 129 active+clean; 925 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-10T22:25:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:17 smithi154 ceph-mon[34495]: pgmap v1091: 129 pgs: 129 active+clean; 925 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-10T22:25:18.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:17 smithi171 ceph-mon[34138]: pgmap v1091: 129 pgs: 129 active+clean; 925 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-09-10T22:25:18.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39496 4.29 MB/sec execute 369 sec latency 290.043 ms 2024-09-10T22:25:18.909 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '66d48a8d', total seconds elapsed: 36.1 2024-09-10T22:25:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:18 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:19.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:18 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:19.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:18 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:19.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39618 4.29 MB/sec execute 370 sec latency 79.378 ms 2024-09-10T22:25:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:19 smithi090 ceph-mon[30350]: pgmap v1092: 129 pgs: 129 active+clean; 926 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 141 KiB/s wr, 18 op/s 2024-09-10T22:25:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:19 smithi154 ceph-mon[34495]: pgmap v1092: 129 pgs: 129 active+clean; 926 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 141 KiB/s wr, 18 op/s 2024-09-10T22:25:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:19 smithi171 ceph-mon[34138]: pgmap v1092: 129 pgs: 129 active+clean; 926 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 141 KiB/s wr, 18 op/s 2024-09-10T22:25:20.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39718 4.29 MB/sec execute 371 sec latency 178.942 ms 2024-09-10T22:25:21.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:20 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:25:21.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:20 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:25:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:20 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:25:21.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39840 4.30 MB/sec execute 372 sec latency 71.318 ms 2024-09-10T22:25:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:21 smithi154 ceph-mon[34495]: pgmap v1093: 129 pgs: 129 active+clean; 930 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 571 KiB/s wr, 35 op/s 2024-09-10T22:25:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:21 smithi171 ceph-mon[34138]: pgmap v1093: 129 pgs: 129 active+clean; 930 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 571 KiB/s wr, 35 op/s 2024-09-10T22:25:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:21 smithi090 ceph-mon[30350]: pgmap v1093: 129 pgs: 129 active+clean; 930 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 571 KiB/s wr, 35 op/s 2024-09-10T22:25:22.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39960 4.30 MB/sec execute 373 sec latency 82.209 ms 2024-09-10T22:25:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:22 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:25:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:25:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:25:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:22 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:25:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:25:23.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:25:23.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:22 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:23.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:25:23.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:25:23.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:25:23.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 40102 4.31 MB/sec execute 374 sec latency 90.755 ms 2024-09-10T22:25:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:23 smithi154 ceph-mon[34495]: pgmap v1094: 129 pgs: 129 active+clean; 932 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 730 KiB/s wr, 48 op/s 2024-09-10T22:25:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:23 smithi171 ceph-mon[34138]: pgmap v1094: 129 pgs: 129 active+clean; 932 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 730 KiB/s wr, 48 op/s 2024-09-10T22:25:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:23 smithi090 ceph-mon[30350]: pgmap v1094: 129 pgs: 129 active+clean; 932 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 730 KiB/s wr, 48 op/s 2024-09-10T22:25:24.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 40245 4.31 MB/sec execute 375 sec latency 92.170 ms 2024-09-10T22:25:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:24 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:24 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:25.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:24 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:25.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 40376 4.33 MB/sec execute 376 sec latency 184.310 ms 2024-09-10T22:25:26.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:25 smithi154 ceph-mon[34495]: pgmap v1095: 129 pgs: 129 active+clean; 941 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-09-10T22:25:26.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:25 smithi171 ceph-mon[34138]: pgmap v1095: 129 pgs: 129 active+clean; 941 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-09-10T22:25:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:25 smithi090 ceph-mon[30350]: pgmap v1095: 129 pgs: 129 active+clean; 941 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.5 MiB/s wr, 82 op/s 2024-09-10T22:25:26.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 40506 4.33 MB/sec execute 377 sec latency 287.605 ms 2024-09-10T22:25:27.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 40625 4.34 MB/sec execute 378 sec latency 73.134 ms 2024-09-10T22:25:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:27 smithi154 ceph-mon[34495]: pgmap v1096: 129 pgs: 129 active+clean; 950 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 139 op/s 2024-09-10T22:25:28.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:27 smithi171 ceph-mon[34138]: pgmap v1096: 129 pgs: 129 active+clean; 950 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 139 op/s 2024-09-10T22:25:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:27 smithi090 ceph-mon[30350]: pgmap v1096: 129 pgs: 129 active+clean; 950 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 139 op/s 2024-09-10T22:25:28.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 40768 4.36 MB/sec execute 379 sec latency 61.138 ms 2024-09-10T22:25:29.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 40897 4.37 MB/sec execute 380 sec latency 71.055 ms 2024-09-10T22:25:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:29 smithi154 ceph-mon[34495]: pgmap v1097: 129 pgs: 129 active+clean; 956 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 160 op/s 2024-09-10T22:25:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:29 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:30.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:29 smithi171 ceph-mon[34138]: pgmap v1097: 129 pgs: 129 active+clean; 956 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 160 op/s 2024-09-10T22:25:30.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:29 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:30.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:29 smithi090 ceph-mon[30350]: pgmap v1097: 129 pgs: 129 active+clean; 956 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 160 op/s 2024-09-10T22:25:30.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:29 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:30.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41030 4.38 MB/sec execute 381 sec latency 90.548 ms 2024-09-10T22:25:31.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41175 4.39 MB/sec execute 382 sec latency 64.811 ms 2024-09-10T22:25:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:31 smithi154 ceph-mon[34495]: pgmap v1098: 129 pgs: 129 active+clean; 965 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 210 op/s 2024-09-10T22:25:32.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:31 smithi171 ceph-mon[34138]: pgmap v1098: 129 pgs: 129 active+clean; 965 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 210 op/s 2024-09-10T22:25:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:31 smithi090 ceph-mon[30350]: pgmap v1098: 129 pgs: 129 active+clean; 965 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 210 op/s 2024-09-10T22:25:32.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41309 4.39 MB/sec execute 383 sec latency 67.167 ms 2024-09-10T22:25:33.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41418 4.39 MB/sec execute 384 sec latency 73.020 ms 2024-09-10T22:25:34.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:33 smithi171 ceph-mon[34138]: pgmap v1099: 129 pgs: 129 active+clean; 969 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 209 op/s 2024-09-10T22:25:34.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:33 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:33 smithi090 ceph-mon[30350]: pgmap v1099: 129 pgs: 129 active+clean; 969 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 209 op/s 2024-09-10T22:25:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:33 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:33 smithi154 ceph-mon[34495]: pgmap v1099: 129 pgs: 129 active+clean; 969 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 209 op/s 2024-09-10T22:25:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:33 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:34.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41547 4.39 MB/sec execute 385 sec latency 70.158 ms 2024-09-10T22:25:35.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:34 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:34 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:35.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:34 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:35.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41634 4.39 MB/sec execute 386 sec latency 394.107 ms 2024-09-10T22:25:36.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:35 smithi171 ceph-mon[34138]: pgmap v1100: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 261 op/s 2024-09-10T22:25:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:35 smithi090 ceph-mon[30350]: pgmap v1100: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 261 op/s 2024-09-10T22:25:36.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:35 smithi154 ceph-mon[34495]: pgmap v1100: 129 pgs: 129 active+clean; 979 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 261 op/s 2024-09-10T22:25:36.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41756 4.40 MB/sec execute 387 sec latency 90.910 ms 2024-09-10T22:25:37.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41901 4.42 MB/sec execute 388 sec latency 64.468 ms 2024-09-10T22:25:38.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:37 smithi171 ceph-mon[34138]: pgmap v1101: 129 pgs: 129 active+clean; 984 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 261 op/s 2024-09-10T22:25:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:37 smithi090 ceph-mon[30350]: pgmap v1101: 129 pgs: 129 active+clean; 984 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 261 op/s 2024-09-10T22:25:38.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:37 smithi154 ceph-mon[34495]: pgmap v1101: 129 pgs: 129 active+clean; 984 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 261 op/s 2024-09-10T22:25:38.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42046 4.42 MB/sec execute 389 sec latency 70.714 ms 2024-09-10T22:25:39.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42216 4.43 MB/sec execute 390 sec latency 77.801 ms 2024-09-10T22:25:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:39 smithi171 ceph-mon[34138]: pgmap v1102: 129 pgs: 129 active+clean; 987 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 239 op/s 2024-09-10T22:25:40.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:39 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:39 smithi090 ceph-mon[30350]: pgmap v1102: 129 pgs: 129 active+clean; 987 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 239 op/s 2024-09-10T22:25:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:39 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:39 smithi154 ceph-mon[34495]: pgmap v1102: 129 pgs: 129 active+clean; 987 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 239 op/s 2024-09-10T22:25:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:39 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:40.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42288 4.44 MB/sec execute 391 sec latency 420.923 ms 2024-09-10T22:25:41.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42379 4.43 MB/sec execute 392 sec latency 438.896 ms 2024-09-10T22:25:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:41 smithi171 ceph-mon[34138]: pgmap v1103: 129 pgs: 129 active+clean; 998 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 294 op/s 2024-09-10T22:25:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:41 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:41 smithi090 ceph-mon[30350]: pgmap v1103: 129 pgs: 129 active+clean; 998 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 294 op/s 2024-09-10T22:25:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:41 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:41 smithi154 ceph-mon[34495]: pgmap v1103: 129 pgs: 129 active+clean; 998 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 294 op/s 2024-09-10T22:25:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:41 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:42.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42478 4.43 MB/sec execute 393 sec latency 87.631 ms 2024-09-10T22:25:43.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42612 4.44 MB/sec execute 394 sec latency 82.214 ms 2024-09-10T22:25:44.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:43 smithi171 ceph-mon[34138]: pgmap v1104: 129 pgs: 129 active+clean; 998 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 242 op/s 2024-09-10T22:25:44.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:43 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:43 smithi090 ceph-mon[30350]: pgmap v1104: 129 pgs: 129 active+clean; 998 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 242 op/s 2024-09-10T22:25:44.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:43 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:43 smithi154 ceph-mon[34495]: pgmap v1104: 129 pgs: 129 active+clean; 998 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 242 op/s 2024-09-10T22:25:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:43 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:44.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42731 4.44 MB/sec execute 395 sec latency 82.488 ms 2024-09-10T22:25:45.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42860 4.45 MB/sec execute 396 sec latency 83.181 ms 2024-09-10T22:25:46.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:45 smithi171 ceph-mon[34138]: pgmap v1105: 129 pgs: 129 active+clean; 997 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 261 op/s 2024-09-10T22:25:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:45 smithi090 ceph-mon[30350]: pgmap v1105: 129 pgs: 129 active+clean; 997 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 261 op/s 2024-09-10T22:25:46.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:45 smithi154 ceph-mon[34495]: pgmap v1105: 129 pgs: 129 active+clean; 997 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 261 op/s 2024-09-10T22:25:46.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42996 4.46 MB/sec execute 397 sec latency 65.201 ms 2024-09-10T22:25:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:47 smithi090 ceph-mon[30350]: pgmap v1106: 129 pgs: 129 active+clean; 1002 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 262 op/s 2024-09-10T22:25:47.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:47 smithi154 ceph-mon[34495]: pgmap v1106: 129 pgs: 129 active+clean; 1002 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 262 op/s 2024-09-10T22:25:47.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:47 smithi171 ceph-mon[34138]: pgmap v1106: 129 pgs: 129 active+clean; 1002 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 262 op/s 2024-09-10T22:25:47.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 43125 4.46 MB/sec execute 398 sec latency 64.089 ms 2024-09-10T22:25:48.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 43236 4.46 MB/sec execute 399 sec latency 74.852 ms 2024-09-10T22:25:49.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:49 smithi090 ceph-mon[30350]: pgmap v1107: 129 pgs: 129 active+clean; 1006 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-09-10T22:25:49.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:49 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:49 smithi154 ceph-mon[34495]: pgmap v1107: 129 pgs: 129 active+clean; 1006 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-09-10T22:25:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:49 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:49.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:49 smithi171 ceph-mon[34138]: pgmap v1107: 129 pgs: 129 active+clean; 1006 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 256 op/s 2024-09-10T22:25:49.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:49 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:49.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 43366 4.47 MB/sec execute 400 sec latency 86.464 ms 2024-09-10T22:25:50.903 INFO:tasks.workunit.client.0.smithi090.stdout: 10 43503 4.48 MB/sec execute 401 sec latency 136.378 ms 2024-09-10T22:25:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:51 smithi090 ceph-mon[30350]: pgmap v1108: 129 pgs: 129 active+clean; 1012 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-09-10T22:25:51.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:51 smithi154 ceph-mon[34495]: pgmap v1108: 129 pgs: 129 active+clean; 1012 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-09-10T22:25:51.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:51 smithi171 ceph-mon[34138]: pgmap v1108: 129 pgs: 129 active+clean; 1012 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-09-10T22:25:51.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 43655 4.48 MB/sec execute 402 sec latency 81.678 ms 2024-09-10T22:25:52.903 INFO:tasks.workunit.client.0.smithi090.stdout: 10 43796 4.49 MB/sec execute 403 sec latency 104.901 ms 2024-09-10T22:25:53.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:53 smithi090 ceph-mon[30350]: pgmap v1109: 129 pgs: 129 active+clean; 1012 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-09-10T22:25:53.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:53 smithi154 ceph-mon[34495]: pgmap v1109: 129 pgs: 129 active+clean; 1012 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-09-10T22:25:53.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:53 smithi171 ceph-mon[34138]: pgmap v1109: 129 pgs: 129 active+clean; 1012 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 197 op/s 2024-09-10T22:25:53.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 43909 4.49 MB/sec execute 404 sec latency 113.226 ms 2024-09-10T22:25:54.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44064 4.51 MB/sec execute 405 sec latency 83.987 ms 2024-09-10T22:25:55.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:55 smithi090 ceph-mon[30350]: pgmap v1110: 129 pgs: 129 active+clean; 1018 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 264 op/s 2024-09-10T22:25:55.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:55 smithi154 ceph-mon[34495]: pgmap v1110: 129 pgs: 129 active+clean; 1018 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 264 op/s 2024-09-10T22:25:55.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:55 smithi171 ceph-mon[34138]: pgmap v1110: 129 pgs: 129 active+clean; 1018 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 264 op/s 2024-09-10T22:25:55.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44191 4.52 MB/sec execute 406 sec latency 74.991 ms 2024-09-10T22:25:56.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44308 4.52 MB/sec execute 407 sec latency 93.264 ms 2024-09-10T22:25:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:57 smithi090 ceph-mon[30350]: pgmap v1111: 129 pgs: 129 active+clean; 1022 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 278 op/s 2024-09-10T22:25:57.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:57 smithi154 ceph-mon[34495]: pgmap v1111: 129 pgs: 129 active+clean; 1022 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 278 op/s 2024-09-10T22:25:57.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:57 smithi171 ceph-mon[34138]: pgmap v1111: 129 pgs: 129 active+clean; 1022 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 278 op/s 2024-09-10T22:25:57.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44447 4.53 MB/sec execute 408 sec latency 82.349 ms 2024-09-10T22:25:58.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44604 4.54 MB/sec execute 409 sec latency 62.993 ms 2024-09-10T22:25:59.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:59 smithi090 ceph-mon[30350]: pgmap v1112: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 243 op/s 2024-09-10T22:25:59.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:25:59 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:59.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:59 smithi154 ceph-mon[34495]: pgmap v1112: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 243 op/s 2024-09-10T22:25:59.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:25:59 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:59.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:59 smithi171 ceph-mon[34138]: pgmap v1112: 129 pgs: 129 active+clean; 1.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 243 op/s 2024-09-10T22:25:59.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:25:59 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:25:59.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44754 4.55 MB/sec execute 410 sec latency 116.304 ms 2024-09-10T22:26:00.904 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44809 4.54 MB/sec execute 411 sec latency 585.501 ms 2024-09-10T22:26:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:01 smithi090 ceph-mon[30350]: pgmap v1113: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 258 op/s 2024-09-10T22:26:01.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:01 smithi154 ceph-mon[34495]: pgmap v1113: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 258 op/s 2024-09-10T22:26:01.500 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:01 smithi171 ceph-mon[34138]: pgmap v1113: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 258 op/s 2024-09-10T22:26:01.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44896 4.53 MB/sec execute 412 sec latency 601.568 ms 2024-09-10T22:26:02.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45023 4.55 MB/sec execute 413 sec latency 73.588 ms 2024-09-10T22:26:03.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:03 smithi090 ceph-mon[30350]: pgmap v1114: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 233 op/s 2024-09-10T22:26:03.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:03 smithi154 ceph-mon[34495]: pgmap v1114: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 233 op/s 2024-09-10T22:26:03.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:03 smithi171 ceph-mon[34138]: pgmap v1114: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.6 MiB/s wr, 233 op/s 2024-09-10T22:26:03.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45152 4.56 MB/sec execute 414 sec latency 92.457 ms 2024-09-10T22:26:04.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:04 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:26:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:04 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:26:04.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:04 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:26:04.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45247 4.56 MB/sec execute 415 sec latency 151.234 ms 2024-09-10T22:26:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:05 smithi154 ceph-mon[34495]: pgmap v1115: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 278 op/s 2024-09-10T22:26:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:05 smithi154 ceph-mon[34495]: osdmap e123: 12 total, 12 up, 12 in 2024-09-10T22:26:05.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:05 smithi171 ceph-mon[34138]: pgmap v1115: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 278 op/s 2024-09-10T22:26:05.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:05 smithi171 ceph-mon[34138]: osdmap e123: 12 total, 12 up, 12 in 2024-09-10T22:26:05.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:05 smithi090 ceph-mon[30350]: pgmap v1115: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 278 op/s 2024-09-10T22:26:05.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:05 smithi090 ceph-mon[30350]: osdmap e123: 12 total, 12 up, 12 in 2024-09-10T22:26:05.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45397 4.58 MB/sec execute 416 sec latency 164.748 ms 2024-09-10T22:26:06.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45483 4.58 MB/sec execute 417 sec latency 504.091 ms 2024-09-10T22:26:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:07 smithi154 ceph-mon[34495]: pgmap v1117: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 264 op/s 2024-09-10T22:26:07.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:07 smithi171 ceph-mon[34138]: pgmap v1117: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 264 op/s 2024-09-10T22:26:07.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:07 smithi090 ceph-mon[30350]: pgmap v1117: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.1 MiB/s wr, 264 op/s 2024-09-10T22:26:07.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45656 4.60 MB/sec execute 418 sec latency 59.062 ms 2024-09-10T22:26:08.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45737 4.59 MB/sec execute 419 sec latency 78.952 ms 2024-09-10T22:26:09.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:09 smithi154 ceph-mon[34495]: pgmap v1118: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 254 op/s 2024-09-10T22:26:09.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:09 smithi171 ceph-mon[34138]: pgmap v1118: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 254 op/s 2024-09-10T22:26:09.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:09 smithi090 ceph-mon[30350]: pgmap v1118: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 254 op/s 2024-09-10T22:26:09.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45829 4.58 MB/sec execute 420 sec latency 69.176 ms 2024-09-10T22:26:10.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45972 4.59 MB/sec execute 421 sec latency 114.464 ms 2024-09-10T22:26:11.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46002 4.58 MB/sec execute 422 sec latency 781.263 ms 2024-09-10T22:26:12.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:11 smithi090 ceph-mon[30350]: pgmap v1119: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 243 op/s 2024-09-10T22:26:12.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:11 smithi090 ceph-mon[30350]: osdmap e124: 12 total, 12 up, 12 in 2024-09-10T22:26:12.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:11 smithi154 ceph-mon[34495]: pgmap v1119: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 243 op/s 2024-09-10T22:26:12.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:11 smithi154 ceph-mon[34495]: osdmap e124: 12 total, 12 up, 12 in 2024-09-10T22:26:12.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:11 smithi171 ceph-mon[34138]: pgmap v1119: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 243 op/s 2024-09-10T22:26:12.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:11 smithi171 ceph-mon[34138]: osdmap e124: 12 total, 12 up, 12 in 2024-09-10T22:26:12.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46118 4.58 MB/sec execute 423 sec latency 83.344 ms 2024-09-10T22:26:13.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46237 4.58 MB/sec execute 424 sec latency 73.890 ms 2024-09-10T22:26:14.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:13 smithi090 ceph-mon[30350]: pgmap v1121: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 214 op/s 2024-09-10T22:26:14.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:13 smithi154 ceph-mon[34495]: pgmap v1121: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 214 op/s 2024-09-10T22:26:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:13 smithi171 ceph-mon[34138]: pgmap v1121: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.9 MiB/s wr, 214 op/s 2024-09-10T22:26:14.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46348 4.58 MB/sec execute 425 sec latency 71.901 ms 2024-09-10T22:26:15.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46478 4.59 MB/sec execute 426 sec latency 73.426 ms 2024-09-10T22:26:16.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:15 smithi090 ceph-mon[30350]: pgmap v1122: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 246 op/s 2024-09-10T22:26:16.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:15 smithi154 ceph-mon[34495]: pgmap v1122: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 246 op/s 2024-09-10T22:26:16.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:15 smithi171 ceph-mon[34138]: pgmap v1122: 129 pgs: 129 active+clean; 1.0 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 246 op/s 2024-09-10T22:26:16.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46557 4.59 MB/sec execute 427 sec latency 427.115 ms 2024-09-10T22:26:17.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46708 4.61 MB/sec execute 428 sec latency 81.140 ms 2024-09-10T22:26:18.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:17 smithi090 ceph-mon[30350]: pgmap v1123: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 215 op/s 2024-09-10T22:26:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:17 smithi154 ceph-mon[34495]: pgmap v1123: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 215 op/s 2024-09-10T22:26:18.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:17 smithi171 ceph-mon[34138]: pgmap v1123: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 215 op/s 2024-09-10T22:26:18.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46849 4.62 MB/sec execute 429 sec latency 75.353 ms 2024-09-10T22:26:19.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46975 4.61 MB/sec execute 430 sec latency 82.019 ms 2024-09-10T22:26:20.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:19 smithi090 ceph-mon[30350]: pgmap v1124: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 225 op/s 2024-09-10T22:26:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:19 smithi154 ceph-mon[34495]: pgmap v1124: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 225 op/s 2024-09-10T22:26:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:19 smithi171 ceph-mon[34138]: pgmap v1124: 129 pgs: 129 active+clean; 1.0 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 225 op/s 2024-09-10T22:26:20.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47100 4.61 MB/sec execute 431 sec latency 93.849 ms 2024-09-10T22:26:21.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47189 4.62 MB/sec execute 432 sec latency 427.866 ms 2024-09-10T22:26:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:21 smithi090 ceph-mon[30350]: pgmap v1125: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 254 op/s 2024-09-10T22:26:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:21 smithi154 ceph-mon[34495]: pgmap v1125: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 254 op/s 2024-09-10T22:26:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:21 smithi171 ceph-mon[34138]: pgmap v1125: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 254 op/s 2024-09-10T22:26:22.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47305 4.61 MB/sec execute 433 sec latency 98.584 ms 2024-09-10T22:26:23.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:22 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:26:23.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:22 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:26:23.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:22 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:26:23.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47429 4.62 MB/sec execute 434 sec latency 83.219 ms 2024-09-10T22:26:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:23 smithi154 ceph-mon[34495]: pgmap v1126: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2024-09-10T22:26:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:23 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:26:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:23 smithi171 ceph-mon[34138]: pgmap v1126: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2024-09-10T22:26:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:23 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:26:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:23 smithi090 ceph-mon[30350]: pgmap v1126: 129 pgs: 129 active+clean; 1.1 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 237 op/s 2024-09-10T22:26:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:23 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:26:24.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47566 4.63 MB/sec execute 435 sec latency 74.195 ms 2024-09-10T22:26:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:26:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:26:25.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:24 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:26:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:26:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:26:25.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:24 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:26:25.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:26:25.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:26:25.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:24 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:26:25.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47682 4.64 MB/sec execute 436 sec latency 70.907 ms 2024-09-10T22:26:26.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:25 smithi154 ceph-mon[34495]: pgmap v1127: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 261 op/s 2024-09-10T22:26:26.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:25 smithi171 ceph-mon[34138]: pgmap v1127: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 261 op/s 2024-09-10T22:26:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:25 smithi090 ceph-mon[30350]: pgmap v1127: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 261 op/s 2024-09-10T22:26:26.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47784 4.63 MB/sec execute 437 sec latency 81.579 ms 2024-09-10T22:26:27.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47921 4.64 MB/sec execute 438 sec latency 94.766 ms 2024-09-10T22:26:28.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:27 smithi154 ceph-mon[34495]: pgmap v1128: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-09-10T22:26:28.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:27 smithi171 ceph-mon[34138]: pgmap v1128: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-09-10T22:26:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:27 smithi090 ceph-mon[30350]: pgmap v1128: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-09-10T22:26:28.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 48059 4.64 MB/sec execute 439 sec latency 97.917 ms 2024-09-10T22:26:29.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 48233 4.65 MB/sec execute 440 sec latency 66.889 ms 2024-09-10T22:26:30.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:29 smithi154 ceph-mon[34495]: pgmap v1129: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2024-09-10T22:26:30.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:29 smithi171 ceph-mon[34138]: pgmap v1129: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2024-09-10T22:26:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:29 smithi090 ceph-mon[30350]: pgmap v1129: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2024-09-10T22:26:30.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 48376 4.66 MB/sec execute 441 sec latency 93.722 ms 2024-09-10T22:26:31.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 48485 4.66 MB/sec execute 442 sec latency 75.712 ms 2024-09-10T22:26:32.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:31 smithi154 ceph-mon[34495]: pgmap v1130: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 260 op/s 2024-09-10T22:26:32.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:31 smithi171 ceph-mon[34138]: pgmap v1130: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 260 op/s 2024-09-10T22:26:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:31 smithi090 ceph-mon[30350]: pgmap v1130: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 260 op/s 2024-09-10T22:26:32.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 48598 4.67 MB/sec execute 443 sec latency 68.594 ms 2024-09-10T22:26:33.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 48762 4.71 MB/sec execute 444 sec latency 92.620 ms 2024-09-10T22:26:34.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:33 smithi154 ceph-mon[34495]: pgmap v1131: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 215 op/s 2024-09-10T22:26:34.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:33 smithi171 ceph-mon[34138]: pgmap v1131: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 215 op/s 2024-09-10T22:26:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:33 smithi090 ceph-mon[30350]: pgmap v1131: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.0 MiB/s wr, 215 op/s 2024-09-10T22:26:34.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 48875 4.71 MB/sec execute 445 sec latency 296.349 ms 2024-09-10T22:26:35.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49051 4.72 MB/sec execute 446 sec latency 152.134 ms 2024-09-10T22:26:36.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:35 smithi154 ceph-mon[34495]: pgmap v1132: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 258 op/s 2024-09-10T22:26:36.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:35 smithi171 ceph-mon[34138]: pgmap v1132: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 258 op/s 2024-09-10T22:26:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:35 smithi090 ceph-mon[30350]: pgmap v1132: 129 pgs: 129 active+clean; 1.1 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 258 op/s 2024-09-10T22:26:36.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49147 4.72 MB/sec execute 447 sec latency 300.308 ms 2024-09-10T22:26:37.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49233 4.72 MB/sec execute 448 sec latency 116.024 ms 2024-09-10T22:26:38.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:37 smithi154 ceph-mon[34495]: pgmap v1133: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-09-10T22:26:38.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:37 smithi171 ceph-mon[34138]: pgmap v1133: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-09-10T22:26:38.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:37 smithi090 ceph-mon[30350]: pgmap v1133: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-09-10T22:26:38.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49366 4.73 MB/sec execute 449 sec latency 57.721 ms 2024-09-10T22:26:39.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49480 4.72 MB/sec execute 450 sec latency 70.801 ms 2024-09-10T22:26:40.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:39 smithi154 ceph-mon[34495]: pgmap v1134: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 206 op/s 2024-09-10T22:26:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:39 smithi171 ceph-mon[34138]: pgmap v1134: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 206 op/s 2024-09-10T22:26:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:39 smithi090 ceph-mon[30350]: pgmap v1134: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 206 op/s 2024-09-10T22:26:40.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49594 4.72 MB/sec execute 451 sec latency 87.989 ms 2024-09-10T22:26:41.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49703 4.72 MB/sec execute 452 sec latency 66.435 ms 2024-09-10T22:26:42.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:41 smithi154 ceph-mon[34495]: pgmap v1135: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2024-09-10T22:26:42.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:41 smithi171 ceph-mon[34138]: pgmap v1135: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2024-09-10T22:26:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:41 smithi090 ceph-mon[30350]: pgmap v1135: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2024-09-10T22:26:42.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49807 4.71 MB/sec execute 453 sec latency 60.570 ms 2024-09-10T22:26:43.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49940 4.72 MB/sec execute 454 sec latency 82.817 ms 2024-09-10T22:26:44.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:43 smithi154 ceph-mon[34495]: pgmap v1136: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 205 op/s 2024-09-10T22:26:44.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:43 smithi171 ceph-mon[34138]: pgmap v1136: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 205 op/s 2024-09-10T22:26:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:43 smithi090 ceph-mon[30350]: pgmap v1136: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 205 op/s 2024-09-10T22:26:44.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50086 4.74 MB/sec execute 455 sec latency 260.446 ms 2024-09-10T22:26:45.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50233 4.74 MB/sec execute 456 sec latency 291.371 ms 2024-09-10T22:26:46.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:45 smithi171 ceph-mon[34138]: pgmap v1137: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 252 op/s 2024-09-10T22:26:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:45 smithi090 ceph-mon[30350]: pgmap v1137: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 252 op/s 2024-09-10T22:26:46.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:45 smithi154 ceph-mon[34495]: pgmap v1137: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 252 op/s 2024-09-10T22:26:46.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50378 4.74 MB/sec execute 457 sec latency 99.780 ms 2024-09-10T22:26:47.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50480 4.74 MB/sec execute 458 sec latency 76.677 ms 2024-09-10T22:26:48.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:47 smithi171 ceph-mon[34138]: pgmap v1138: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 245 op/s 2024-09-10T22:26:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:47 smithi090 ceph-mon[30350]: pgmap v1138: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 245 op/s 2024-09-10T22:26:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:47 smithi154 ceph-mon[34495]: pgmap v1138: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.4 MiB/s wr, 245 op/s 2024-09-10T22:26:48.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50609 4.75 MB/sec execute 459 sec latency 99.950 ms 2024-09-10T22:26:49.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50723 4.76 MB/sec execute 460 sec latency 215.408 ms 2024-09-10T22:26:50.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:49 smithi171 ceph-mon[34138]: pgmap v1139: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-09-10T22:26:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:49 smithi090 ceph-mon[30350]: pgmap v1139: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-09-10T22:26:50.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:49 smithi154 ceph-mon[34495]: pgmap v1139: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 252 op/s 2024-09-10T22:26:50.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50871 4.78 MB/sec execute 461 sec latency 72.538 ms 2024-09-10T22:26:51.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50964 4.77 MB/sec execute 462 sec latency 473.168 ms 2024-09-10T22:26:52.220 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:51 smithi154 ceph-mon[34495]: pgmap v1140: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.4 MiB/s wr, 291 op/s 2024-09-10T22:26:52.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:51 smithi171 ceph-mon[34138]: pgmap v1140: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.4 MiB/s wr, 291 op/s 2024-09-10T22:26:52.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:51 smithi090 ceph-mon[30350]: pgmap v1140: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.4 MiB/s wr, 291 op/s 2024-09-10T22:26:52.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51084 4.76 MB/sec execute 463 sec latency 308.533 ms 2024-09-10T22:26:53.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51206 4.76 MB/sec execute 464 sec latency 213.889 ms 2024-09-10T22:26:54.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:53 smithi171 ceph-mon[34138]: pgmap v1141: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-10T22:26:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:53 smithi090 ceph-mon[30350]: pgmap v1141: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-10T22:26:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:53 smithi154 ceph-mon[34495]: pgmap v1141: 129 pgs: 129 active+clean; 1.1 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.5 MiB/s wr, 231 op/s 2024-09-10T22:26:54.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51324 4.75 MB/sec execute 465 sec latency 146.858 ms 2024-09-10T22:26:55.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51417 4.75 MB/sec execute 466 sec latency 889.865 ms 2024-09-10T22:26:56.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:55 smithi171 ceph-mon[34138]: pgmap v1142: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2024-09-10T22:26:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:55 smithi090 ceph-mon[30350]: pgmap v1142: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2024-09-10T22:26:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:55 smithi154 ceph-mon[34495]: pgmap v1142: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 280 op/s 2024-09-10T22:26:56.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51568 4.75 MB/sec execute 467 sec latency 74.516 ms 2024-09-10T22:26:57.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51725 4.76 MB/sec execute 468 sec latency 74.036 ms 2024-09-10T22:26:58.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:57 smithi171 ceph-mon[34138]: pgmap v1143: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 263 op/s 2024-09-10T22:26:58.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:57 smithi090 ceph-mon[30350]: pgmap v1143: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 263 op/s 2024-09-10T22:26:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:57 smithi154 ceph-mon[34495]: pgmap v1143: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 263 op/s 2024-09-10T22:26:58.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51848 4.77 MB/sec execute 469 sec latency 73.612 ms 2024-09-10T22:26:59.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51996 4.78 MB/sec execute 470 sec latency 229.075 ms 2024-09-10T22:27:00.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:26:59 smithi171 ceph-mon[34138]: pgmap v1144: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-09-10T22:27:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:26:59 smithi090 ceph-mon[30350]: pgmap v1144: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-09-10T22:27:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:26:59 smithi154 ceph-mon[34495]: pgmap v1144: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 243 op/s 2024-09-10T22:27:00.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52119 4.80 MB/sec execute 471 sec latency 100.808 ms 2024-09-10T22:27:01.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52244 4.81 MB/sec execute 472 sec latency 441.144 ms 2024-09-10T22:27:02.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:01 smithi171 ceph-mon[34138]: pgmap v1145: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 248 op/s 2024-09-10T22:27:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:01 smithi090 ceph-mon[30350]: pgmap v1145: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 248 op/s 2024-09-10T22:27:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:01 smithi154 ceph-mon[34495]: pgmap v1145: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 248 op/s 2024-09-10T22:27:02.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52399 4.82 MB/sec execute 473 sec latency 452.387 ms 2024-09-10T22:27:03.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52551 4.83 MB/sec execute 474 sec latency 78.996 ms 2024-09-10T22:27:04.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:03 smithi171 ceph-mon[34138]: pgmap v1146: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 214 op/s 2024-09-10T22:27:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:03 smithi090 ceph-mon[30350]: pgmap v1146: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 214 op/s 2024-09-10T22:27:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:03 smithi154 ceph-mon[34495]: pgmap v1146: 129 pgs: 129 active+clean; 1.1 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.8 MiB/s wr, 214 op/s 2024-09-10T22:27:04.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52652 4.84 MB/sec execute 475 sec latency 383.754 ms 2024-09-10T22:27:05.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:04 smithi171 ceph-mon[34138]: osdmap e125: 12 total, 12 up, 12 in 2024-09-10T22:27:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:04 smithi090 ceph-mon[30350]: osdmap e125: 12 total, 12 up, 12 in 2024-09-10T22:27:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:04 smithi154 ceph-mon[34495]: osdmap e125: 12 total, 12 up, 12 in 2024-09-10T22:27:05.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52733 4.83 MB/sec execute 476 sec latency 69.875 ms 2024-09-10T22:27:06.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:05 smithi171 ceph-mon[34138]: pgmap v1148: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 263 op/s 2024-09-10T22:27:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:05 smithi090 ceph-mon[30350]: pgmap v1148: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 263 op/s 2024-09-10T22:27:06.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:05 smithi154 ceph-mon[34495]: pgmap v1148: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 263 op/s 2024-09-10T22:27:06.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52822 4.83 MB/sec execute 477 sec latency 268.354 ms 2024-09-10T22:27:07.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52903 4.83 MB/sec execute 478 sec latency 643.387 ms 2024-09-10T22:27:08.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:07 smithi171 ceph-mon[34138]: pgmap v1149: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.2 MiB/s wr, 253 op/s 2024-09-10T22:27:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:07 smithi090 ceph-mon[30350]: pgmap v1149: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.2 MiB/s wr, 253 op/s 2024-09-10T22:27:08.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:07 smithi154 ceph-mon[34495]: pgmap v1149: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.2 MiB/s wr, 253 op/s 2024-09-10T22:27:08.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53022 4.84 MB/sec execute 479 sec latency 82.634 ms 2024-09-10T22:27:09.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53141 4.84 MB/sec execute 480 sec latency 264.314 ms 2024-09-10T22:27:10.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:09 smithi171 ceph-mon[34138]: pgmap v1150: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 250 op/s 2024-09-10T22:27:10.339 INFO:tasks.quiescer.fs.[cephfs]:Total duration (37.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T22:27:10.339 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-10T22:27:10.340 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:27:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:09 smithi090 ceph-mon[30350]: pgmap v1150: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 250 op/s 2024-09-10T22:27:10.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:09 smithi154 ceph-mon[34495]: pgmap v1150: 129 pgs: 129 active+clean; 1.1 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.4 MiB/s wr, 250 op/s 2024-09-10T22:27:10.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53257 4.84 MB/sec execute 481 sec latency 326.687 ms 2024-09-10T22:27:11.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53390 4.84 MB/sec execute 482 sec latency 384.105 ms 2024-09-10T22:27:12.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:11 smithi090 ceph-mon[30350]: pgmap v1151: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-09-10T22:27:12.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:11 smithi171 ceph-mon[34138]: pgmap v1151: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-09-10T22:27:12.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:11 smithi154 ceph-mon[34495]: pgmap v1151: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 242 op/s 2024-09-10T22:27:12.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53436 4.84 MB/sec execute 483 sec latency 511.539 ms 2024-09-10T22:27:13.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53523 4.84 MB/sec execute 484 sec latency 466.429 ms 2024-09-10T22:27:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:13 smithi171 ceph-mon[34138]: pgmap v1152: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 249 op/s 2024-09-10T22:27:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:13 smithi090 ceph-mon[30350]: pgmap v1152: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 249 op/s 2024-09-10T22:27:14.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:13 smithi154 ceph-mon[34495]: pgmap v1152: 129 pgs: 129 active+clean; 1.2 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 249 op/s 2024-09-10T22:27:14.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53603 4.83 MB/sec execute 485 sec latency 240.739 ms 2024-09-10T22:27:15.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53696 4.83 MB/sec execute 486 sec latency 363.287 ms 2024-09-10T22:27:16.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:15 smithi171 ceph-mon[34138]: pgmap v1153: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 185 op/s 2024-09-10T22:27:16.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:15 smithi171 ceph-mon[34138]: osdmap e126: 12 total, 12 up, 12 in 2024-09-10T22:27:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:15 smithi090 ceph-mon[30350]: pgmap v1153: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 185 op/s 2024-09-10T22:27:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:15 smithi090 ceph-mon[30350]: osdmap e126: 12 total, 12 up, 12 in 2024-09-10T22:27:16.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:15 smithi154 ceph-mon[34495]: pgmap v1153: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 185 op/s 2024-09-10T22:27:16.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:15 smithi154 ceph-mon[34495]: osdmap e126: 12 total, 12 up, 12 in 2024-09-10T22:27:16.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53775 4.83 MB/sec execute 487 sec latency 442.852 ms 2024-09-10T22:27:17.287 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:27:17.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53900 4.84 MB/sec execute 488 sec latency 92.332 ms 2024-09-10T22:27:18.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:17 smithi090 ceph-mon[30350]: pgmap v1155: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 227 op/s 2024-09-10T22:27:18.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:17 smithi171 ceph-mon[34138]: pgmap v1155: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 227 op/s 2024-09-10T22:27:18.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:17 smithi154 ceph-mon[34495]: pgmap v1155: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 227 op/s 2024-09-10T22:27:18.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54027 4.85 MB/sec execute 489 sec latency 109.060 ms 2024-09-10T22:27:19.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54076 4.84 MB/sec execute 490 sec latency 611.285 ms 2024-09-10T22:27:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:19 smithi171 ceph-mon[34138]: pgmap v1156: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 210 op/s 2024-09-10T22:27:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:19 smithi090 ceph-mon[30350]: pgmap v1156: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 210 op/s 2024-09-10T22:27:20.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:19 smithi154 ceph-mon[34495]: pgmap v1156: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 210 op/s 2024-09-10T22:27:20.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.483+0000 7fab56200640 1 -- 172.21.15.90:0/218575613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab500776d0 msgr2=0x7fab50077ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:20.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.483+0000 7fab56200640 1 --2- 172.21.15.90:0/218575613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab500776d0 0x7fab50077ab0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab44009860 tx=0x7fab4402a340 comp rx=0 tx=0).stop 2024-09-10T22:27:20.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.484+0000 7fab56200640 1 -- 172.21.15.90:0/218575613 shutdown_connections 2024-09-10T22:27:20.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.484+0000 7fab56200640 1 --2- 172.21.15.90:0/218575613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab50113800 0x7fab50115bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.484+0000 7fab56200640 1 --2- 172.21.15.90:0/218575613 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab50077ff0 0x7fab501132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.484+0000 7fab56200640 1 --2- 172.21.15.90:0/218575613 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab500776d0 0x7fab50077ab0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.484+0000 7fab56200640 1 -- 172.21.15.90:0/218575613 >> 172.21.15.90:0/218575613 conn(0x7fab50100840 msgr2=0x7fab50102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:20.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.485+0000 7fab56200640 1 -- 172.21.15.90:0/218575613 shutdown_connections 2024-09-10T22:27:20.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.485+0000 7fab56200640 1 -- 172.21.15.90:0/218575613 wait complete. 2024-09-10T22:27:20.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.486+0000 7fab56200640 1 Processor -- start 2024-09-10T22:27:20.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.487+0000 7fab56200640 1 -- start start 2024-09-10T22:27:20.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.487+0000 7fab56200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab500776d0 0x7fab5019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:20.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.487+0000 7fab56200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab50077ff0 0x7fab5019fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:20.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.487+0000 7fab56200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab50113800 0x7fab501a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:20.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.488+0000 7fab56200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fab501190f0 con 0x7fab50113800 2024-09-10T22:27:20.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.488+0000 7fab56200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fab50118f70 con 0x7fab500776d0 2024-09-10T22:27:20.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.488+0000 7fab56200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fab50119270 con 0x7fab50077ff0 2024-09-10T22:27:20.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.488+0000 7fab4fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab50077ff0 0x7fab5019fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:20.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.488+0000 7fab4fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab50077ff0 0x7fab5019fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:54432/0 (socket says 172.21.15.90:54432) 2024-09-10T22:27:20.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.488+0000 7fab4fe00640 1 -- 172.21.15.90:0/4190191485 learned_addr learned my addr 172.21.15.90:0/4190191485 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:20.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.488+0000 7fab55800640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab50113800 0x7fab501a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:20.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.488+0000 7fab54e00640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab500776d0 0x7fab5019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:20.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.489+0000 7fab4fe00640 1 -- 172.21.15.90:0/4190191485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab500776d0 msgr2=0x7fab5019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:20.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.489+0000 7fab4fe00640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab500776d0 0x7fab5019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.489+0000 7fab4fe00640 1 -- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab50113800 msgr2=0x7fab501a4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:20.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.489+0000 7fab4fe00640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab50113800 0x7fab501a4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.489+0000 7fab4fe00640 1 -- 172.21.15.90:0/4190191485 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab501a5680 con 0x7fab50077ff0 2024-09-10T22:27:20.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.489+0000 7fab4fe00640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab50077ff0 0x7fab5019fb10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00e9f0 tx=0x7fab3c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:20.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.489+0000 7fab4d600640 1 -- 172.21.15.90:0/4190191485 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00cd80 con 0x7fab50077ff0 2024-09-10T22:27:20.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.490+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab501a5970 con 0x7fab50077ff0 2024-09-10T22:27:20.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.490+0000 7fab4d600640 1 -- 172.21.15.90:0/4190191485 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fab3c0108c0 con 0x7fab50077ff0 2024-09-10T22:27:20.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.490+0000 7fab4d600640 1 -- 172.21.15.90:0/4190191485 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c0187c0 con 0x7fab50077ff0 2024-09-10T22:27:20.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.490+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab501ac200 con 0x7fab50077ff0 2024-09-10T22:27:20.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.493+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab50078be0 con 0x7fab50077ff0 2024-09-10T22:27:20.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.494+0000 7fab4d600640 1 -- 172.21.15.90:0/4190191485 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fab3c010430 con 0x7fab50077ff0 2024-09-10T22:27:20.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.496+0000 7fab4d600640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fab380b55d0 0x7fab380b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:20.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.496+0000 7fab4d600640 1 -- 172.21.15.90:0/4190191485 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7fab3c0e67f0 con 0x7fab50077ff0 2024-09-10T22:27:20.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.496+0000 7fab54e00640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fab380b55d0 0x7fab380b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:20.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.498+0000 7fab54e00640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fab380b55d0 0x7fab380b7a90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab4402a930 tx=0x7fab44009580 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:20.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.500+0000 7fab4d600640 1 -- 172.21.15.90:0/4190191485 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fab3c0acf40 con 0x7fab50077ff0 2024-09-10T22:27:20.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54197 4.84 MB/sec execute 491 sec latency 101.777 ms 2024-09-10T22:27:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.923+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab501ac3d0 con 0x7fab50077ff0 2024-09-10T22:27:20.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.924+0000 7fab4d600640 1 -- 172.21.15.90:0/4190191485 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+11999 (secure 0 0 0) 0x7fab3c0ac840 con 0x7fab50077ff0 2024-09-10T22:27:20.927 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:27:20.927 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":91,"btime":"2024-09-10T22:26:22:850763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":91,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:26:22.770344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:27:20.927 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 91 2024-09-10T22:27:20.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.927+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fab380b55d0 msgr2=0x7fab380b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:20.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.927+0000 7fab56200640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fab380b55d0 0x7fab380b7a90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fab4402a930 tx=0x7fab44009580 comp rx=0 tx=0).stop 2024-09-10T22:27:20.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.928+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab50077ff0 msgr2=0x7fab5019fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:20.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.928+0000 7fab56200640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab50077ff0 0x7fab5019fb10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00e9f0 tx=0x7fab3c00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.928+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 shutdown_connections 2024-09-10T22:27:20.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.928+0000 7fab56200640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fab380b55d0 0x7fab380b7a90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.928+0000 7fab56200640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab50113800 0x7fab501a4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.928+0000 7fab56200640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fab50077ff0 0x7fab5019fb10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.928+0000 7fab56200640 1 --2- 172.21.15.90:0/4190191485 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fab500776d0 0x7fab5019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:20.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.928+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 >> 172.21.15.90:0/4190191485 conn(0x7fab50100840 msgr2=0x7fab50102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:20.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.928+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 shutdown_connections 2024-09-10T22:27:20.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:20.929+0000 7fab56200640 1 -- 172.21.15.90:0/4190191485 wait complete. 2024-09-10T22:27:21.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:21 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4190191485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:21.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:21 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4190191485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:21.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:21 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4190191485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:21.905 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-10T22:27:21.906 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-10T22:27:21.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54315 4.84 MB/sec execute 492 sec latency 79.475 ms 2024-09-10T22:27:22.319 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:22 smithi154 ceph-mon[34495]: pgmap v1157: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 227 op/s 2024-09-10T22:27:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:22 smithi090 ceph-mon[30350]: pgmap v1157: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 227 op/s 2024-09-10T22:27:22.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:22 smithi171 ceph-mon[34138]: pgmap v1157: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.3 MiB/s wr, 227 op/s 2024-09-10T22:27:22.606 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:27:22.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54415 4.84 MB/sec execute 493 sec latency 78.502 ms 2024-09-10T22:27:23.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54467 4.83 MB/sec execute 494 sec latency 536.346 ms 2024-09-10T22:27:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:24 smithi090 ceph-mon[30350]: pgmap v1158: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-10T22:27:24.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:24 smithi154 ceph-mon[34495]: pgmap v1158: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-10T22:27:24.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:24 smithi171 ceph-mon[34138]: pgmap v1158: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.1 MiB/s wr, 227 op/s 2024-09-10T22:27:24.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54559 4.83 MB/sec execute 495 sec latency 835.890 ms 2024-09-10T22:27:25.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:25 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:27:25.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:25 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:27:25.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:25 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:27:25.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54620 4.83 MB/sec execute 496 sec latency 606.667 ms 2024-09-10T22:27:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:26 smithi090 ceph-mon[30350]: pgmap v1159: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-09-10T22:27:26.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:26 smithi154 ceph-mon[34495]: pgmap v1159: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-09-10T22:27:26.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:26 smithi171 ceph-mon[34138]: pgmap v1159: 129 pgs: 129 active+clean; 1.2 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.3 MiB/s wr, 252 op/s 2024-09-10T22:27:26.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54714 4.82 MB/sec execute 497 sec latency 445.241 ms 2024-09-10T22:27:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:27 smithi090 ceph-mon[30350]: pgmap v1160: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2024-09-10T22:27:27.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:27 smithi154 ceph-mon[34495]: pgmap v1160: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2024-09-10T22:27:27.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:27 smithi171 ceph-mon[34138]: pgmap v1160: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 204 op/s 2024-09-10T22:27:27.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54789 4.82 MB/sec execute 498 sec latency 681.606 ms 2024-09-10T22:27:28.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54908 4.82 MB/sec execute 499 sec latency 114.393 ms 2024-09-10T22:27:29.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.015+0000 7ff1bea00640 1 -- 172.21.15.90:0/3729255838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b8109150 msgr2=0x7ff1b810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:29.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.015+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3729255838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b8109150 0x7ff1b810f950 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009ca0 tx=0x7ff1b002a8e0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.015+0000 7ff1bea00640 1 -- 172.21.15.90:0/3729255838 shutdown_connections 2024-09-10T22:27:29.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.015+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3729255838 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b8109150 0x7ff1b810f950 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.015+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3729255838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1b81087d0 0x7ff1b8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.015+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3729255838 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b8102dc0 0x7ff1b81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.015+0000 7ff1bea00640 1 -- 172.21.15.90:0/3729255838 >> 172.21.15.90:0/3729255838 conn(0x7ff1b80fe830 msgr2=0x7ff1b8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:29.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.015+0000 7ff1bea00640 1 -- 172.21.15.90:0/3729255838 shutdown_connections 2024-09-10T22:27:29.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.016+0000 7ff1bea00640 1 -- 172.21.15.90:0/3729255838 wait complete. 2024-09-10T22:27:29.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.017+0000 7ff1bea00640 1 Processor -- start 2024-09-10T22:27:29.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.017+0000 7ff1bea00640 1 -- start start 2024-09-10T22:27:29.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.018+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b8102dc0 0x7ff1b819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.018+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b81087d0 0x7ff1b819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.018+0000 7ff1bea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1b8109150 0x7ff1b81a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.018+0000 7ff1bea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b80768c0 con 0x7ff1b8109150 2024-09-10T22:27:29.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.018+0000 7ff1bea00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8076740 con 0x7ff1b8102dc0 2024-09-10T22:27:29.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.018+0000 7ff1bea00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8076a40 con 0x7ff1b81087d0 2024-09-10T22:27:29.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.019+0000 7ff1b7400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b81087d0 0x7ff1b819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.019+0000 7ff1b7400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b81087d0 0x7ff1b819b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:57950/0 (socket says 172.21.15.90:57950) 2024-09-10T22:27:29.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.019+0000 7ff1b7400640 1 -- 172.21.15.90:0/3716137523 learned_addr learned my addr 172.21.15.90:0/3716137523 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:29.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.019+0000 7ff1bca00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1b8109150 0x7ff1b81a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.019+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b8102dc0 0x7ff1b819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.020+0000 7ff1b7e00640 1 -- 172.21.15.90:0/3716137523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b81087d0 msgr2=0x7ff1b819b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.020+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b81087d0 0x7ff1b819b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.020+0000 7ff1b7e00640 1 -- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1b8109150 msgr2=0x7ff1b81a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.020+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1b8109150 0x7ff1b81a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.020+0000 7ff1b7e00640 1 -- 172.21.15.90:0/3716137523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b81a13d0 con 0x7ff1b8102dc0 2024-09-10T22:27:29.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.020+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b8102dc0 0x7ff1b819b320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800e9f0 tx=0x7ff1a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:29.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.021+0000 7ff1b4c00640 1 -- 172.21.15.90:0/3716137523 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800ccc0 con 0x7ff1b8102dc0 2024-09-10T22:27:29.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.021+0000 7ff1bea00640 1 -- 172.21.15.90:0/3716137523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b81a16c0 con 0x7ff1b8102dc0 2024-09-10T22:27:29.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.021+0000 7ff1bea00640 1 -- 172.21.15.90:0/3716137523 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b81a7f50 con 0x7ff1b8102dc0 2024-09-10T22:27:29.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.021+0000 7ff1b4c00640 1 -- 172.21.15.90:0/3716137523 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1a80108c0 con 0x7ff1b8102dc0 2024-09-10T22:27:29.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.021+0000 7ff1b4c00640 1 -- 172.21.15.90:0/3716137523 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80187c0 con 0x7ff1b8102dc0 2024-09-10T22:27:29.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.026+0000 7ff1b4c00640 1 -- 172.21.15.90:0/3716137523 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff1a8010430 con 0x7ff1b8102dc0 2024-09-10T22:27:29.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.029+0000 7ff1b4c00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff19c0b5550 0x7ff19c0b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.029+0000 7ff1b4c00640 1 -- 172.21.15.90:0/3716137523 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7ff1a80e64d0 con 0x7ff1b8102dc0 2024-09-10T22:27:29.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.029+0000 7ff1b7400640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff19c0b5550 0x7ff19c0b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.030+0000 7ff1b7400640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff19c0b5550 0x7ff19c0b7a10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80772f0 tx=0x7ff1a4005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:29.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.057+0000 7ff1c165c740 1 Processor -- start 2024-09-10T22:27:29.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.057+0000 7ff1c165c740 1 -- start start 2024-09-10T22:27:29.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.058+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860cc80 0x55828860b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.058+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860b6e0 0x55828860bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.058+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288622110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.058+0000 7ff1c165c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x558288580a90 con 0x55828860b6e0 2024-09-10T22:27:29.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.058+0000 7ff1c165c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55828856fec0 con 0x55828861fd20 2024-09-10T22:27:29.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.058+0000 7ff1c165c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558288573cd0 con 0x55828860cc80 2024-09-10T22:27:29.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.058+0000 7ff1bca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288622110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.058+0000 7ff1bca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288622110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:41436/0 (socket says 172.21.15.90:41436) 2024-09-10T22:27:29.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.058+0000 7ff1bca00640 1 -- 172.21.15.90:0/3030265245 learned_addr learned my addr 172.21.15.90:0/3030265245 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:29.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.059+0000 7ff1b7400640 1 --2- 172.21.15.90:0/3030265245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860b6e0 0x55828860bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.059+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/3030265245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860cc80 0x55828860b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.059+0000 7ff1bca00640 1 -- 172.21.15.90:0/3030265245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860cc80 msgr2=0x55828860b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:29.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.059+0000 7ff1bca00640 1 --2- 172.21.15.90:0/3030265245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860cc80 0x55828860b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.059+0000 7ff1bca00640 1 -- 172.21.15.90:0/3030265245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860b6e0 msgr2=0x55828860bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:29.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.059+0000 7ff1bca00640 1 --2- 172.21.15.90:0/3030265245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860b6e0 0x55828860bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.059+0000 7ff1bca00640 1 -- 172.21.15.90:0/3030265245 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582885159d0 con 0x55828861fd20 2024-09-10T22:27:29.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.060+0000 7ff1bca00640 1 --2- 172.21.15.90:0/3030265245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288622110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009ca0 tx=0x7ff1b0005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8412a0a3b5ca2ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:29.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.060+0000 7ff1ad200640 1 -- 172.21.15.90:0/3030265245 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00317d0 con 0x55828861fd20 2024-09-10T22:27:29.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.060+0000 7ff1ad200640 1 -- 172.21.15.90:0/3030265245 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1b0036040 con 0x55828861fd20 2024-09-10T22:27:29.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.060+0000 7ff1ad200640 1 -- 172.21.15.90:0/3030265245 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0046420 con 0x55828861fd20 2024-09-10T22:27:29.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.061+0000 7ff1c165c740 1 -- 172.21.15.90:0/3030265245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 msgr2=0x558288622110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:29.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.061+0000 7ff1c165c740 1 --2- 172.21.15.90:0/3030265245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288622110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009ca0 tx=0x7ff1b0005f80 comp rx=0 tx=0).stop 2024-09-10T22:27:29.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.062+0000 7ff1c165c740 1 -- 172.21.15.90:0/3030265245 shutdown_connections 2024-09-10T22:27:29.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.062+0000 7ff1c165c740 1 --2- 172.21.15.90:0/3030265245 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288622110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.062+0000 7ff1c165c740 1 --2- 172.21.15.90:0/3030265245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860b6e0 0x55828860bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.062+0000 7ff1c165c740 1 --2- 172.21.15.90:0/3030265245 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860cc80 0x55828860b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.062+0000 7ff1c165c740 1 -- 172.21.15.90:0/3030265245 >> 172.21.15.90:0/3030265245 conn(0x5582885d56f0 msgr2=0x558288608fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:29.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.062+0000 7ff1c165c740 1 -- 172.21.15.90:0/3030265245 shutdown_connections 2024-09-10T22:27:29.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.062+0000 7ff1c165c740 1 -- 172.21.15.90:0/3030265245 wait complete. 2024-09-10T22:27:29.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.063+0000 7ff1c165c740 1 -- start start 2024-09-10T22:27:29.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.063+0000 7ff1c165c740 1 Processor -- start 2024-09-10T22:27:29.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860b6e0 0x55828861cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x55828861d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828861fd20 0x55828861d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1c165c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5582883d17b0 con 0x55828860cc80 2024-09-10T22:27:29.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1c165c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x558288514650 con 0x55828860b6e0 2024-09-10T22:27:29.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1c165c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x558288622ad0 con 0x55828861fd20 2024-09-10T22:27:29.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1b7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860b6e0 0x55828861cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1b7e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860b6e0 0x55828861cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:41440/0 (socket says 172.21.15.90:41440) 2024-09-10T22:27:29.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1b7e00640 1 -- 172.21.15.90:0/1588694924 learned_addr learned my addr 172.21.15.90:0/1588694924 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:29.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.064+0000 7ff1c165c740 10 client.0 _ll_register_callbacks cb 0x55828827b0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:27:29.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1bca00640 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828861fd20 0x55828861d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1b7400640 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x55828861d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1bca00640 1 -- 172.21.15.90:0/1588694924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860b6e0 msgr2=0x55828861cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:29.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1bca00640 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860b6e0 0x55828861cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1bca00640 1 -- 172.21.15.90:0/1588694924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 msgr2=0x55828861d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:29.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1bca00640 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x55828861d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1bca00640 1 -- 172.21.15.90:0/1588694924 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582882ddba0 con 0x55828861fd20 2024-09-10T22:27:29.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1b7400640 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x55828861d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:27:29.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1bca00640 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828861fd20 0x55828861d760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006010 tx=0x7ff1b00307f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c7f5e590a1a8843 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:29.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.065+0000 7ff1a1600640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:27:29.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.066+0000 7ff1a1600640 1 -- 172.21.15.90:0/1588694924 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b0031570 con 0x55828861fd20 2024-09-10T22:27:29.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.066+0000 7ff1a1600640 1 -- 172.21.15.90:0/1588694924 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7ff1b0030960 con 0x55828861fd20 2024-09-10T22:27:29.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.066+0000 7ff1c165c740 1 -- 172.21.15.90:0/1588694924 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b0036040 con 0x55828861fd20 2024-09-10T22:27:29.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.067+0000 7ff1a1600640 1 -- 172.21.15.90:0/1588694924 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff1b0030ba0 con 0x55828861fd20 2024-09-10T22:27:29.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.067+0000 7ff1c165c740 10 client.26122 fetch_fsmap learned FSMap version 91 2024-09-10T22:27:29.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.067+0000 7ff1c165c740 1 -- 172.21.15.90:0/1588694924 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) -- 0x55828862e6c0 con 0x55828861fd20 2024-09-10T22:27:29.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.068+0000 7ff1a1600640 1 -- 172.21.15.90:0/1588694924 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1b0036040 con 0x55828861fd20 2024-09-10T22:27:29.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.068+0000 7ff1a1600640 1 -- 172.21.15.90:0/1588694924 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 91) ==== 8402+0+0 (secure 0 0 0) 0x7ff1b005b420 con 0x55828861fd20 2024-09-10T22:27:29.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.068+0000 7ff1c165c740 10 client.26122 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T22:27:29.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.068+0000 7ff1c165c740 10 client.26122 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:27:29.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.068+0000 7ff1c165c740 20 client.26122 populate_metadata read hostname 'smithi090' 2024-09-10T22:27:29.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.068+0000 7ff1c165c740 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55828862fa10 0x558288631e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.068+0000 7ff1c165c740 4 client.26122 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:27:29.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.068+0000 7ff1c165c740 1 -- 172.21.15.90:0/1588694924 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582885829e0 con 0x55828862fa10 2024-09-10T22:27:29.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.069+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55828862fa10 0x558288631e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.070+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55828862fa10 0x558288631e00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b23c25dc4cdbdab server_cookie=b885ccadce6cfec3 in_seq=0 out_seq=0 2024-09-10T22:27:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.070+0000 7ff1a1600640 10 client.26122 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.071+0000 7ff1a1600640 1 -- 172.21.15.90:0/1588694924 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582885829e0 con 0x55828862fa10 2024-09-10T22:27:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.071+0000 7ff1a1600640 10 client.26122 handle_command_reply: tid=0 2024-09-10T22:27:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 client.26122 shutdown 2024-09-10T22:27:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 -- 172.21.15.90:0/1588694924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828861fd20 msgr2=0x55828861d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:29.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828861fd20 0x55828861d760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0006010 tx=0x7ff1b00307f0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 -- 172.21.15.90:0/1588694924 shutdown_connections 2024-09-10T22:27:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55828862fa10 0x558288631e00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828861fd20 0x55828861d760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x55828861d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 --2- 172.21.15.90:0/1588694924 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860b6e0 0x55828861cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 -- 172.21.15.90:0/1588694924 >> 172.21.15.90:0/1588694924 conn(0x5582885d56f0 msgr2=0x55828860eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:29.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1a1600640 0 client.26122 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:29.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.072+0000 7ff1c165c740 1 -- 172.21.15.90:0/1588694924 shutdown_connections 2024-09-10T22:27:29.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.073+0000 7ff1c165c740 1 -- 172.21.15.90:0/1588694924 wait complete. 2024-09-10T22:27:29.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.073+0000 7ff1c165c740 20 client.26122 trim_cache size 0 max 16384 2024-09-10T22:27:29.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.106+0000 7ff1c165c740 1 Processor -- start 2024-09-10T22:27:29.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.106+0000 7ff1c165c740 1 -- start start 2024-09-10T22:27:29.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.106+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288622070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.106+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860cc80 0x55828860ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.106+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828861fd20 0x558288608fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.106+0000 7ff1c165c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff1980100a0 con 0x55828861fd20 2024-09-10T22:27:29.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.106+0000 7ff1c165c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff198011d60 con 0x55828860cc80 2024-09-10T22:27:29.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.106+0000 7ff1c165c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff198010830 con 0x55828860b6e0 2024-09-10T22:27:29.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.107+0000 7ff1b7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288622070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.107+0000 7ff1b7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288622070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:57978/0 (socket says 172.21.15.90:57978) 2024-09-10T22:27:29.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.107+0000 7ff1b7e00640 1 -- 172.21.15.90:0/3547251848 learned_addr learned my addr 172.21.15.90:0/3547251848 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:29.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.107+0000 7ff1b7400640 1 --2- 172.21.15.90:0/3547251848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860cc80 0x55828860ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.108+0000 7ff1b7e00640 1 -- 172.21.15.90:0/3547251848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860cc80 msgr2=0x55828860ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:29.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.108+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/3547251848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860cc80 0x55828860ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.108+0000 7ff1b7e00640 1 -- 172.21.15.90:0/3547251848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828861fd20 msgr2=0x558288608fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:29.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.108+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/3547251848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828861fd20 0x558288608fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.108+0000 7ff1b7e00640 1 -- 172.21.15.90:0/3547251848 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55828861b1e0 con 0x55828860b6e0 2024-09-10T22:27:29.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.108+0000 7ff1b7400640 1 --2- 172.21.15.90:0/3547251848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860cc80 0x55828860ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:27:29.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.109+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/3547251848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288622070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000be0 tx=0x7ff1a8009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2b2a4de52280e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:29.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.109+0000 7ff1a3e00640 1 -- 172.21.15.90:0/3547251848 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80ac0e0 con 0x55828860b6e0 2024-09-10T22:27:29.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.110+0000 7ff1a3e00640 1 -- 172.21.15.90:0/3547251848 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1a80ac5d0 con 0x55828860b6e0 2024-09-10T22:27:29.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.110+0000 7ff1a3e00640 1 -- 172.21.15.90:0/3547251848 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80e60c0 con 0x55828860b6e0 2024-09-10T22:27:29.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.110+0000 7ff1c165c740 1 -- 172.21.15.90:0/3547251848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 msgr2=0x558288622070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:29.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.110+0000 7ff1c165c740 1 --2- 172.21.15.90:0/3547251848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288622070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8000be0 tx=0x7ff1a8009700 comp rx=0 tx=0).stop 2024-09-10T22:27:29.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.110+0000 7ff1c165c740 1 -- 172.21.15.90:0/3547251848 shutdown_connections 2024-09-10T22:27:29.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.110+0000 7ff1c165c740 1 --2- 172.21.15.90:0/3547251848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828861fd20 0x558288608fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.110+0000 7ff1c165c740 1 --2- 172.21.15.90:0/3547251848 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828860cc80 0x55828860ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.110+0000 7ff1c165c740 1 --2- 172.21.15.90:0/3547251848 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288622070 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.110+0000 7ff1c165c740 1 -- 172.21.15.90:0/3547251848 >> 172.21.15.90:0/3547251848 conn(0x5582885d56f0 msgr2=0x558288607890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:29.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.111+0000 7ff1c165c740 1 -- 172.21.15.90:0/3547251848 shutdown_connections 2024-09-10T22:27:29.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.111+0000 7ff1c165c740 1 -- 172.21.15.90:0/3547251848 wait complete. 2024-09-10T22:27:29.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.111+0000 7ff1c165c740 1 -- start start 2024-09-10T22:27:29.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.111+0000 7ff1c165c740 1 Processor -- start 2024-09-10T22:27:29.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288635080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x558288635600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1c165c740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288631e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1c165c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff198011630 con 0x55828860cc80 2024-09-10T22:27:29.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1c165c740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff198010f00 con 0x55828861fd20 2024-09-10T22:27:29.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1c165c740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff19800e0a0 con 0x55828860b6e0 2024-09-10T22:27:29.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1c165c740 10 client.0 _ll_register_callbacks cb 0x7ff1a40099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:27:29.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1b7400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x558288635600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1b7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288635080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1b7e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288635080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:57980/0 (socket says 172.21.15.90:57980) 2024-09-10T22:27:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1b7e00640 1 -- 172.21.15.90:0/484751012 learned_addr learned my addr 172.21.15.90:0/484751012 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.112+0000 7ff1bca00640 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288631e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.113+0000 7ff1b7e00640 1 -- 172.21.15.90:0/484751012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 msgr2=0x558288631e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.113+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288631e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.113+0000 7ff1b7e00640 1 -- 172.21.15.90:0/484751012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 msgr2=0x558288635600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:29.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.113+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x558288635600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.113+0000 7ff1b7e00640 1 -- 172.21.15.90:0/484751012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558288632610 con 0x55828860b6e0 2024-09-10T22:27:29.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.113+0000 7ff1b7400640 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x558288635600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:27:29.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.113+0000 7ff1b7e00640 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288635080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009250 tx=0x7ff1a80e3a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=710df507d26da886 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:29.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.113+0000 7ff1a1600640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:27:29.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.114+0000 7ff1a1600640 1 -- 172.21.15.90:0/484751012 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80d3040 con 0x55828860b6e0 2024-09-10T22:27:29.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.114+0000 7ff1a1600640 1 -- 172.21.15.90:0/484751012 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7ff1a80b5620 con 0x55828860b6e0 2024-09-10T22:27:29.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.114+0000 7ff1c165c740 1 -- 172.21.15.90:0/484751012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a80ac5d0 con 0x55828860b6e0 2024-09-10T22:27:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.114+0000 7ff1a1600640 1 -- 172.21.15.90:0/484751012 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff1a80b1000 con 0x55828860b6e0 2024-09-10T22:27:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.114+0000 7ff1c165c740 10 client.26128 fetch_fsmap learned FSMap version 91 2024-09-10T22:27:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.114+0000 7ff1c165c740 1 -- 172.21.15.90:0/484751012 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=4+}) -- 0x55828869d060 con 0x55828860b6e0 2024-09-10T22:27:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.114+0000 7ff1a1600640 1 -- 172.21.15.90:0/484751012 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff1a80c5030 con 0x55828860b6e0 2024-09-10T22:27:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.114+0000 7ff1a1600640 1 -- 172.21.15.90:0/484751012 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 91) ==== 8402+0+0 (secure 0 0 0) 0x7ff1a80d31e0 con 0x55828860b6e0 2024-09-10T22:27:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.115+0000 7ff1c165c740 10 client.26128 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T22:27:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.115+0000 7ff1c165c740 10 client.26128 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:27:29.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.115+0000 7ff1c165c740 20 client.26128 populate_metadata read hostname 'smithi090' 2024-09-10T22:27:29.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.115+0000 7ff1c165c740 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55828869d830 0x55828869fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:29.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.115+0000 7ff1c165c740 4 client.26128 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-10T22:27:29.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.115+0000 7ff1c165c740 1 -- 172.21.15.90:0/484751012 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x55828857e490 con 0x55828869d830 2024-09-10T22:27:29.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.115+0000 7ff1b7400640 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55828869d830 0x55828869fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:29.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.118+0000 7ff1b7400640 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55828869d830 0x55828869fc50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3102fa0530cd8a04 server_cookie=ad9d3cdaa59a25ef in_seq=0 out_seq=0 2024-09-10T22:27:29.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.118+0000 7ff1a1600640 10 client.26128 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1a1600640 1 -- 172.21.15.90:0/484751012 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55828857e490 con 0x55828869d830 2024-09-10T22:27:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1a1600640 10 client.26128 handle_command_reply: tid=0 2024-09-10T22:27:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 client.26128 shutdown 2024-09-10T22:27:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 -- 172.21.15.90:0/484751012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 msgr2=0x558288635080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288635080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009250 tx=0x7ff1a80e3a70 comp rx=0 tx=0).stop 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 -- 172.21.15.90:0/484751012 shutdown_connections 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55828869d830 0x55828869fc50 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55828861fd20 0x558288631e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55828860cc80 0x558288635600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 --2- 172.21.15.90:0/484751012 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55828860b6e0 0x558288635080 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1a1600640 0 client.26128 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 -- 172.21.15.90:0/484751012 >> 172.21.15.90:0/484751012 conn(0x5582885d56f0 msgr2=0x55828861a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 -- 172.21.15.90:0/484751012 shutdown_connections 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.532+0000 7ff1c165c740 1 -- 172.21.15.90:0/484751012 wait complete. 2024-09-10T22:27:29.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.533+0000 7ff1c165c740 20 client.26128 trim_cache size 0 max 16384 2024-09-10T22:27:29.535 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:27:29.535 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 91, 2024-09-10T22:27:29.535 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:27:29.535 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 70, 2024-09-10T22:27:29.535 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:27:29.535 INFO:teuthology.orchestra.run.smithi090.stdout: "6f421676": { 2024-09-10T22:27:29.535 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 70, 2024-09-10T22:27:29.535 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 138.8, 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:29.536 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:29.537 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:29.537 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:29.537 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:29.537 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:27:29.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:29 smithi090 ceph-mon[30350]: pgmap v1161: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 190 op/s 2024-09-10T22:27:29.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:27:29.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:27:29.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:27:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.533+0000 7ff1bea00640 1 -- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff19c0b5550 msgr2=0x7ff19c0b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.533+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff19c0b5550 0x7ff19c0b7a10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80772f0 tx=0x7ff1a4005f90 comp rx=0 tx=0).stop 2024-09-10T22:27:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.533+0000 7ff1bea00640 1 -- 172.21.15.90:0/3716137523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b8102dc0 msgr2=0x7ff1b819b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.533+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b8102dc0 0x7ff1b819b320 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff1a800e9f0 tx=0x7ff1a800eeb0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.534+0000 7ff1bea00640 1 -- 172.21.15.90:0/3716137523 shutdown_connections 2024-09-10T22:27:29.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.534+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff19c0b5550 0x7ff19c0b7a10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.534+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1b8109150 0x7ff1b81a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.534+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff1b81087d0 0x7ff1b819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.534+0000 7ff1bea00640 1 --2- 172.21.15.90:0/3716137523 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff1b8102dc0 0x7ff1b819b320 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.534+0000 7ff1bea00640 1 -- 172.21.15.90:0/3716137523 >> 172.21.15.90:0/3716137523 conn(0x7ff1b80fe830 msgr2=0x7ff1b8101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.535+0000 7ff1bea00640 1 -- 172.21.15.90:0/3716137523 shutdown_connections 2024-09-10T22:27:29.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:29.535+0000 7ff1bea00640 1 -- 172.21.15.90:0/3716137523 wait complete. 2024-09-10T22:27:29.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:29 smithi154 ceph-mon[34495]: pgmap v1161: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 190 op/s 2024-09-10T22:27:29.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:27:29.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:27:29.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:27:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:29 smithi171 ceph-mon[34138]: pgmap v1161: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.3 MiB/s wr, 190 op/s 2024-09-10T22:27:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:27:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:27:29.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:27:29.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54938 4.81 MB/sec execute 500 sec latency 799.750 ms 2024-09-10T22:27:30.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54938 4.80 MB/sec execute 501 sec latency 1799.872 ms 2024-09-10T22:27:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:31 smithi090 ceph-mon[30350]: pgmap v1162: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 225 op/s 2024-09-10T22:27:31.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:31 smithi154 ceph-mon[34495]: pgmap v1162: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 225 op/s 2024-09-10T22:27:31.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:31 smithi171 ceph-mon[34138]: pgmap v1162: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 225 op/s 2024-09-10T22:27:31.822 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6f421676', quiesce took 21.5 seconds. Will release after: -2.6499999999999986 2024-09-10T22:27:31.822 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:27:31.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54938 4.79 MB/sec execute 502 sec latency 2799.944 ms 2024-09-10T22:27:32.219 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:27:32.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54938 4.78 MB/sec execute 503 sec latency 3800.010 ms 2024-09-10T22:27:32.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.972+0000 7fe7ad600640 1 -- 172.21.15.90:0/3327918835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a8101fc0 msgr2=0x7fe7a810ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.972+0000 7fe7ad600640 1 --2- 172.21.15.90:0/3327918835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a8101fc0 0x7fe7a810ed10 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe790009900 tx=0x7fe79002a480 comp rx=0 tx=0).stop 2024-09-10T22:27:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.972+0000 7fe7ad600640 1 -- 172.21.15.90:0/3327918835 shutdown_connections 2024-09-10T22:27:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.972+0000 7fe7ad600640 1 --2- 172.21.15.90:0/3327918835 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a810f250 0x7fe7a8111640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.972+0000 7fe7ad600640 1 --2- 172.21.15.90:0/3327918835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a8101fc0 0x7fe7a810ed10 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.972+0000 7fe7ad600640 1 --2- 172.21.15.90:0/3327918835 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a81016a0 0x7fe7a8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.972+0000 7fe7ad600640 1 -- 172.21.15.90:0/3327918835 >> 172.21.15.90:0/3327918835 conn(0x7fe7a80fd5b0 msgr2=0x7fe7a80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.972+0000 7fe7ad600640 1 -- 172.21.15.90:0/3327918835 shutdown_connections 2024-09-10T22:27:32.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.972+0000 7fe7ad600640 1 -- 172.21.15.90:0/3327918835 wait complete. 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.973+0000 7fe7ad600640 1 Processor -- start 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.973+0000 7fe7ad600640 1 -- start start 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7ad600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a81016a0 0x7fe7a819b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7ad600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a8101fc0 0x7fe7a819b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7ad600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a810f250 0x7fe7a81a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7ad600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8114c80 con 0x7fe7a810f250 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7ad600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8114b00 con 0x7fe7a8101fc0 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7ad600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a8114e00 con 0x7fe7a81016a0 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7acc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a810f250 0x7fe7a81a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7acc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a810f250 0x7fe7a81a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50786/0 (socket says 172.21.15.90:50786) 2024-09-10T22:27:32.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7acc00640 1 -- 172.21.15.90:0/2202164365 learned_addr learned my addr 172.21.15.90:0/2202164365 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe7a7e00640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a81016a0 0x7fe7a819b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.974+0000 7fe79f400640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a8101fc0 0x7fe7a819b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7acc00640 1 -- 172.21.15.90:0/2202164365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a81016a0 msgr2=0x7fe7a819b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7acc00640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a81016a0 0x7fe7a819b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7acc00640 1 -- 172.21.15.90:0/2202164365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a8101fc0 msgr2=0x7fe7a819b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7acc00640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a8101fc0 0x7fe7a819b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7acc00640 1 -- 172.21.15.90:0/2202164365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a81a1120 con 0x7fe7a810f250 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7acc00640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a810f250 0x7fe7a81a0950 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000eaa0 tx=0x7fe7a000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7a5600640 1 -- 172.21.15.90:0/2202164365 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0004d20 con 0x7fe7a810f250 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7a5600640 1 -- 172.21.15.90:0/2202164365 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7a0005020 con 0x7fe7a810f250 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a81a1410 con 0x7fe7a810f250 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7a5600640 1 -- 172.21.15.90:0/2202164365 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00116d0 con 0x7fe7a810f250 2024-09-10T22:27:32.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.975+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a81a7ca0 con 0x7fe7a810f250 2024-09-10T22:27:32.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.976+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a8101a80 con 0x7fe7a810f250 2024-09-10T22:27:32.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.979+0000 7fe7a5600640 1 -- 172.21.15.90:0/2202164365 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe7a0022060 con 0x7fe7a810f250 2024-09-10T22:27:32.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.981+0000 7fe7a5600640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe78c0b5580 0x7fe78c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:32.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.981+0000 7fe7a5600640 1 -- 172.21.15.90:0/2202164365 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7fe7a00e5d80 con 0x7fe7a810f250 2024-09-10T22:27:32.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.982+0000 7fe7a7e00640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe78c0b5580 0x7fe78c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:32.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.982+0000 7fe7a7e00640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe78c0b5580 0x7fe78c0b7a40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe79800abe0 tx=0x7fe79800a2d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:32.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:32.987+0000 7fe7a5600640 1 -- 172.21.15.90:0/2202164365 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe7a0016030 con 0x7fe7a810f250 2024-09-10T22:27:33.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.207+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7a81a7e70 con 0x7fe7a810f250 2024-09-10T22:27:33.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.208+0000 7fe7a5600640 1 -- 172.21.15.90:0/2202164365 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+11999 (secure 0 0 0) 0x7fe7a00ac530 con 0x7fe7a810f250 2024-09-10T22:27:33.209 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:27:33.209 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":91,"btime":"2024-09-10T22:26:22:850763+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":91,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:26:22.770344+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:27:33.209 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 91 2024-09-10T22:27:33.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.210+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe78c0b5580 msgr2=0x7fe78c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:33.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.210+0000 7fe7ad600640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe78c0b5580 0x7fe78c0b7a40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe79800abe0 tx=0x7fe79800a2d0 comp rx=0 tx=0).stop 2024-09-10T22:27:33.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.210+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a810f250 msgr2=0x7fe7a81a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:33.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.210+0000 7fe7ad600640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a810f250 0x7fe7a81a0950 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000eaa0 tx=0x7fe7a000eea0 comp rx=0 tx=0).stop 2024-09-10T22:27:33.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.211+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 shutdown_connections 2024-09-10T22:27:33.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.211+0000 7fe7ad600640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe78c0b5580 0x7fe78c0b7a40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:33.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.211+0000 7fe7ad600640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7a810f250 0x7fe7a81a0950 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:33.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.211+0000 7fe7ad600640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe7a8101fc0 0x7fe7a819b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:33.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.211+0000 7fe7ad600640 1 --2- 172.21.15.90:0/2202164365 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe7a81016a0 0x7fe7a819b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:33.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.211+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 >> 172.21.15.90:0/2202164365 conn(0x7fe7a80fd5b0 msgr2=0x7fe7a810fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:33.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.211+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 shutdown_connections 2024-09-10T22:27:33.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:33.211+0000 7fe7ad600640 1 -- 172.21.15.90:0/2202164365 wait complete. 2024-09-10T22:27:33.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:33 smithi090 ceph-mon[30350]: pgmap v1163: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2024-09-10T22:27:33.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:33 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2202164365' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:33.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:33 smithi154 ceph-mon[34495]: pgmap v1163: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2024-09-10T22:27:33.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:33 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2202164365' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:33.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:33 smithi171 ceph-mon[34138]: pgmap v1163: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 177 op/s 2024-09-10T22:27:33.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:33 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2202164365' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:33.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54938 4.77 MB/sec execute 504 sec latency 4800.371 ms 2024-09-10T22:27:34.179 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 6f421676 --release --await' 2024-09-10T22:27:34.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 6f421676 --release --await 2024-09-10T22:27:34.553 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:27:34.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:34 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:34.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:34 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:34.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:34 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:34.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54938 4.76 MB/sec execute 505 sec latency 5800.416 ms 2024-09-10T22:27:35.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.264+0000 7f7312000640 1 -- 172.21.15.90:0/520233132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c104df0 msgr2=0x7f730c1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:35.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.264+0000 7f7312000640 1 --2- 172.21.15.90:0/520233132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c104df0 0x7f730c1051d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f7300009900 tx=0x7f730002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.264+0000 7f7312000640 1 -- 172.21.15.90:0/520233132 shutdown_connections 2024-09-10T22:27:35.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.264+0000 7f7312000640 1 --2- 172.21.15.90:0/520233132 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f730c10b180 0x7f730c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.264+0000 7f7312000640 1 --2- 172.21.15.90:0/520233132 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f730c10a800 0x7f730c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.264+0000 7f7312000640 1 --2- 172.21.15.90:0/520233132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c104df0 0x7f730c1051d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.264+0000 7f7312000640 1 -- 172.21.15.90:0/520233132 >> 172.21.15.90:0/520233132 conn(0x7f730c100840 msgr2=0x7f730c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:35.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.264+0000 7f7312000640 1 -- 172.21.15.90:0/520233132 shutdown_connections 2024-09-10T22:27:35.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.264+0000 7f7312000640 1 -- 172.21.15.90:0/520233132 wait complete. 2024-09-10T22:27:35.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.265+0000 7f7312000640 1 Processor -- start 2024-09-10T22:27:35.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.265+0000 7f7312000640 1 -- start start 2024-09-10T22:27:35.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7312000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f730c104df0 0x7f730c19b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7312000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f730c10a800 0x7f730c19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7312000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c10b180 0x7f730c1a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7312000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f730c114cf0 con 0x7f730c10b180 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7312000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f730c114b70 con 0x7f730c104df0 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7312000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f730c114e70 con 0x7f730c10a800 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7311600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c10b180 0x7f730c1a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7311600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c10b180 0x7f730c1a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50806/0 (socket says 172.21.15.90:50806) 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7311600640 1 -- 172.21.15.90:0/1643550916 learned_addr learned my addr 172.21.15.90:0/1643550916 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f7310c00640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f730c104df0 0x7f730c19b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.266+0000 7f730be00640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f730c10a800 0x7f730c19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7311600640 1 -- 172.21.15.90:0/1643550916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f730c10a800 msgr2=0x7f730c19b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7311600640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f730c10a800 0x7f730c19b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7311600640 1 -- 172.21.15.90:0/1643550916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f730c104df0 msgr2=0x7f730c19b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7311600640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f730c104df0 0x7f730c19b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7311600640 1 -- 172.21.15.90:0/1643550916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f730c1a1130 con 0x7f730c10b180 2024-09-10T22:27:35.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7311600640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c10b180 0x7f730c1a0960 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f730400cad0 tx=0x7f730400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:35.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7309600640 1 -- 172.21.15.90:0/1643550916 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730400ad60 con 0x7f730c10b180 2024-09-10T22:27:35.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7309600640 1 -- 172.21.15.90:0/1643550916 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7304005020 con 0x7f730c10b180 2024-09-10T22:27:35.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7309600640 1 -- 172.21.15.90:0/1643550916 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730400f6b0 con 0x7f730c10b180 2024-09-10T22:27:35.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7312000640 1 -- 172.21.15.90:0/1643550916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f730c1a1420 con 0x7f730c10b180 2024-09-10T22:27:35.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.267+0000 7f7312000640 1 -- 172.21.15.90:0/1643550916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f730c1a7cb0 con 0x7f730c10b180 2024-09-10T22:27:35.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.270+0000 7f7309600640 1 -- 172.21.15.90:0/1643550916 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f7304004a40 con 0x7f730c10b180 2024-09-10T22:27:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.272+0000 7f7309600640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72f40b5580 0x7f72f40b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.272+0000 7f7309600640 1 -- 172.21.15.90:0/1643550916 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7f73040e62c0 con 0x7f730c10b180 2024-09-10T22:27:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.272+0000 7f7310c00640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72f40b5580 0x7f72f40b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.273+0000 7f7310c00640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72f40b5580 0x7f72f40b7a40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f73000057c0 tx=0x7f73000058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:35.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7316029740 1 Processor -- start 2024-09-10T22:27:35.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7316029740 1 -- start start 2024-09-10T22:27:35.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7316029740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db491d0 0x55d06db495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7316029740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 0x55d06db55540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7316029740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db57e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7316029740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d06dac0290 con 0x55d06db55a80 2024-09-10T22:27:35.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7316029740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d06daaf6c0 con 0x55d06db491d0 2024-09-10T22:27:35.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7316029740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d06dab34d0 con 0x55d06db49af0 2024-09-10T22:27:35.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7311600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db57e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7311600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db57e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50818/0 (socket says 172.21.15.90:50818) 2024-09-10T22:27:35.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7311600640 1 -- 172.21.15.90:0/2771762864 learned_addr learned my addr 172.21.15.90:0/2771762864 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:35.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.281+0000 7f7310c00640 1 --2- 172.21.15.90:0/2771762864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db491d0 0x55d06db495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f730be00640 1 --2- 172.21.15.90:0/2771762864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 0x55d06db55540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f7311600640 1 -- 172.21.15.90:0/2771762864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 msgr2=0x55d06db55540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:35.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f7311600640 1 --2- 172.21.15.90:0/2771762864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 0x55d06db55540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f7311600640 1 -- 172.21.15.90:0/2771762864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db491d0 msgr2=0x55d06db495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:35.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f7311600640 1 --2- 172.21.15.90:0/2771762864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db491d0 0x55d06db495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f7311600640 1 -- 172.21.15.90:0/2771762864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d06da3c460 con 0x55d06db55a80 2024-09-10T22:27:35.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f7311600640 1 --2- 172.21.15.90:0/2771762864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db57e70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f73040ac2a0 tx=0x7f73040abb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bc3120c98a80f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f72fdc00640 1 -- 172.21.15.90:0/2771762864 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730401d070 con 0x55d06db55a80 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f72fdc00640 1 -- 172.21.15.90:0/2771762864 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73040ac310 con 0x55d06db55a80 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.282+0000 7f72fdc00640 1 -- 172.21.15.90:0/2771762864 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73040b5070 con 0x55d06db55a80 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.283+0000 7f7316029740 1 -- 172.21.15.90:0/2771762864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 msgr2=0x55d06db57e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.283+0000 7f7316029740 1 --2- 172.21.15.90:0/2771762864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db57e70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f73040ac2a0 tx=0x7f73040abb40 comp rx=0 tx=0).stop 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.283+0000 7f7316029740 1 -- 172.21.15.90:0/2771762864 shutdown_connections 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.283+0000 7f7316029740 1 --2- 172.21.15.90:0/2771762864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db57e70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.283+0000 7f7316029740 1 --2- 172.21.15.90:0/2771762864 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 0x55d06db55540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.283+0000 7f7316029740 1 --2- 172.21.15.90:0/2771762864 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db491d0 0x55d06db495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.283+0000 7f7316029740 1 -- 172.21.15.90:0/2771762864 >> 172.21.15.90:0/2771762864 conn(0x55d06db47000 msgr2=0x55d06db481a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.283+0000 7f7316029740 1 -- 172.21.15.90:0/2771762864 shutdown_connections 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.283+0000 7f7316029740 1 -- 172.21.15.90:0/2771762864 wait complete. 2024-09-10T22:27:35.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.284+0000 7f7316029740 1 -- start start 2024-09-10T22:27:35.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.284+0000 7f7316029740 1 Processor -- start 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.284+0000 7f7316029740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db52a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.284+0000 7f7316029740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db52fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.284+0000 7f7316029740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db55a80 0x55d06db53500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.284+0000 7f7316029740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d06d910bb0 con 0x55d06db491d0 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.284+0000 7f7316029740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55d06da4a6c0 con 0x55d06db49af0 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.284+0000 7f7316029740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55d06db58670 con 0x55d06db55a80 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7316029740 10 client.0 _ll_register_callbacks cb 0x55d06da65490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7310c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db52a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7310c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db52a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50830/0 (socket says 172.21.15.90:50830) 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7310c00640 1 -- 172.21.15.90:0/1217340267 learned_addr learned my addr 172.21.15.90:0/1217340267 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f730be00640 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db52fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7311600640 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db55a80 0x55d06db53500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7310c00640 1 -- 172.21.15.90:0/1217340267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db55a80 msgr2=0x55d06db53500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7310c00640 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db55a80 0x55d06db53500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7310c00640 1 -- 172.21.15.90:0/1217340267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 msgr2=0x55d06db52fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7310c00640 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db52fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.285+0000 7f7310c00640 1 -- 172.21.15.90:0/1217340267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d06d80db00 con 0x55d06db491d0 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.286+0000 7f7310c00640 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db52a80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f730002a910 tx=0x7f730002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd70af3c6c427317 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.286+0000 7f72f2000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.286+0000 7f72f2000640 1 -- 172.21.15.90:0/1217340267 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300035050 con 0x55d06db491d0 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.286+0000 7f7316029740 1 -- 172.21.15.90:0/1217340267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73040ac310 con 0x55d06db491d0 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.286+0000 7f72f2000640 1 -- 172.21.15.90:0/1217340267 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7f73000038b0 con 0x55d06db491d0 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.286+0000 7f72f2000640 1 -- 172.21.15.90:0/1217340267 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7300003ab0 con 0x55d06db491d0 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.286+0000 7f7316029740 10 client.16530 fetch_fsmap learned FSMap version 92 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.286+0000 7f7316029740 1 -- 172.21.15.90:0/1217340267 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x55d06d884960 con 0x55d06db491d0 2024-09-10T22:27:35.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.286+0000 7f72f2000640 1 -- 172.21.15.90:0/1217340267 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73040ac310 con 0x55d06db491d0 2024-09-10T22:27:35.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.287+0000 7f72f2000640 1 -- 172.21.15.90:0/1217340267 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 92) ==== 8398+0+0 (secure 0 0 0) 0x7f730003cc60 con 0x55d06db491d0 2024-09-10T22:27:35.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.287+0000 7f7316029740 10 client.16530 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T22:27:35.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.287+0000 7f7316029740 10 client.16530 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:27:35.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.287+0000 7f7316029740 20 client.16530 populate_metadata read hostname 'smithi090' 2024-09-10T22:27:35.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.287+0000 7f7316029740 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d06db65590 0x55d06db67a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.287+0000 7f7316029740 4 client.16530 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:27:35.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.287+0000 7f7316029740 1 -- 172.21.15.90:0/1217340267 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d06dac21e0 con 0x55d06db65590 2024-09-10T22:27:35.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.287+0000 7f730be00640 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d06db65590 0x55d06db67a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.288+0000 7f730be00640 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d06db65590 0x55d06db67a00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b46663ac8aad038 server_cookie=c56aa57aa384dfe6 in_seq=0 out_seq=0 2024-09-10T22:27:35.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.288+0000 7f72f2000640 10 client.16530 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:35.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.289+0000 7f72f2000640 1 -- 172.21.15.90:0/1217340267 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d06dac21e0 con 0x55d06db65590 2024-09-10T22:27:35.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.289+0000 7f72f2000640 10 client.16530 handle_command_reply: tid=0 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.289+0000 7f7316029740 1 client.16530 shutdown 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.289+0000 7f7316029740 1 -- 172.21.15.90:0/1217340267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 msgr2=0x55d06db52a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.289+0000 7f7316029740 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db52a80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f730002a910 tx=0x7f730002a3f0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f7316029740 1 -- 172.21.15.90:0/1217340267 shutdown_connections 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f7316029740 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d06db65590 0x55d06db67a00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f7316029740 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db55a80 0x55d06db53500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f7316029740 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db52fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f72f2000640 0 client.16530 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f7316029740 1 --2- 172.21.15.90:0/1217340267 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db52a80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f7316029740 1 -- 172.21.15.90:0/1217340267 >> 172.21.15.90:0/1217340267 conn(0x55d06db47000 msgr2=0x55d06db4cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f7316029740 1 -- 172.21.15.90:0/1217340267 shutdown_connections 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f7316029740 1 -- 172.21.15.90:0/1217340267 wait complete. 2024-09-10T22:27:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.290+0000 7f7316029740 20 client.16530 trim_cache size 0 max 16384 2024-09-10T22:27:35.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.308+0000 7f7316029740 1 Processor -- start 2024-09-10T22:27:35.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.308+0000 7f7316029740 1 -- start start 2024-09-10T22:27:35.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7316029740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db491d0 0x55d06db458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7316029740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db45de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7316029740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db46320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7316029740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1b1d50 con 0x55d06db55a80 2024-09-10T22:27:35.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7316029740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1b39e0 con 0x55d06db49af0 2024-09-10T22:27:35.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7316029740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1b24e0 con 0x55d06db491d0 2024-09-10T22:27:35.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7311600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db46320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f730be00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db45de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7311600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db46320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50846/0 (socket says 172.21.15.90:50846) 2024-09-10T22:27:35.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7311600640 1 -- 172.21.15.90:0/239379052 learned_addr learned my addr 172.21.15.90:0/239379052 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:35.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f730be00640 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db45de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:41508/0 (socket says 172.21.15.90:41508) 2024-09-10T22:27:35.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.309+0000 7f7310c00640 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db491d0 0x55d06db458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f7311600640 1 -- 172.21.15.90:0/239379052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db491d0 msgr2=0x55d06db458a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:35.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f7311600640 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db491d0 0x55d06db458a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f7311600640 1 -- 172.21.15.90:0/239379052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 msgr2=0x55d06db45de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:35.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f7311600640 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db45de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f7311600640 1 -- 172.21.15.90:0/239379052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d06db68080 con 0x55d06db55a80 2024-09-10T22:27:35.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f730be00640 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db45de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:27:35.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f7311600640 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db46320 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7304007850 tx=0x7f73040abbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5792d2eb729880c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:35.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f72fd200640 1 -- 172.21.15.90:0/239379052 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730401d040 con 0x55d06db55a80 2024-09-10T22:27:35.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f72fd200640 1 -- 172.21.15.90:0/239379052 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73040b0df0 con 0x55d06db55a80 2024-09-10T22:27:35.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.310+0000 7f72fd200640 1 -- 172.21.15.90:0/239379052 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73040b5040 con 0x55d06db55a80 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.311+0000 7f7316029740 1 -- 172.21.15.90:0/239379052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 msgr2=0x55d06db46320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.311+0000 7f7316029740 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db46320 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7304007850 tx=0x7f73040abbb0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.311+0000 7f7316029740 1 -- 172.21.15.90:0/239379052 shutdown_connections 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.311+0000 7f7316029740 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db55a80 0x55d06db46320 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.311+0000 7f7316029740 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db49af0 0x55d06db45de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.311+0000 7f7316029740 1 --2- 172.21.15.90:0/239379052 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db491d0 0x55d06db458a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.311+0000 7f7316029740 1 -- 172.21.15.90:0/239379052 >> 172.21.15.90:0/239379052 conn(0x55d06db47000 msgr2=0x55d06db4dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.311+0000 7f7316029740 1 -- 172.21.15.90:0/239379052 shutdown_connections 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.311+0000 7f7316029740 1 -- 172.21.15.90:0/239379052 wait complete. 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7316029740 1 -- start start 2024-09-10T22:27:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7316029740 1 Processor -- start 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7316029740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db5daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7316029740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 0x55d06db5e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7316029740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db55a80 0x55d06db588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7316029740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1b32b0 con 0x55d06db491d0 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7316029740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1b2bb0 con 0x55d06db55a80 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7316029740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f730c1afd50 con 0x55d06db49af0 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7316029740 10 client.0 _ll_register_callbacks cb 0x7f7300032f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.312+0000 7f7310c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db5daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7310c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db5daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50850/0 (socket says 172.21.15.90:50850) 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7310c00640 1 -- 172.21.15.90:0/2484957253 learned_addr learned my addr 172.21.15.90:0/2484957253 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f730be00640 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 0x55d06db5e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7311600640 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db55a80 0x55d06db588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7310c00640 1 -- 172.21.15.90:0/2484957253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 msgr2=0x55d06db5e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:35.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7310c00640 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 0x55d06db5e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7310c00640 1 -- 172.21.15.90:0/2484957253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db55a80 msgr2=0x55d06db588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:35.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7310c00640 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db55a80 0x55d06db588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7310c00640 1 -- 172.21.15.90:0/2484957253 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d06db58ed0 con 0x55d06db491d0 2024-09-10T22:27:35.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f730be00640 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 0x55d06db5e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:27:35.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7311600640 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db55a80 0x55d06db588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:27:35.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f7310c00640 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db5daf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f73000319f0 tx=0x7f7300003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387a3386dbb1b741 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:35.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.313+0000 7f72f2000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:27:35.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.314+0000 7f72f2000640 1 -- 172.21.15.90:0/2484957253 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300065020 con 0x55d06db491d0 2024-09-10T22:27:35.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.314+0000 7f72f2000640 1 -- 172.21.15.90:0/2484957253 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7f7300052940 con 0x55d06db491d0 2024-09-10T22:27:35.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.314+0000 7f7316029740 1 -- 172.21.15.90:0/2484957253 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d06dabdc90 con 0x55d06db491d0 2024-09-10T22:27:35.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.314+0000 7f72f2000640 1 -- 172.21.15.90:0/2484957253 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f7300035040 con 0x55d06db491d0 2024-09-10T22:27:35.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.314+0000 7f7316029740 10 client.16536 fetch_fsmap learned FSMap version 92 2024-09-10T22:27:35.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.314+0000 7f7316029740 1 -- 172.21.15.90:0/2484957253 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x55d06dbd2eb0 con 0x55d06db491d0 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.314+0000 7f72f2000640 1 -- 172.21.15.90:0/2484957253 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d06dabdc90 con 0x55d06db491d0 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.315+0000 7f72f2000640 1 -- 172.21.15.90:0/2484957253 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 92) ==== 8398+0+0 (secure 0 0 0) 0x7f730003c9f0 con 0x55d06db491d0 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.315+0000 7f7316029740 10 client.16536 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.315+0000 7f7316029740 10 client.16536 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.315+0000 7f7316029740 20 client.16536 populate_metadata read hostname 'smithi090' 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.315+0000 7f7316029740 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d06dbd3540 0x55d06dbd5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.315+0000 7f7316029740 4 client.16536 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6f421676", "release": true, "await": true}] 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.315+0000 7f7316029740 1 -- 172.21.15.90:0/2484957253 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6f421676", "release": true, "await": true}) -- 0x55d06dabb0c0 con 0x55d06dbd3540 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.315+0000 7f730be00640 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d06dbd3540 0x55d06dbd5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.316+0000 7f730be00640 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d06dbd3540 0x55d06dbd5960 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f5d29523d27cd7 server_cookie=f1e7b83b80457df8 in_seq=0 out_seq=0 2024-09-10T22:27:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.316+0000 7f72f2000640 10 client.16536 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.361+0000 7f72f2000640 1 -- 172.21.15.90:0/2484957253 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d06dabb0c0 con 0x55d06dbd3540 2024-09-10T22:27:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.361+0000 7f72f2000640 10 client.16536 handle_command_reply: tid=0 2024-09-10T22:27:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.361+0000 7f7316029740 1 client.16536 shutdown 2024-09-10T22:27:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.361+0000 7f7316029740 1 -- 172.21.15.90:0/2484957253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 msgr2=0x55d06db5daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:35.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.361+0000 7f7316029740 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db5daf0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f73000319f0 tx=0x7f7300003f00 comp rx=0 tx=0).stop 2024-09-10T22:27:35.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f7316029740 1 -- 172.21.15.90:0/2484957253 shutdown_connections 2024-09-10T22:27:35.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f7316029740 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55d06dbd3540 0x55d06dbd5960 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f7316029740 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55d06db55a80 0x55d06db588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f7316029740 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55d06db49af0 0x55d06db5e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f72f2000640 0 client.16536 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:35.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f7316029740 1 --2- 172.21.15.90:0/2484957253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d06db491d0 0x55d06db5daf0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f7316029740 1 -- 172.21.15.90:0/2484957253 >> 172.21.15.90:0/2484957253 conn(0x55d06db47000 msgr2=0x55d06db458a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:35.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f7316029740 1 -- 172.21.15.90:0/2484957253 shutdown_connections 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f7316029740 1 -- 172.21.15.90:0/2484957253 wait complete. 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.362+0000 7f7316029740 20 client.16536 trim_cache size 0 max 16384 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 92, 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 72, 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stdout: "6f421676": { 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 72, 2024-09-10T22:27:35.364 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 138.8, 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:35.365 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:35.366 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:35.366 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:35.366 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:35.366 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:27:35.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 -- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72f40b5580 msgr2=0x7f72f40b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:35.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72f40b5580 0x7f72f40b7a40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f73000057c0 tx=0x7f73000058d0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 -- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c10b180 msgr2=0x7f730c1a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:35.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c10b180 0x7f730c1a0960 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f730400cad0 tx=0x7f730400ced0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 -- 172.21.15.90:0/1643550916 shutdown_connections 2024-09-10T22:27:35.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f72f40b5580 0x7f72f40b7a40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f730c10b180 0x7f730c1a0960 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f730c10a800 0x7f730c19b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 --2- 172.21.15.90:0/1643550916 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f730c104df0 0x7f730c19b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:35.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 -- 172.21.15.90:0/1643550916 >> 172.21.15.90:0/1643550916 conn(0x7f730c100840 msgr2=0x7f730c103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:35.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 -- 172.21.15.90:0/1643550916 shutdown_connections 2024-09-10T22:27:35.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:35.363+0000 7f7312000640 1 -- 172.21.15.90:0/1643550916 wait complete. 2024-09-10T22:27:35.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:35 smithi090 ceph-mon[30350]: pgmap v1164: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 195 op/s 2024-09-10T22:27:35.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:35 smithi154 ceph-mon[34495]: pgmap v1164: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 195 op/s 2024-09-10T22:27:35.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:35 smithi171 ceph-mon[34138]: pgmap v1164: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 195 op/s 2024-09-10T22:27:35.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55017 4.77 MB/sec execute 506 sec latency 6272.962 ms 2024-09-10T22:27:36.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55109 4.76 MB/sec execute 507 sec latency 1545.583 ms 2024-09-10T22:27:36.975 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6f421676', total seconds elapsed: 26.6 2024-09-10T22:27:36.976 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.85 2024-09-10T22:27:36.976 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:27:37.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:37 smithi090 ceph-mon[30350]: pgmap v1165: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-10T22:27:37.665 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:27:37.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:37 smithi154 ceph-mon[34495]: pgmap v1165: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-10T22:27:37.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:37 smithi171 ceph-mon[34138]: pgmap v1165: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-09-10T22:27:37.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55275 4.78 MB/sec execute 508 sec latency 2348.127 ms 2024-09-10T22:27:38.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55376 4.78 MB/sec execute 509 sec latency 273.631 ms 2024-09-10T22:27:39.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.535+0000 7f849da00640 1 -- 172.21.15.90:0/996267043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498104e10 msgr2=0x7f84981051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:39.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.535+0000 7f849da00640 1 --2- 172.21.15.90:0/996267043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498104e10 0x7f84981051f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f8488009900 tx=0x7f848802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.535+0000 7f849da00640 1 -- 172.21.15.90:0/996267043 shutdown_connections 2024-09-10T22:27:39.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.535+0000 7f849da00640 1 --2- 172.21.15.90:0/996267043 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f849810b1a0 0x7f84981119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.535+0000 7f849da00640 1 --2- 172.21.15.90:0/996267043 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f849810a820 0x7f849810ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.535+0000 7f849da00640 1 --2- 172.21.15.90:0/996267043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498104e10 0x7f84981051f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.535+0000 7f849da00640 1 -- 172.21.15.90:0/996267043 >> 172.21.15.90:0/996267043 conn(0x7f8498100840 msgr2=0x7f8498102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:39.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.535+0000 7f849da00640 1 -- 172.21.15.90:0/996267043 shutdown_connections 2024-09-10T22:27:39.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.535+0000 7f849da00640 1 -- 172.21.15.90:0/996267043 wait complete. 2024-09-10T22:27:39.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.537+0000 7f849da00640 1 Processor -- start 2024-09-10T22:27:39.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.537+0000 7f849da00640 1 -- start start 2024-09-10T22:27:39.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.538+0000 7f849da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f849810a820 0x7f849819b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:39.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.538+0000 7f849da00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f849810b1a0 0x7f849819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:39.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.538+0000 7f849da00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84981a0c70 0x7f84981a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:39.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.538+0000 7f849da00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8498114d60 con 0x7f849810a820 2024-09-10T22:27:39.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.538+0000 7f849da00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f8498114be0 con 0x7f849810b1a0 2024-09-10T22:27:39.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.538+0000 7f849da00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f8498114ee0 con 0x7f84981a0c70 2024-09-10T22:27:39.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.538+0000 7f848fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f849810b1a0 0x7f849819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:39.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.538+0000 7f848fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f849810b1a0 0x7f849819b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:59168/0 (socket says 172.21.15.90:59168) 2024-09-10T22:27:39.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.538+0000 7f848fe00640 1 -- 172.21.15.90:0/1643167468 learned_addr learned my addr 172.21.15.90:0/1643167468 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:39.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.539+0000 7f849d000640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84981a0c70 0x7f84981a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.539+0000 7f848fe00640 1 -- 172.21.15.90:0/1643167468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84981a0c70 msgr2=0x7f84981a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.539+0000 7f848fe00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84981a0c70 0x7f84981a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.539+0000 7f848fe00640 1 -- 172.21.15.90:0/1643167468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f849810a820 msgr2=0x7f849819b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.539+0000 7f848fe00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f849810a820 0x7f849819b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.539+0000 7f848fe00640 1 -- 172.21.15.90:0/1643167468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84981a1800 con 0x7f849810b1a0 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.539+0000 7f849d000640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84981a0c70 0x7f84981a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.539+0000 7f848fe00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f849810b1a0 0x7f849819b8d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f848000d950 tx=0x7f848000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.540+0000 7f8495600640 1 -- 172.21.15.90:0/1643167468 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480017070 con 0x7f849810b1a0 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.540+0000 7f8495600640 1 -- 172.21.15.90:0/1643167468 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8480009c30 con 0x7f849810b1a0 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.540+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8498077590 con 0x7f849810b1a0 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.540+0000 7f8495600640 1 -- 172.21.15.90:0/1643167468 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848000f620 con 0x7f849810b1a0 2024-09-10T22:27:39.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.540+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8498077b20 con 0x7f849810b1a0 2024-09-10T22:27:39.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.541+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84981051f0 con 0x7f849810b1a0 2024-09-10T22:27:39.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.545+0000 7f8495600640 1 -- 172.21.15.90:0/1643167468 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f8480005030 con 0x7f849810b1a0 2024-09-10T22:27:39.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.546+0000 7f8495600640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f847c0b5580 0x7f847c0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:39.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.547+0000 7f8495600640 1 -- 172.21.15.90:0/1643167468 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7f84800e5ff0 con 0x7f849810b1a0 2024-09-10T22:27:39.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.547+0000 7f8497e00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f847c0b5580 0x7f847c0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:39.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.548+0000 7f8495600640 1 -- 172.21.15.90:0/1643167468 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f84800ac820 con 0x7f849810b1a0 2024-09-10T22:27:39.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.549+0000 7f8497e00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f847c0b5580 0x7f847c0b7a40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8488009230 tx=0x7f8488000bc0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:39.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:39 smithi090 ceph-mon[30350]: pgmap v1166: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 134 op/s 2024-09-10T22:27:39.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:39 smithi154 ceph-mon[34495]: pgmap v1166: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 134 op/s 2024-09-10T22:27:39.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:39 smithi171 ceph-mon[34138]: pgmap v1166: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 134 op/s 2024-09-10T22:27:39.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55539 4.80 MB/sec execute 510 sec latency 87.363 ms 2024-09-10T22:27:39.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.968+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f849819c4c0 con 0x7f849810b1a0 2024-09-10T22:27:39.971 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:27:39.972 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":92,"btime":"2024-09-10T22:27:33:287798+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":92,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:27:32.691498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:27:39.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.969+0000 7f8495600640 1 -- 172.21.15.90:0/1643167468 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+11997 (secure 0 0 0) 0x7f8480009de0 con 0x7f849810b1a0 2024-09-10T22:27:39.972 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 92 2024-09-10T22:27:39.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f847c0b5580 msgr2=0x7f847c0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:39.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f847c0b5580 0x7f847c0b7a40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8488009230 tx=0x7f8488000bc0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f849810b1a0 msgr2=0x7f849819b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:39.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f849810b1a0 0x7f849819b8d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f848000d950 tx=0x7f848000de10 comp rx=0 tx=0).stop 2024-09-10T22:27:39.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 shutdown_connections 2024-09-10T22:27:39.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f847c0b5580 0x7f847c0b7a40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f84981a0c70 0x7f84981a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f849810b1a0 0x7f849819b8d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 --2- 172.21.15.90:0/1643167468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f849810a820 0x7f849819b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:39.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 >> 172.21.15.90:0/1643167468 conn(0x7f8498100840 msgr2=0x7f8498102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:39.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 shutdown_connections 2024-09-10T22:27:39.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:39.973+0000 7f849da00640 1 -- 172.21.15.90:0/1643167468 wait complete. 2024-09-10T22:27:40.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:40 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1643167468' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:40.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:40 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1643167468' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:40.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:40 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1643167468' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:40.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55668 4.80 MB/sec execute 511 sec latency 78.134 ms 2024-09-10T22:27:40.951 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0' 2024-09-10T22:27:40.952 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 138.85 --await-for 45.0 2024-09-10T22:27:41.536 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:27:41.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:41 smithi090 ceph-mon[30350]: pgmap v1167: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 151 op/s 2024-09-10T22:27:41.669 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:41 smithi154 ceph-mon[34495]: pgmap v1167: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 151 op/s 2024-09-10T22:27:41.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:41 smithi171 ceph-mon[34138]: pgmap v1167: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 151 op/s 2024-09-10T22:27:41.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55783 4.80 MB/sec execute 512 sec latency 88.121 ms 2024-09-10T22:27:42.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55898 4.81 MB/sec execute 513 sec latency 106.365 ms 2024-09-10T22:27:43.327 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:43 smithi090 ceph-mon[30350]: pgmap v1168: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.9 MiB/s wr, 112 op/s 2024-09-10T22:27:43.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:43 smithi154 ceph-mon[34495]: pgmap v1168: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.9 MiB/s wr, 112 op/s 2024-09-10T22:27:43.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:43 smithi171 ceph-mon[34138]: pgmap v1168: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 1.9 MiB/s wr, 112 op/s 2024-09-10T22:27:43.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55994 4.81 MB/sec execute 514 sec latency 168.827 ms 2024-09-10T22:27:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.404+0000 7ff277200640 1 -- 172.21.15.90:0/1864054691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff278102220 msgr2=0x7ff2781066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.404+0000 7ff277200640 1 --2- 172.21.15.90:0/1864054691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff278102220 0x7ff2781066f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff270009ca0 tx=0x7ff27002a8b0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.404+0000 7ff277200640 1 -- 172.21.15.90:0/1864054691 shutdown_connections 2024-09-10T22:27:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.404+0000 7ff277200640 1 --2- 172.21.15.90:0/1864054691 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff278106d40 0x7ff278107190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.404+0000 7ff277200640 1 --2- 172.21.15.90:0/1864054691 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff278102220 0x7ff2781066f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.404+0000 7ff277200640 1 --2- 172.21.15.90:0/1864054691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff278101900 0x7ff278101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.404+0000 7ff277200640 1 -- 172.21.15.90:0/1864054691 >> 172.21.15.90:0/1864054691 conn(0x7ff2780fd5b0 msgr2=0x7ff2780ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.404+0000 7ff277200640 1 -- 172.21.15.90:0/1864054691 shutdown_connections 2024-09-10T22:27:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.404+0000 7ff277200640 1 -- 172.21.15.90:0/1864054691 wait complete. 2024-09-10T22:27:44.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.406+0000 7ff277200640 1 Processor -- start 2024-09-10T22:27:44.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.406+0000 7ff277200640 1 -- start start 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.406+0000 7ff277200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff278101900 0x7ff27819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.406+0000 7ff277200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff278102220 0x7ff27819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.406+0000 7ff277200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff278106d40 0x7ff2781a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.406+0000 7ff277200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff2781190b0 con 0x7ff278102220 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.406+0000 7ff277200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff278118f30 con 0x7ff278101900 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.406+0000 7ff277200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff278119230 con 0x7ff278106d40 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.407+0000 7ff275e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff278101900 0x7ff27819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.407+0000 7ff275e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff278101900 0x7ff27819f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:59174/0 (socket says 172.21.15.90:59174) 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.407+0000 7ff275e00640 1 -- 172.21.15.90:0/3835865100 learned_addr learned my addr 172.21.15.90:0/3835865100 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.407+0000 7ff276800640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff278106d40 0x7ff2781a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.407+0000 7ff275400640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff278102220 0x7ff27819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.408+0000 7ff276800640 1 -- 172.21.15.90:0/3835865100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff278101900 msgr2=0x7ff27819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:44.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.408+0000 7ff276800640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff278101900 0x7ff27819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.408+0000 7ff276800640 1 -- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff278102220 msgr2=0x7ff27819faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.408+0000 7ff276800640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff278102220 0x7ff27819faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.408+0000 7ff276800640 1 -- 172.21.15.90:0/3835865100 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2781a5610 con 0x7ff278106d40 2024-09-10T22:27:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.408+0000 7ff276800640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff278106d40 0x7ff2781a4e40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff26800d680 tx=0x7ff26800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.408+0000 7ff26ea00640 1 -- 172.21.15.90:0/3835865100 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268017070 con 0x7ff278106d40 2024-09-10T22:27:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.409+0000 7ff277200640 1 -- 172.21.15.90:0/3835865100 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2781a5900 con 0x7ff278106d40 2024-09-10T22:27:44.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.409+0000 7ff277200640 1 -- 172.21.15.90:0/3835865100 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2781abfc0 con 0x7ff278106d40 2024-09-10T22:27:44.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.409+0000 7ff26ea00640 1 -- 172.21.15.90:0/3835865100 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff268009820 con 0x7ff278106d40 2024-09-10T22:27:44.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.409+0000 7ff26ea00640 1 -- 172.21.15.90:0/3835865100 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268004c60 con 0x7ff278106d40 2024-09-10T22:27:44.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.414+0000 7ff26ea00640 1 -- 172.21.15.90:0/3835865100 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7ff268010460 con 0x7ff278106d40 2024-09-10T22:27:44.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.420+0000 7ff26ea00640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff25c0b5600 0x7ff25c0b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.421+0000 7ff26ea00640 1 -- 172.21.15.90:0/3835865100 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7ff2680e6b30 con 0x7ff278106d40 2024-09-10T22:27:44.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.421+0000 7ff275e00640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff25c0b5600 0x7ff25c0b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.430+0000 7ff275e00640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff25c0b5600 0x7ff25c0b7ac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff2600058c0 tx=0x7ff26000a380 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:44.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.432+0000 7ff27f862740 1 Processor -- start 2024-09-10T22:27:44.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.432+0000 7ff27f862740 1 -- start start 2024-09-10T22:27:44.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.434+0000 7ff27f862740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebd6e10 0x55e1cebd5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.434+0000 7ff27f862740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebd5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.434+0000 7ff27f862740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebe16e0 0x55e1cebe3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.434+0000 7ff27f862740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e1ceb4aa90 con 0x55e1cebd6e10 2024-09-10T22:27:44.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.434+0000 7ff27f862740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e1ceb39ec0 con 0x55e1cebe16e0 2024-09-10T22:27:44.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.434+0000 7ff27f862740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e1ceb3dcd0 con 0x55e1cebd58b0 2024-09-10T22:27:44.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.434+0000 7ff276800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebe16e0 0x55e1cebe3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.434+0000 7ff276800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebe16e0 0x55e1cebe3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:59190/0 (socket says 172.21.15.90:59190) 2024-09-10T22:27:44.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.434+0000 7ff276800640 1 -- 172.21.15.90:0/2171827148 learned_addr learned my addr 172.21.15.90:0/2171827148 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:44.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.435+0000 7ff275400640 1 --2- 172.21.15.90:0/2171827148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebd5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.435+0000 7ff276800640 1 -- 172.21.15.90:0/2171827148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 msgr2=0x55e1cebd5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:44.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.435+0000 7ff276800640 1 --2- 172.21.15.90:0/2171827148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebd5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.435+0000 7ff276800640 1 -- 172.21.15.90:0/2171827148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebd6e10 msgr2=0x55e1cebd5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:44.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.435+0000 7ff276800640 1 --2- 172.21.15.90:0/2171827148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebd6e10 0x55e1cebd5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.435+0000 7ff276800640 1 -- 172.21.15.90:0/2171827148 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ceadf9d0 con 0x55e1cebe16e0 2024-09-10T22:27:44.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.435+0000 7ff276800640 1 --2- 172.21.15.90:0/2171827148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebe16e0 0x55e1cebe3ad0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2680a9150 tx=0x7ff2680ac690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c08c6f6e85a11770 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:44.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.435+0000 7ff275400640 1 --2- 172.21.15.90:0/2171827148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebd5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:27:44.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.436+0000 7ff267000640 1 -- 172.21.15.90:0/2171827148 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268020070 con 0x55e1cebe16e0 2024-09-10T22:27:44.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.436+0000 7ff267000640 1 -- 172.21.15.90:0/2171827148 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2680acb80 con 0x55e1cebe16e0 2024-09-10T22:27:44.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.436+0000 7ff267000640 1 -- 172.21.15.90:0/2171827148 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2680b13e0 con 0x55e1cebe16e0 2024-09-10T22:27:44.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.437+0000 7ff27f862740 1 -- 172.21.15.90:0/2171827148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebe16e0 msgr2=0x55e1cebe3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:44.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.437+0000 7ff27f862740 1 --2- 172.21.15.90:0/2171827148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebe16e0 0x55e1cebe3ad0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2680a9150 tx=0x7ff2680ac690 comp rx=0 tx=0).stop 2024-09-10T22:27:44.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.438+0000 7ff27f862740 1 -- 172.21.15.90:0/2171827148 shutdown_connections 2024-09-10T22:27:44.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.438+0000 7ff27f862740 1 --2- 172.21.15.90:0/2171827148 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebe16e0 0x55e1cebe3ad0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.438+0000 7ff27f862740 1 --2- 172.21.15.90:0/2171827148 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebd5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.438+0000 7ff27f862740 1 --2- 172.21.15.90:0/2171827148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebd6e10 0x55e1cebd5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.438+0000 7ff27f862740 1 -- 172.21.15.90:0/2171827148 >> 172.21.15.90:0/2171827148 conn(0x55e1ceb9f6f0 msgr2=0x55e1cebd31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:44.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.439+0000 7ff27f862740 1 -- 172.21.15.90:0/2171827148 shutdown_connections 2024-09-10T22:27:44.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.439+0000 7ff27f862740 1 -- 172.21.15.90:0/2171827148 wait complete. 2024-09-10T22:27:44.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.440+0000 7ff27f862740 1 -- start start 2024-09-10T22:27:44.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.440+0000 7ff27f862740 1 Processor -- start 2024-09-10T22:27:44.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.441+0000 7ff27f862740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebde610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.441+0000 7ff27f862740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 0x55e1cebdeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.441+0000 7ff27f862740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebdf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.442+0000 7ff275e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebde610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.442+0000 7ff275e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebde610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:55878/0 (socket says 172.21.15.90:55878) 2024-09-10T22:27:44.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.443+0000 7ff276800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebdf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.443+0000 7ff276800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebdf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54154/0 (socket says 172.21.15.90:54154) 2024-09-10T22:27:44.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.443+0000 7ff275400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 0x55e1cebdeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.443+0000 7ff275400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 0x55e1cebdeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:59194/0 (socket says 172.21.15.90:59194) 2024-09-10T22:27:44.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.443+0000 7ff275e00640 1 -- 172.21.15.90:0/2921451568 learned_addr learned my addr 172.21.15.90:0/2921451568 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:44.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.443+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e1ce99b7b0 con 0x55e1cebe16e0 2024-09-10T22:27:44.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.444+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55e1ceade650 con 0x55e1cebd6e10 2024-09-10T22:27:44.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.444+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55e1cebe44a0 con 0x55e1cebd58b0 2024-09-10T22:27:44.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.444+0000 7ff275e00640 1 -- 172.21.15.90:0/2921451568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 msgr2=0x55e1cebdeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:44.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.444+0000 7ff275e00640 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 0x55e1cebdeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.444+0000 7ff275e00640 1 -- 172.21.15.90:0/2921451568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 msgr2=0x55e1cebdf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:44.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.444+0000 7ff275e00640 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebdf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.444+0000 7ff275e00640 1 -- 172.21.15.90:0/2921451568 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ce8a7ba0 con 0x55e1cebd58b0 2024-09-10T22:27:44.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.445+0000 7ff275e00640 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebde610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff260010af0 tx=0x7ff2600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7300d7e8ace6b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:44.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.445+0000 7ff25b400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:27:44.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.445+0000 7ff27f862740 10 client.0 _ll_register_callbacks cb 0x55e1ce8450a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:27:44.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.445+0000 7ff25b400640 1 -- 172.21.15.90:0/2921451568 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260016070 con 0x55e1cebd58b0 2024-09-10T22:27:44.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.446+0000 7ff25b400640 1 -- 172.21.15.90:0/2921451568 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7ff260027480 con 0x55e1cebd58b0 2024-09-10T22:27:44.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.446+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2680acb80 con 0x55e1cebd58b0 2024-09-10T22:27:44.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.446+0000 7ff25b400640 1 -- 172.21.15.90:0/2921451568 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff260030050 con 0x55e1cebd58b0 2024-09-10T22:27:44.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.447+0000 7ff27f862740 10 client.26164 fetch_fsmap learned FSMap version 92 2024-09-10T22:27:44.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.447+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({fsmap=92}) -- 0x55e1cebefdd0 con 0x55e1cebd58b0 2024-09-10T22:27:44.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.447+0000 7ff25b400640 1 -- 172.21.15.90:0/2921451568 <== mon.1 v2:172.21.15.171:3300/0 4 ==== fsmap(e 92) ==== 8398+0+0 (secure 0 0 0) 0x7ff260027680 con 0x55e1cebd58b0 2024-09-10T22:27:44.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.447+0000 7ff27f862740 10 client.26164 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T22:27:44.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.447+0000 7ff27f862740 10 client.26164 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:27:44.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.447+0000 7ff27f862740 20 client.26164 populate_metadata read hostname 'smithi090' 2024-09-10T22:27:44.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.447+0000 7ff27f862740 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55e1cebf1120 0x55e1cebf35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.447+0000 7ff27f862740 4 client.26164 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:27:44.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.447+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1ceb4c9e0 con 0x55e1cebf1120 2024-09-10T22:27:44.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.448+0000 7ff275400640 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55e1cebf1120 0x55e1cebf35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.449+0000 7ff275400640 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55e1cebf1120 0x55e1cebf35c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9824b00640b09cfc server_cookie=ca76c537cd43c8b5 in_seq=0 out_seq=0 2024-09-10T22:27:44.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.449+0000 7ff25b400640 10 client.26164 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:44.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.450+0000 7ff25b400640 1 -- 172.21.15.90:0/2921451568 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e1ceb4c9e0 con 0x55e1cebf1120 2024-09-10T22:27:44.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.450+0000 7ff25b400640 10 client.26164 handle_command_reply: tid=0 2024-09-10T22:27:44.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.450+0000 7ff27f862740 1 client.26164 shutdown 2024-09-10T22:27:44.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.450+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 msgr2=0x55e1cebde610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:44.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.450+0000 7ff27f862740 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebde610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff260010af0 tx=0x7ff2600046c0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.451+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 shutdown_connections 2024-09-10T22:27:44.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.451+0000 7ff27f862740 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55e1cebf1120 0x55e1cebf35c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.451+0000 7ff27f862740 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebdf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.451+0000 7ff25b400640 0 client.26164 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:44.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.451+0000 7ff27f862740 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 0x55e1cebdeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.451+0000 7ff27f862740 1 --2- 172.21.15.90:0/2921451568 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebde610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.451+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 >> 172.21.15.90:0/2921451568 conn(0x55e1ceb9f6f0 msgr2=0x55e1cebd8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:44.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.453+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 shutdown_connections 2024-09-10T22:27:44.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.453+0000 7ff27f862740 1 -- 172.21.15.90:0/2921451568 wait complete. 2024-09-10T22:27:44.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.453+0000 7ff27f862740 20 client.26164 trim_cache size 0 max 16384 2024-09-10T22:27:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.491+0000 7ff27f862740 1 Processor -- start 2024-09-10T22:27:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.491+0000 7ff27f862740 1 -- start start 2024-09-10T22:27:44.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff27f862740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd58b0 0x55e1cebd3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff27f862740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebd6e10 0x55e1cebece00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff27f862740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebe16e0 0x55e1cebdb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff27f862740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff25400ff50 con 0x55e1cebd6e10 2024-09-10T22:27:44.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff27f862740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff254011be0 con 0x55e1cebd58b0 2024-09-10T22:27:44.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff27f862740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff2540106e0 con 0x55e1cebe16e0 2024-09-10T22:27:44.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff276800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebe16e0 0x55e1cebdb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff275e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd58b0 0x55e1cebd3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff276800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebe16e0 0x55e1cebdb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:55884/0 (socket says 172.21.15.90:55884) 2024-09-10T22:27:44.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.492+0000 7ff276800640 1 -- 172.21.15.90:0/4255765393 learned_addr learned my addr 172.21.15.90:0/4255765393 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:44.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.493+0000 7ff275400640 1 --2- 172.21.15.90:0/4255765393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebd6e10 0x55e1cebece00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.493+0000 7ff276800640 1 -- 172.21.15.90:0/4255765393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd58b0 msgr2=0x55e1cebd3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:44.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.493+0000 7ff276800640 1 --2- 172.21.15.90:0/4255765393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd58b0 0x55e1cebd3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.493+0000 7ff276800640 1 -- 172.21.15.90:0/4255765393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebd6e10 msgr2=0x55e1cebece00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:44.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.493+0000 7ff276800640 1 --2- 172.21.15.90:0/4255765393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebd6e10 0x55e1cebece00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.493+0000 7ff276800640 1 -- 172.21.15.90:0/4255765393 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1cebd36d0 con 0x55e1cebe16e0 2024-09-10T22:27:44.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.493+0000 7ff276800640 1 --2- 172.21.15.90:0/4255765393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebe16e0 0x55e1cebdb820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2680a9df0 tx=0x7ff2680ac9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed380e6441cbb98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:44.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.494+0000 7ff266600640 1 -- 172.21.15.90:0/4255765393 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268020040 con 0x55e1cebe16e0 2024-09-10T22:27:44.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.494+0000 7ff266600640 1 -- 172.21.15.90:0/4255765393 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2680e42d0 con 0x55e1cebe16e0 2024-09-10T22:27:44.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.494+0000 7ff266600640 1 -- 172.21.15.90:0/4255765393 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2680a8810 con 0x55e1cebe16e0 2024-09-10T22:27:44.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.495+0000 7ff27f862740 1 -- 172.21.15.90:0/4255765393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebe16e0 msgr2=0x55e1cebdb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:44.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.495+0000 7ff27f862740 1 --2- 172.21.15.90:0/4255765393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebe16e0 0x55e1cebdb820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff2680a9df0 tx=0x7ff2680ac9e0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.495+0000 7ff27f862740 1 -- 172.21.15.90:0/4255765393 shutdown_connections 2024-09-10T22:27:44.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.495+0000 7ff27f862740 1 --2- 172.21.15.90:0/4255765393 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebe16e0 0x55e1cebdb820 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.495+0000 7ff27f862740 1 --2- 172.21.15.90:0/4255765393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebd6e10 0x55e1cebece00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.495+0000 7ff27f862740 1 --2- 172.21.15.90:0/4255765393 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd58b0 0x55e1cebd3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.495+0000 7ff27f862740 1 -- 172.21.15.90:0/4255765393 >> 172.21.15.90:0/4255765393 conn(0x55e1ceb9f6f0 msgr2=0x55e1cebec6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:44.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.496+0000 7ff27f862740 1 -- 172.21.15.90:0/4255765393 shutdown_connections 2024-09-10T22:27:44.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.496+0000 7ff27f862740 1 -- 172.21.15.90:0/4255765393 wait complete. 2024-09-10T22:27:44.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.496+0000 7ff27f862740 1 -- start start 2024-09-10T22:27:44.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.497+0000 7ff27f862740 1 Processor -- start 2024-09-10T22:27:44.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.497+0000 7ff27f862740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebe9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff27f862740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 0x55e1cebe6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff27f862740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebe6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff27f862740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff2540114b0 con 0x55e1cebe16e0 2024-09-10T22:27:44.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff27f862740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7ff254010db0 con 0x55e1cebd6e10 2024-09-10T22:27:44.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff27f862740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7ff25400df50 con 0x55e1cebd58b0 2024-09-10T22:27:44.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff27f862740 10 client.0 _ll_register_callbacks cb 0x7ff26000a8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:27:44.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff275e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebe9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff275e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebe9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:55886/0 (socket says 172.21.15.90:55886) 2024-09-10T22:27:44.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff275e00640 1 -- 172.21.15.90:0/2022534288 learned_addr learned my addr 172.21.15.90:0/2022534288 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:44.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff275e00640 1 -- 172.21.15.90:0/2022534288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 msgr2=0x55e1cebe6410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:27:44.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff275e00640 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 0x55e1cebe6410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.498+0000 7ff275e00640 1 -- 172.21.15.90:0/2022534288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 msgr2=0x55e1cebe6950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:27:44.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.499+0000 7ff276800640 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebe6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.499+0000 7ff275e00640 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebe6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.499+0000 7ff275e00640 1 -- 172.21.15.90:0/2022534288 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1cebe7150 con 0x55e1cebd58b0 2024-09-10T22:27:44.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.499+0000 7ff276800640 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebe6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:27:44.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.500+0000 7ff275e00640 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebe9ee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff260006010 tx=0x7ff260016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8370145058cf706 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:44.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.500+0000 7ff25b400640 10 client.0 ms_handle_connect on v2:172.21.15.171:3300/0 2024-09-10T22:27:44.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.500+0000 7ff25b400640 1 -- 172.21.15.90:0/2022534288 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26002d040 con 0x55e1cebd58b0 2024-09-10T22:27:44.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.501+0000 7ff27f862740 1 -- 172.21.15.90:0/2022534288 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2680e42d0 con 0x55e1cebd58b0 2024-09-10T22:27:44.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.501+0000 7ff25b400640 1 -- 172.21.15.90:0/2022534288 <== mon.1 v2:172.21.15.171:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7ff26001be40 con 0x55e1cebd58b0 2024-09-10T22:27:44.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.501+0000 7ff25b400640 1 -- 172.21.15.90:0/2022534288 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff260005d20 con 0x55e1cebd58b0 2024-09-10T22:27:44.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.501+0000 7ff27f862740 10 client.26170 fetch_fsmap learned FSMap version 92 2024-09-10T22:27:44.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.501+0000 7ff27f862740 1 -- 172.21.15.90:0/2022534288 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x55e1cec5f150 con 0x55e1cebd58b0 2024-09-10T22:27:44.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.502+0000 7ff25b400640 1 -- 172.21.15.90:0/2022534288 <== mon.1 v2:172.21.15.171:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2680e42d0 con 0x55e1cebd58b0 2024-09-10T22:27:44.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.502+0000 7ff25b400640 1 -- 172.21.15.90:0/2022534288 <== mon.1 v2:172.21.15.171:3300/0 5 ==== fsmap(e 92) ==== 8398+0+0 (secure 0 0 0) 0x7ff260005460 con 0x55e1cebd58b0 2024-09-10T22:27:44.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.502+0000 7ff27f862740 10 client.26170 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T22:27:44.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.502+0000 7ff27f862740 10 client.26170 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:27:44.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.502+0000 7ff27f862740 20 client.26170 populate_metadata read hostname 'smithi090' 2024-09-10T22:27:44.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.502+0000 7ff27f862740 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55e1cec5fa20 0x55e1cec61e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:44.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.502+0000 7ff27f862740 4 client.26170 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}] 2024-09-10T22:27:44.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.502+0000 7ff27f862740 1 -- 172.21.15.90:0/2022534288 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.85, "await_for": 45.0}) -- 0x55e1ceb48490 con 0x55e1cec5fa20 2024-09-10T22:27:44.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.502+0000 7ff275400640 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55e1cec5fa20 0x55e1cec61e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:44.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.503+0000 7ff25b400640 10 client.26170 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:44.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.504+0000 7ff275400640 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55e1cec5fa20 0x55e1cec61e40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e77bc662404f3de8 server_cookie=326cf16a5307ff4f in_seq=0 out_seq=0 2024-09-10T22:27:44.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff25b400640 1 -- 172.21.15.90:0/2022534288 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e1ceb48490 con 0x55e1cec5fa20 2024-09-10T22:27:44.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff25b400640 10 client.26170 handle_command_reply: tid=0 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff27f862740 1 client.26170 shutdown 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff27f862740 1 -- 172.21.15.90:0/2022534288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 msgr2=0x55e1cebe9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff27f862740 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebe9ee0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff260006010 tx=0x7ff260016000 comp rx=0 tx=0).stop 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff27f862740 1 -- 172.21.15.90:0/2022534288 shutdown_connections 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff27f862740 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55e1cec5fa20 0x55e1cec61e40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff27f862740 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1cebe16e0 0x55e1cebe6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff27f862740 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55e1cebd6e10 0x55e1cebe6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff27f862740 1 --2- 172.21.15.90:0/2022534288 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55e1cebd58b0 0x55e1cebe9ee0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.884+0000 7ff27f862740 1 -- 172.21.15.90:0/2022534288 >> 172.21.15.90:0/2022534288 conn(0x55e1ceb9f6f0 msgr2=0x55e1cebecf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.885+0000 7ff25b400640 0 client.26170 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.885+0000 7ff27f862740 1 -- 172.21.15.90:0/2022534288 shutdown_connections 2024-09-10T22:27:44.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.885+0000 7ff27f862740 1 -- 172.21.15.90:0/2022534288 wait complete. 2024-09-10T22:27:44.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.885+0000 7ff27f862740 20 client.26170 trim_cache size 0 max 16384 2024-09-10T22:27:44.888 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:27:44.888 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 92, 2024-09-10T22:27:44.888 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:27:44.888 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 74, 2024-09-10T22:27:44.888 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:27:44.888 INFO:teuthology.orchestra.run.smithi090.stdout: "7387791c": { 2024-09-10T22:27:44.888 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 74, 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 138.8, 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:27:44.889 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:44.890 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:44.890 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:44.890 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:44.890 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:44.890 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:27:44.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 -- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff25c0b5600 msgr2=0x7ff25c0b7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:44.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff25c0b5600 0x7ff25c0b7ac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff2600058c0 tx=0x7ff26000a380 comp rx=0 tx=0).stop 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 -- 172.21.15.90:0/3835865100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff278106d40 msgr2=0x7ff2781a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff278106d40 0x7ff2781a4e40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff26800d680 tx=0x7ff26800db40 comp rx=0 tx=0).stop 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 -- 172.21.15.90:0/3835865100 shutdown_connections 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7ff25c0b5600 0x7ff25c0b7ac0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7ff278106d40 0x7ff2781a4e40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff278102220 0x7ff27819faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 --2- 172.21.15.90:0/3835865100 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7ff278101900 0x7ff27819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.886+0000 7ff277200640 1 -- 172.21.15.90:0/3835865100 >> 172.21.15.90:0/3835865100 conn(0x7ff2780fd5b0 msgr2=0x7ff278104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.887+0000 7ff277200640 1 -- 172.21.15.90:0/3835865100 shutdown_connections 2024-09-10T22:27:44.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:44.887+0000 7ff277200640 1 -- 172.21.15.90:0/3835865100 wait complete. 2024-09-10T22:27:44.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.81 MB/sec execute 515 sec latency 420.726 ms 2024-09-10T22:27:45.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:45 smithi090 ceph-mon[30350]: pgmap v1169: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 144 op/s 2024-09-10T22:27:45.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:45 smithi154 ceph-mon[34495]: pgmap v1169: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 144 op/s 2024-09-10T22:27:45.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:45 smithi171 ceph-mon[34138]: pgmap v1169: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 144 op/s 2024-09-10T22:27:45.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.80 MB/sec execute 516 sec latency 1420.803 ms 2024-09-10T22:27:45.922 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7387791c', quiesce took 8.9 seconds. Will release after: 9.950000000000001 2024-09-10T22:27:46.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.79 MB/sec execute 517 sec latency 2420.960 ms 2024-09-10T22:27:47.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:47 smithi090 ceph-mon[30350]: pgmap v1170: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 154 op/s 2024-09-10T22:27:47.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:47 smithi154 ceph-mon[34495]: pgmap v1170: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 154 op/s 2024-09-10T22:27:47.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:47 smithi171 ceph-mon[34138]: pgmap v1170: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 154 op/s 2024-09-10T22:27:47.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.78 MB/sec execute 518 sec latency 3421.107 ms 2024-09-10T22:27:48.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.78 MB/sec execute 519 sec latency 4421.230 ms 2024-09-10T22:27:49.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:49 smithi090 ceph-mon[30350]: pgmap v1171: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 166 op/s 2024-09-10T22:27:49.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:49 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:49.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:49 smithi154 ceph-mon[34495]: pgmap v1171: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 166 op/s 2024-09-10T22:27:49.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:49 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:49.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:49 smithi171 ceph-mon[34138]: pgmap v1171: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 166 op/s 2024-09-10T22:27:49.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:49 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:49.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.77 MB/sec execute 520 sec latency 5421.386 ms 2024-09-10T22:27:50.649 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:50 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:50.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:50 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:50.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:50 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:50.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.76 MB/sec execute 521 sec latency 6421.509 ms 2024-09-10T22:27:51.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:51 smithi154 ceph-mon[34495]: pgmap v1172: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 179 op/s 2024-09-10T22:27:51.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:51 smithi171 ceph-mon[34138]: pgmap v1172: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 179 op/s 2024-09-10T22:27:51.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:51 smithi090 ceph-mon[30350]: pgmap v1172: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.6 MiB/s wr, 179 op/s 2024-09-10T22:27:51.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.75 MB/sec execute 522 sec latency 7421.647 ms 2024-09-10T22:27:52.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.74 MB/sec execute 523 sec latency 8421.756 ms 2024-09-10T22:27:53.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:53 smithi154 ceph-mon[34495]: pgmap v1173: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 146 op/s 2024-09-10T22:27:53.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:53 smithi171 ceph-mon[34138]: pgmap v1173: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 146 op/s 2024-09-10T22:27:53.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:53 smithi090 ceph-mon[30350]: pgmap v1173: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 146 op/s 2024-09-10T22:27:53.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.73 MB/sec execute 524 sec latency 9421.921 ms 2024-09-10T22:27:54.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.72 MB/sec execute 525 sec latency 10422.053 ms 2024-09-10T22:27:55.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:55 smithi154 ceph-mon[34495]: pgmap v1174: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-09-10T22:27:55.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:55 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:55.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:55 smithi171 ceph-mon[34138]: pgmap v1174: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-09-10T22:27:55.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:55 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:55 smithi090 ceph-mon[30350]: pgmap v1174: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.6 MiB/s wr, 145 op/s 2024-09-10T22:27:55.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:55 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:27:55.873 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:27:55.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.71 MB/sec execute 526 sec latency 11422.149 ms 2024-09-10T22:27:56.254 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:27:56.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.70 MB/sec execute 527 sec latency 12422.298 ms 2024-09-10T22:27:57.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.121+0000 7f3d8e400640 1 -- 172.21.15.90:0/200484859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 msgr2=0x7f3d8810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:57.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.121+0000 7f3d8e400640 1 --2- 172.21.15.90:0/200484859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 0x7f3d8810f950 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3d80009ca0 tx=0x7f3d8002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.121+0000 7f3d8e400640 1 -- 172.21.15.90:0/200484859 shutdown_connections 2024-09-10T22:27:57.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.121+0000 7f3d8e400640 1 --2- 172.21.15.90:0/200484859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 0x7f3d8810f950 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.121+0000 7f3d8e400640 1 --2- 172.21.15.90:0/200484859 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d881087d0 0x7f3d88108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.121+0000 7f3d8e400640 1 --2- 172.21.15.90:0/200484859 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d88102dc0 0x7f3d881031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.121+0000 7f3d8e400640 1 -- 172.21.15.90:0/200484859 >> 172.21.15.90:0/200484859 conn(0x7f3d880fe830 msgr2=0x7f3d88100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:57.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.121+0000 7f3d8e400640 1 -- 172.21.15.90:0/200484859 shutdown_connections 2024-09-10T22:27:57.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.121+0000 7f3d8e400640 1 -- 172.21.15.90:0/200484859 wait complete. 2024-09-10T22:27:57.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.122+0000 7f3d8e400640 1 Processor -- start 2024-09-10T22:27:57.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.122+0000 7f3d8e400640 1 -- start start 2024-09-10T22:27:57.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.122+0000 7f3d8e400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d88102dc0 0x7f3d880754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:57.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.122+0000 7f3d8e400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d881087d0 0x7f3d880759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:57.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.122+0000 7f3d8e400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 0x7f3d88075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:57.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.122+0000 7f3d8e400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88112d10 con 0x7f3d88109150 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.122+0000 7f3d8e400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88112b90 con 0x7f3d881087d0 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.122+0000 7f3d8e400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3d88112e90 con 0x7f3d88102dc0 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d87e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 0x7f3d88075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d87e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 0x7f3d88075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59972/0 (socket says 172.21.15.90:59972) 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d87e00640 1 -- 172.21.15.90:0/1454905441 learned_addr learned my addr 172.21.15.90:0/1454905441 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d87400640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d88102dc0 0x7f3d880754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d86a00640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d881087d0 0x7f3d880759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d87e00640 1 -- 172.21.15.90:0/1454905441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d88102dc0 msgr2=0x7f3d880754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d87e00640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d88102dc0 0x7f3d880754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d87e00640 1 -- 172.21.15.90:0/1454905441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d881087d0 msgr2=0x7f3d880759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d87e00640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d881087d0 0x7f3d880759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.123+0000 7f3d87e00640 1 -- 172.21.15.90:0/1454905441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d881a7600 con 0x7f3d88109150 2024-09-10T22:27:57.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.124+0000 7f3d87e00640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 0x7f3d88075f20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3d80031040 tx=0x7f3d80003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:57.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.124+0000 7f3d7fe00640 1 -- 172.21.15.90:0/1454905441 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80032c90 con 0x7f3d88109150 2024-09-10T22:27:57.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.124+0000 7f3d7fe00640 1 -- 172.21.15.90:0/1454905441 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3d80032e30 con 0x7f3d88109150 2024-09-10T22:27:57.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.124+0000 7f3d7fe00640 1 -- 172.21.15.90:0/1454905441 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d80030b30 con 0x7f3d88109150 2024-09-10T22:27:57.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.124+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d881a7890 con 0x7f3d88109150 2024-09-10T22:27:57.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.124+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d881a7dd0 con 0x7f3d88109150 2024-09-10T22:27:57.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.125+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d881031a0 con 0x7f3d88109150 2024-09-10T22:27:57.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.129+0000 7f3d7fe00640 1 -- 172.21.15.90:0/1454905441 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3d80030cd0 con 0x7f3d88109150 2024-09-10T22:27:57.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.130+0000 7f3d7fe00640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d6c0b5450 0x7f3d6c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:57.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.131+0000 7f3d7fe00640 1 -- 172.21.15.90:0/1454905441 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7f3d80107610 con 0x7f3d88109150 2024-09-10T22:27:57.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.131+0000 7f3d87400640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d6c0b5450 0x7f3d6c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:57.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.131+0000 7f3d87400640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d6c0b5450 0x7f3d6c0b7910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800bbe0 tx=0x7f3d7800b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:57.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.132+0000 7f3d7fe00640 1 -- 172.21.15.90:0/1454905441 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3d800cdd60 con 0x7f3d88109150 2024-09-10T22:27:57.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.353+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d880767a0 con 0x7f3d88109150 2024-09-10T22:27:57.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.354+0000 7f3d7fe00640 1 -- 172.21.15.90:0/1454905441 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+11992 (secure 0 0 0) 0x7f3d880767a0 con 0x7f3d88109150 2024-09-10T22:27:57.356 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:27:57.356 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":95,"btime":"2024-09-10T22:27:54:365111+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":95,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:27:53.798502+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:27:57.356 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 95 2024-09-10T22:27:57.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d6c0b5450 msgr2=0x7f3d6c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:57.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d6c0b5450 0x7f3d6c0b7910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800bbe0 tx=0x7f3d7800b390 comp rx=0 tx=0).stop 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 msgr2=0x7f3d88075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 0x7f3d88075f20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3d80031040 tx=0x7f3d80003f80 comp rx=0 tx=0).stop 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 shutdown_connections 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3d6c0b5450 0x7f3d6c0b7910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d88109150 0x7f3d88075f20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3d881087d0 0x7f3d880759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 --2- 172.21.15.90:0/1454905441 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3d88102dc0 0x7f3d880754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 >> 172.21.15.90:0/1454905441 conn(0x7f3d880fe830 msgr2=0x7f3d88100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 shutdown_connections 2024-09-10T22:27:57.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:57.357+0000 7f3d8e400640 1 -- 172.21.15.90:0/1454905441 wait complete. 2024-09-10T22:27:57.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:57 smithi090 ceph-mon[30350]: pgmap v1175: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-10T22:27:57.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:57 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1454905441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:57.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:57 smithi154 ceph-mon[34495]: pgmap v1175: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-10T22:27:57.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:57 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1454905441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:57.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:57 smithi171 ceph-mon[34138]: pgmap v1175: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-10T22:27:57.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:57 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1454905441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:27:57.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.69 MB/sec execute 528 sec latency 13422.423 ms 2024-09-10T22:27:58.307 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 7387791c --release --await' 2024-09-10T22:27:58.308 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 7387791c --release --await 2024-09-10T22:27:58.719 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:27:58.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56081 4.69 MB/sec execute 529 sec latency 14422.501 ms 2024-09-10T22:27:59.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.466+0000 7f52c8000640 1 -- 172.21.15.90:0/3211760226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 msgr2=0x7f52c01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.466+0000 7f52c8000640 1 --2- 172.21.15.90:0/3211760226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 0x7f52c01031a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009ca0 tx=0x7f52b402a8b0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.466+0000 7f52c8000640 1 -- 172.21.15.90:0/3211760226 shutdown_connections 2024-09-10T22:27:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.466+0000 7f52c8000640 1 --2- 172.21.15.90:0/3211760226 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c0109150 0x7f52c010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.466+0000 7f52c8000640 1 --2- 172.21.15.90:0/3211760226 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c01087d0 0x7f52c0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.466+0000 7f52c8000640 1 --2- 172.21.15.90:0/3211760226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 0x7f52c01031a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.466+0000 7f52c8000640 1 -- 172.21.15.90:0/3211760226 >> 172.21.15.90:0/3211760226 conn(0x7f52c00fe830 msgr2=0x7f52c0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.466+0000 7f52c8000640 1 -- 172.21.15.90:0/3211760226 shutdown_connections 2024-09-10T22:27:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.466+0000 7f52c8000640 1 -- 172.21.15.90:0/3211760226 wait complete. 2024-09-10T22:27:59.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.467+0000 7f52c8000640 1 Processor -- start 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.467+0000 7f52c8000640 1 -- start start 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.467+0000 7f52c8000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 0x7f52c01991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.467+0000 7f52c8000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c01087d0 0x7f52c0199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.467+0000 7f52c8000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c0109150 0x7f52c019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.467+0000 7f52c8000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0112d10 con 0x7f52c0102dc0 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.467+0000 7f52c8000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0112b90 con 0x7f52c01087d0 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.467+0000 7f52c8000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0112e90 con 0x7f52c0109150 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.467+0000 7f52c5600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 0x7f52c01991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c5600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 0x7f52c01991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39230/0 (socket says 172.21.15.90:39230) 2024-09-10T22:27:59.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c5600640 1 -- 172.21.15.90:0/3523131244 learned_addr learned my addr 172.21.15.90:0/3523131244 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c6000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c0109150 0x7f52c019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c4c00640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c01087d0 0x7f52c0199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c5600640 1 -- 172.21.15.90:0/3523131244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c0109150 msgr2=0x7f52c019eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c5600640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c0109150 0x7f52c019eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c5600640 1 -- 172.21.15.90:0/3523131244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c01087d0 msgr2=0x7f52c0199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c5600640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c01087d0 0x7f52c0199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c5600640 1 -- 172.21.15.90:0/3523131244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c019f270 con 0x7f52c0102dc0 2024-09-10T22:27:59.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.468+0000 7f52c5600640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 0x7f52c01991c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005e30 tx=0x7f52b40039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.469+0000 7f52be000640 1 -- 172.21.15.90:0/3523131244 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4004040 con 0x7f52c0102dc0 2024-09-10T22:27:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.469+0000 7f52be000640 1 -- 172.21.15.90:0/3523131244 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b4040070 con 0x7f52c0102dc0 2024-09-10T22:27:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.469+0000 7f52c8000640 1 -- 172.21.15.90:0/3523131244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c019f500 con 0x7f52c0102dc0 2024-09-10T22:27:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.469+0000 7f52be000640 1 -- 172.21.15.90:0/3523131244 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4045b90 con 0x7f52c0102dc0 2024-09-10T22:27:59.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.469+0000 7f52c8000640 1 -- 172.21.15.90:0/3523131244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c01a5c20 con 0x7f52c0102dc0 2024-09-10T22:27:59.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.472+0000 7f52cabdb740 1 Processor -- start 2024-09-10T22:27:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52cabdb740 1 -- start start 2024-09-10T22:27:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e911400 0x55853e9117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e91d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e91dcb0 0x55853e9200a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52cabdb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55853e884290 con 0x55853e911d20 2024-09-10T22:27:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52cabdb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55853e8736c0 con 0x55853e91dcb0 2024-09-10T22:27:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52cabdb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55853e8774d0 con 0x55853e911400 2024-09-10T22:27:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52c4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e91dcb0 0x55853e9200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52c6000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e911400 0x55853e9117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52c4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e91dcb0 0x55853e9200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:33918/0 (socket says 172.21.15.90:33918) 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.473+0000 7f52c4c00640 1 -- 172.21.15.90:0/101289459 learned_addr learned my addr 172.21.15.90:0/101289459 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.474+0000 7f52c6000640 1 -- 172.21.15.90:0/101289459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e91dcb0 msgr2=0x55853e9200a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.474+0000 7f52c6000640 1 --2- 172.21.15.90:0/101289459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e91dcb0 0x55853e9200a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.474+0000 7f52c6000640 1 -- 172.21.15.90:0/101289459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 msgr2=0x55853e91d770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.474+0000 7f52c6000640 1 --2- 172.21.15.90:0/101289459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e91d770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.474+0000 7f52c6000640 1 -- 172.21.15.90:0/101289459 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853e800460 con 0x55853e911400 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.474+0000 7f52c4c00640 1 --2- 172.21.15.90:0/101289459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e91dcb0 0x55853e9200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.474+0000 7f52be000640 1 -- 172.21.15.90:0/3523131244 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f52b404d030 con 0x7f52c0102dc0 2024-09-10T22:27:59.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.474+0000 7f52c6000640 1 --2- 172.21.15.90:0/101289459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e911400 0x55853e9117e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f52b800aad0 tx=0x7f52b800af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8150336e6b8277fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.475+0000 7f52b2600640 1 -- 172.21.15.90:0/101289459 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8005020 con 0x55853e911400 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.475+0000 7f52b2600640 1 -- 172.21.15.90:0/101289459 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b8005650 con 0x55853e911400 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.475+0000 7f52b2600640 1 -- 172.21.15.90:0/101289459 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8018640 con 0x55853e911400 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.475+0000 7f52cabdb740 1 -- 172.21.15.90:0/101289459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e911400 msgr2=0x55853e9117e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.475+0000 7f52cabdb740 1 --2- 172.21.15.90:0/101289459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e911400 0x55853e9117e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f52b800aad0 tx=0x7f52b800af90 comp rx=0 tx=0).stop 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52cabdb740 1 -- 172.21.15.90:0/101289459 shutdown_connections 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52cabdb740 1 --2- 172.21.15.90:0/101289459 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e91dcb0 0x55853e9200a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52cabdb740 1 --2- 172.21.15.90:0/101289459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e91d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52cabdb740 1 --2- 172.21.15.90:0/101289459 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e911400 0x55853e9117e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52cabdb740 1 -- 172.21.15.90:0/101289459 >> 172.21.15.90:0/101289459 conn(0x55853e90f230 msgr2=0x55853e9103d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:59.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52cabdb740 1 -- 172.21.15.90:0/101289459 shutdown_connections 2024-09-10T22:27:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52cabdb740 1 -- 172.21.15.90:0/101289459 wait complete. 2024-09-10T22:27:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52be000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52a80b53e0 0x7f52a80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52cabdb740 1 -- start start 2024-09-10T22:27:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.476+0000 7f52be000640 1 -- 172.21.15.90:0/3523131244 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7f52b41071c0 con 0x7f52c0102dc0 2024-09-10T22:27:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.477+0000 7f52c6000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52a80b53e0 0x7f52a80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.477+0000 7f52cabdb740 1 Processor -- start 2024-09-10T22:27:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.477+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911400 0x55853e91ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.477+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911d20 0x55853e91b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.477+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e91b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.477+0000 7f52cabdb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55853e6d4bb0 con 0x55853e911400 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.477+0000 7f52cabdb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55853e80e6c0 con 0x55853e911d20 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.477+0000 7f52cabdb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55853e920930 con 0x55853e91dcb0 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.477+0000 7f52cabdb740 10 client.0 _ll_register_callbacks cb 0x55853e829490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c5600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911400 0x55853e91ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911d20 0x55853e91b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c6000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e91b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c5600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911400 0x55853e91ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39234/0 (socket says 172.21.15.90:39234) 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c4c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911d20 0x55853e91b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:33932/0 (socket says 172.21.15.90:33932) 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c5600640 1 -- 172.21.15.90:0/3274115850 learned_addr learned my addr 172.21.15.90:0/3274115850 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:59.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c6000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52a80b53e0 0x7f52a80b78a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f52b800a860 tx=0x7f52b8004d00 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c4c00640 1 -- 172.21.15.90:0/3274115850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 msgr2=0x55853e91b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c4c00640 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e91b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c4c00640 1 -- 172.21.15.90:0/3274115850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911400 msgr2=0x55853e91ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c4c00640 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911400 0x55853e91ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.478+0000 7f52c4c00640 1 -- 172.21.15.90:0/3274115850 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853e5d1b00 con 0x55853e911d20 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.479+0000 7f52c4c00640 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911d20 0x55853e91b150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f52ac004630 tx=0x7f52ac012820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=457584af36e2cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.479+0000 7f52a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.479+0000 7f52a6a00640 1 -- 172.21.15.90:0/3274115850 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac005220 con 0x55853e911d20 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.479+0000 7f52cabdb740 1 -- 172.21.15.90:0/3274115850 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b8005650 con 0x55853e911d20 2024-09-10T22:27:59.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.479+0000 7f52a6a00640 1 -- 172.21.15.90:0/3274115850 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7f52ac0053c0 con 0x55853e911d20 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52a6a00640 1 -- 172.21.15.90:0/3274115850 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f52ac0055c0 con 0x55853e911d20 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52cabdb740 10 client.36191 fetch_fsmap learned FSMap version 95 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52cabdb740 1 -- 172.21.15.90:0/3274115850 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x55853e648960 con 0x55853e911d20 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52a6a00640 1 -- 172.21.15.90:0/3274115850 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52b8005650 con 0x55853e911d20 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52a6a00640 1 -- 172.21.15.90:0/3274115850 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 95) ==== 8386+0+0 (secure 0 0 0) 0x7f52ac019070 con 0x55853e911d20 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52cabdb740 10 client.36191 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52cabdb740 10 client.36191 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52cabdb740 20 client.36191 populate_metadata read hostname 'smithi090' 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55853e90ba90 0x55853e90deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52cabdb740 4 client.36191 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:27:59.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.480+0000 7f52cabdb740 1 -- 172.21.15.90:0/3274115850 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55853e8861e0 con 0x55853e90ba90 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.481+0000 7f52c5600640 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55853e90ba90 0x55853e90deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.481+0000 7f52c5600640 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55853e90ba90 0x55853e90deb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e4a4c15668b498 server_cookie=1a576756d725c0d2 in_seq=0 out_seq=0 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.481+0000 7f52a6a00640 10 client.36191 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.482+0000 7f52a6a00640 1 -- 172.21.15.90:0/3274115850 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f52b40adf90 con 0x55853e90ba90 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.482+0000 7f52a6a00640 10 client.36191 handle_command_reply: tid=0 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.482+0000 7f52cabdb740 1 client.36191 shutdown 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 -- 172.21.15.90:0/3274115850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911d20 msgr2=0x55853e91b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911d20 0x55853e91b150 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f52ac004630 tx=0x7f52ac012820 comp rx=0 tx=0).stop 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 -- 172.21.15.90:0/3274115850 shutdown_connections 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55853e90ba90 0x55853e90deb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e91b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911d20 0x55853e91b150 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3274115850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911400 0x55853e91ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52a6a00640 0 client.36191 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 -- 172.21.15.90:0/3274115850 >> 172.21.15.90:0/3274115850 conn(0x55853e90f230 msgr2=0x55853e914dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 -- 172.21.15.90:0/3274115850 shutdown_connections 2024-09-10T22:27:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 1 -- 172.21.15.90:0/3274115850 wait complete. 2024-09-10T22:27:59.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.483+0000 7f52cabdb740 20 client.36191 trim_cache size 0 max 16384 2024-09-10T22:27:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52cabdb740 1 Processor -- start 2024-09-10T22:27:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52cabdb740 1 -- start start 2024-09-10T22:27:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e9102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e909ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e917e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52cabdb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01adbb0 con 0x55853e911d20 2024-09-10T22:27:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52cabdb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01af7b0 con 0x55853e911400 2024-09-10T22:27:59.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52cabdb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01ae310 con 0x55853e91dcb0 2024-09-10T22:27:59.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52c4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e909ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52c4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e909ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39242/0 (socket says 172.21.15.90:39242) 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.501+0000 7f52c4c00640 1 -- 172.21.15.90:0/3245828373 learned_addr learned my addr 172.21.15.90:0/3245828373 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c6000640 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e917e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c5600640 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e9102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c4c00640 1 -- 172.21.15.90:0/3245828373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 msgr2=0x55853e917e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c4c00640 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e917e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c4c00640 1 -- 172.21.15.90:0/3245828373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 msgr2=0x55853e9102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c4c00640 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e9102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c4c00640 1 -- 172.21.15.90:0/3245828373 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55853e916500 con 0x55853e911d20 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c6000640 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e917e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:27:59.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c5600640 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e9102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:27:59.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52c4c00640 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e909ca0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f52ac010cd0 tx=0x7f52ac005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a259ccb4e1ddefc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:59.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.502+0000 7f52b1c00640 1 -- 172.21.15.90:0/3245828373 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac02c020 con 0x55853e911d20 2024-09-10T22:27:59.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.503+0000 7f52b1c00640 1 -- 172.21.15.90:0/3245828373 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52ac010e70 con 0x55853e911d20 2024-09-10T22:27:59.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.503+0000 7f52b1c00640 1 -- 172.21.15.90:0/3245828373 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac028450 con 0x55853e911d20 2024-09-10T22:27:59.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.503+0000 7f52cabdb740 1 -- 172.21.15.90:0/3245828373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 msgr2=0x55853e909ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:59.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.503+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e909ca0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f52ac010cd0 tx=0x7f52ac005760 comp rx=0 tx=0).stop 2024-09-10T22:27:59.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.503+0000 7f52cabdb740 1 -- 172.21.15.90:0/3245828373 shutdown_connections 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.503+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e917e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.503+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e909ca0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.503+0000 7f52cabdb740 1 --2- 172.21.15.90:0/3245828373 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e9102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.503+0000 7f52cabdb740 1 -- 172.21.15.90:0/3245828373 >> 172.21.15.90:0/3245828373 conn(0x55853e90f230 msgr2=0x55853e909590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.504+0000 7f52cabdb740 1 -- 172.21.15.90:0/3245828373 shutdown_connections 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.504+0000 7f52cabdb740 1 -- 172.21.15.90:0/3245828373 wait complete. 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.504+0000 7f52cabdb740 1 -- start start 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.504+0000 7f52cabdb740 1 Processor -- start 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e931910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e931e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52cabdb740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e92e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52cabdb740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01af0b0 con 0x55853e911d20 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52cabdb740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01ae9e0 con 0x55853e911400 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52cabdb740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01abbb0 con 0x55853e91dcb0 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52cabdb740 10 client.0 _ll_register_callbacks cb 0x7f52ac00b840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52c4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e931e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52c4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e931e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39244/0 (socket says 172.21.15.90:39244) 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52c4c00640 1 -- 172.21.15.90:0/2578360207 learned_addr learned my addr 172.21.15.90:0/2578360207 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52c5600640 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e931910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.505+0000 7f52c6000640 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e92e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52c5600640 1 -- 172.21.15.90:0/2578360207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 msgr2=0x55853e92e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:59.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52c5600640 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e92e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52c5600640 1 -- 172.21.15.90:0/2578360207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 msgr2=0x55853e931e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52c5600640 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e931e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52c5600640 1 -- 172.21.15.90:0/2578360207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55853e92e840 con 0x55853e911400 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52c6000640 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e92e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52c5600640 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e931910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f52b4104540 tx=0x7f52b403e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee3414f981cd8318 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52a6a00640 10 client.0 ms_handle_connect on v2:172.21.15.154:3300/0 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52a6a00640 1 -- 172.21.15.90:0/2578360207 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b40cd910 con 0x55853e911400 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52a6a00640 1 -- 172.21.15.90:0/2578360207 <== mon.2 v2:172.21.15.154:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9490+0+0 (secure 0 0 0) 0x7f52b40c9480 con 0x55853e911400 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.506+0000 7f52cabdb740 1 -- 172.21.15.90:0/2578360207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ac010e70 con 0x55853e911400 2024-09-10T22:27:59.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52a6a00640 1 -- 172.21.15.90:0/2578360207 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f52b40cd210 con 0x55853e911400 2024-09-10T22:27:59.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52cabdb740 10 client.36194 fetch_fsmap learned FSMap version 95 2024-09-10T22:27:59.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52cabdb740 1 -- 172.21.15.90:0/2578360207 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x55853e997440 con 0x55853e911400 2024-09-10T22:27:59.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52a6a00640 1 -- 172.21.15.90:0/2578360207 <== mon.2 v2:172.21.15.154:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52ac010e70 con 0x55853e911400 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52a6a00640 1 -- 172.21.15.90:0/2578360207 <== mon.2 v2:172.21.15.154:3300/0 5 ==== fsmap(e 95) ==== 8386+0+0 (secure 0 0 0) 0x7f52b40f00b0 con 0x55853e911400 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52cabdb740 10 client.36194 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52cabdb740 10 client.36194 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52cabdb740 20 client.36194 populate_metadata read hostname 'smithi090' 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52cabdb740 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55853e997ba0 0x55853e999fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52cabdb740 4 client.36194 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7387791c", "release": true, "await": true}] 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.507+0000 7f52cabdb740 1 -- 172.21.15.90:0/2578360207 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7387791c", "release": true, "await": true}) -- 0x55853e881c90 con 0x55853e997ba0 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.508+0000 7f52c4c00640 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55853e997ba0 0x55853e999fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.508+0000 7f52c4c00640 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55853e997ba0 0x55853e999fc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bba4f50b2716857e server_cookie=af0a3109500cc9fc in_seq=0 out_seq=0 2024-09-10T22:27:59.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.508+0000 7f52a6a00640 10 client.36194 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52a6a00640 1 -- 172.21.15.90:0/2578360207 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55853e881c90 con 0x55853e997ba0 2024-09-10T22:27:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52a6a00640 10 client.36194 handle_command_reply: tid=0 2024-09-10T22:27:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52cabdb740 1 client.36194 shutdown 2024-09-10T22:27:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52cabdb740 1 -- 172.21.15.90:0/2578360207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 msgr2=0x55853e931910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52cabdb740 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e931910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f52b4104540 tx=0x7f52b403e040 comp rx=0 tx=0).stop 2024-09-10T22:27:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52cabdb740 1 -- 172.21.15.90:0/2578360207 shutdown_connections 2024-09-10T22:27:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52cabdb740 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55853e997ba0 0x55853e999fc0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52cabdb740 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55853e91dcb0 0x55853e92e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52cabdb740 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55853e911d20 0x55853e931e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52a6a00640 0 client.36194 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:27:59.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52cabdb740 1 --2- 172.21.15.90:0/2578360207 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55853e911400 0x55853e931910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.555+0000 7f52cabdb740 1 -- 172.21.15.90:0/2578360207 >> 172.21.15.90:0/2578360207 conn(0x55853e90f230 msgr2=0x55853e91ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:59.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.556+0000 7f52cabdb740 1 -- 172.21.15.90:0/2578360207 shutdown_connections 2024-09-10T22:27:59.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.556+0000 7f52cabdb740 1 -- 172.21.15.90:0/2578360207 wait complete. 2024-09-10T22:27:59.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.556+0000 7f52cabdb740 20 client.36194 trim_cache size 0 max 16384 2024-09-10T22:27:59.559 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 95, 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 76, 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "7387791c": { 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 76, 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:27:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 138.8, 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:27:59.562 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:27:59.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 -- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52a80b53e0 msgr2=0x7f52a80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:59.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52a80b53e0 0x7f52a80b78a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f52b800a860 tx=0x7f52b8004d00 comp rx=0 tx=0).stop 2024-09-10T22:27:59.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 -- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 msgr2=0x7f52c01991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:27:59.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 0x7f52c01991c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005e30 tx=0x7f52b40039f0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 -- 172.21.15.90:0/3523131244 shutdown_connections 2024-09-10T22:27:59.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f52a80b53e0 0x7f52a80b78a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f52c0109150 0x7f52c019eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f52c01087d0 0x7f52c0199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 --2- 172.21.15.90:0/3523131244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52c0102dc0 0x7f52c01991c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:27:59.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 -- 172.21.15.90:0/3523131244 >> 172.21.15.90:0/3523131244 conn(0x7f52c00fe830 msgr2=0x7f52c0100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:27:59.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 -- 172.21.15.90:0/3523131244 shutdown_connections 2024-09-10T22:27:59.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:27:59.557+0000 7f52c8000640 1 -- 172.21.15.90:0/3523131244 wait complete. 2024-09-10T22:27:59.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:27:59 smithi154 ceph-mon[34495]: pgmap v1176: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 598 KiB/s wr, 54 op/s 2024-09-10T22:27:59.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:27:59 smithi171 ceph-mon[34138]: pgmap v1176: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 598 KiB/s wr, 54 op/s 2024-09-10T22:27:59.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:27:59 smithi090 ceph-mon[30350]: pgmap v1176: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 598 KiB/s wr, 54 op/s 2024-09-10T22:27:59.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56123 4.69 MB/sec execute 530 sec latency 15085.225 ms 2024-09-10T22:28:00.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56180 4.68 MB/sec execute 531 sec latency 980.950 ms 2024-09-10T22:28:01.452 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7387791c', total seconds elapsed: 24.5 2024-09-10T22:28:01.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:01 smithi171 ceph-mon[34138]: pgmap v1177: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 334 KiB/s wr, 37 op/s 2024-09-10T22:28:01.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:01 smithi090 ceph-mon[30350]: pgmap v1177: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 334 KiB/s wr, 37 op/s 2024-09-10T22:28:01.874 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:01 smithi154 ceph-mon[34495]: pgmap v1177: 129 pgs: 129 active+clean; 1.2 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 334 KiB/s wr, 37 op/s 2024-09-10T22:28:01.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56279 4.68 MB/sec execute 532 sec latency 534.832 ms 2024-09-10T22:28:02.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56400 4.69 MB/sec execute 533 sec latency 74.206 ms 2024-09-10T22:28:03.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:03 smithi154 ceph-mon[34495]: pgmap v1178: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 183 KiB/s wr, 19 op/s 2024-09-10T22:28:03.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:03 smithi171 ceph-mon[34138]: pgmap v1178: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 183 KiB/s wr, 19 op/s 2024-09-10T22:28:03.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:03 smithi090 ceph-mon[30350]: pgmap v1178: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 183 KiB/s wr, 19 op/s 2024-09-10T22:28:03.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56498 4.68 MB/sec execute 534 sec latency 86.592 ms 2024-09-10T22:28:04.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56589 4.68 MB/sec execute 535 sec latency 111.164 ms 2024-09-10T22:28:05.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:05 smithi154 ceph-mon[34495]: pgmap v1179: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 967 KiB/s wr, 62 op/s 2024-09-10T22:28:05.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:05 smithi154 ceph-mon[34495]: osdmap e127: 12 total, 12 up, 12 in 2024-09-10T22:28:05.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:05 smithi171 ceph-mon[34138]: pgmap v1179: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 967 KiB/s wr, 62 op/s 2024-09-10T22:28:05.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:05 smithi171 ceph-mon[34138]: osdmap e127: 12 total, 12 up, 12 in 2024-09-10T22:28:05.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:05 smithi090 ceph-mon[30350]: pgmap v1179: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 967 KiB/s wr, 62 op/s 2024-09-10T22:28:05.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:05 smithi090 ceph-mon[30350]: osdmap e127: 12 total, 12 up, 12 in 2024-09-10T22:28:05.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56727 4.69 MB/sec execute 536 sec latency 74.051 ms 2024-09-10T22:28:06.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56842 4.69 MB/sec execute 537 sec latency 114.476 ms 2024-09-10T22:28:07.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:07 smithi154 ceph-mon[34495]: pgmap v1181: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-09-10T22:28:07.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:07 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:07.747 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:07 smithi171 ceph-mon[34138]: pgmap v1181: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-09-10T22:28:07.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:07 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:07.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:07 smithi090 ceph-mon[30350]: pgmap v1181: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 108 op/s 2024-09-10T22:28:07.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:07 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:07.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56980 4.69 MB/sec execute 538 sec latency 78.714 ms 2024-09-10T22:28:08.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57114 4.70 MB/sec execute 539 sec latency 82.772 ms 2024-09-10T22:28:09.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:09 smithi154 ceph-mon[34495]: pgmap v1182: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 135 op/s 2024-09-10T22:28:09.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:09 smithi171 ceph-mon[34138]: pgmap v1182: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 135 op/s 2024-09-10T22:28:09.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:09 smithi090 ceph-mon[30350]: pgmap v1182: 129 pgs: 5 active+clean+snaptrim, 15 active+clean+snaptrim_wait, 109 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.8 MiB/s wr, 135 op/s 2024-09-10T22:28:09.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57218 4.70 MB/sec execute 540 sec latency 80.582 ms 2024-09-10T22:28:10.748 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:10 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:10.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:10 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:10.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57343 4.71 MB/sec execute 541 sec latency 74.253 ms 2024-09-10T22:28:10.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:10 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:11.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57480 4.72 MB/sec execute 542 sec latency 65.841 ms 2024-09-10T22:28:12.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:11 smithi154 ceph-mon[34495]: pgmap v1183: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 220 op/s 2024-09-10T22:28:12.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:11 smithi154 ceph-mon[34495]: osdmap e128: 12 total, 12 up, 12 in 2024-09-10T22:28:12.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:11 smithi171 ceph-mon[34138]: pgmap v1183: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 220 op/s 2024-09-10T22:28:12.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:11 smithi171 ceph-mon[34138]: osdmap e128: 12 total, 12 up, 12 in 2024-09-10T22:28:12.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:11 smithi090 ceph-mon[30350]: pgmap v1183: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 220 op/s 2024-09-10T22:28:12.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:11 smithi090 ceph-mon[30350]: osdmap e128: 12 total, 12 up, 12 in 2024-09-10T22:28:12.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57613 4.73 MB/sec execute 543 sec latency 69.118 ms 2024-09-10T22:28:13.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:12 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:13.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:12 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:13.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:12 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:13.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57733 4.73 MB/sec execute 544 sec latency 92.670 ms 2024-09-10T22:28:14.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:13 smithi154 ceph-mon[34495]: pgmap v1185: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 224 op/s 2024-09-10T22:28:14.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:13 smithi171 ceph-mon[34138]: pgmap v1185: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 224 op/s 2024-09-10T22:28:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:13 smithi090 ceph-mon[30350]: pgmap v1185: 129 pgs: 129 active+clean; 1.2 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 224 op/s 2024-09-10T22:28:14.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57845 4.72 MB/sec execute 545 sec latency 83.473 ms 2024-09-10T22:28:15.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57959 4.71 MB/sec execute 546 sec latency 85.061 ms 2024-09-10T22:28:16.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:15 smithi154 ceph-mon[34495]: pgmap v1186: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 246 op/s 2024-09-10T22:28:16.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:15 smithi171 ceph-mon[34138]: pgmap v1186: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 246 op/s 2024-09-10T22:28:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:15 smithi090 ceph-mon[30350]: pgmap v1186: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 246 op/s 2024-09-10T22:28:16.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58071 4.71 MB/sec execute 547 sec latency 83.104 ms 2024-09-10T22:28:17.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58216 4.71 MB/sec execute 548 sec latency 76.949 ms 2024-09-10T22:28:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:17 smithi154 ceph-mon[34495]: pgmap v1187: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 293 op/s 2024-09-10T22:28:18.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:17 smithi171 ceph-mon[34138]: pgmap v1187: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 293 op/s 2024-09-10T22:28:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:17 smithi090 ceph-mon[30350]: pgmap v1187: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.8 MiB/s wr, 293 op/s 2024-09-10T22:28:18.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58326 4.71 MB/sec execute 549 sec latency 107.615 ms 2024-09-10T22:28:19.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58453 4.72 MB/sec execute 550 sec latency 106.814 ms 2024-09-10T22:28:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:19 smithi154 ceph-mon[34495]: pgmap v1188: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 275 op/s 2024-09-10T22:28:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:19 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:19 smithi171 ceph-mon[34138]: pgmap v1188: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 275 op/s 2024-09-10T22:28:20.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:19 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:19 smithi090 ceph-mon[30350]: pgmap v1188: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.0 MiB/s wr, 275 op/s 2024-09-10T22:28:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:19 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:20.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58573 4.73 MB/sec execute 551 sec latency 65.498 ms 2024-09-10T22:28:21.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58743 4.74 MB/sec execute 552 sec latency 153.943 ms 2024-09-10T22:28:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:21 smithi154 ceph-mon[34495]: pgmap v1189: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2024-09-10T22:28:22.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:21 smithi171 ceph-mon[34138]: pgmap v1189: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2024-09-10T22:28:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:21 smithi090 ceph-mon[30350]: pgmap v1189: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2024-09-10T22:28:22.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58922 4.76 MB/sec execute 553 sec latency 68.955 ms 2024-09-10T22:28:23.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 59088 4.77 MB/sec execute 554 sec latency 92.511 ms 2024-09-10T22:28:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:23 smithi154 ceph-mon[34495]: pgmap v1190: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 249 op/s 2024-09-10T22:28:24.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:23 smithi171 ceph-mon[34138]: pgmap v1190: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 249 op/s 2024-09-10T22:28:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:23 smithi090 ceph-mon[30350]: pgmap v1190: 129 pgs: 129 active+clean; 1.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.4 MiB/s wr, 249 op/s 2024-09-10T22:28:24.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 59183 4.76 MB/sec execute 555 sec latency 68.988 ms 2024-09-10T22:28:25.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:24 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:25.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:24 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:25.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:24 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:28:25.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 59238 4.76 MB/sec execute 556 sec latency 425.039 ms 2024-09-10T22:28:26.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:25 smithi171 ceph-mon[34138]: pgmap v1191: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 281 op/s 2024-09-10T22:28:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:25 smithi090 ceph-mon[30350]: pgmap v1191: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 281 op/s 2024-09-10T22:28:26.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:25 smithi154 ceph-mon[34495]: pgmap v1191: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.4 MiB/s wr, 281 op/s 2024-09-10T22:28:26.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 59362 4.76 MB/sec execute 557 sec latency 135.128 ms 2024-09-10T22:28:27.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 59504 4.77 MB/sec execute 558 sec latency 73.312 ms 2024-09-10T22:28:28.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:27 smithi171 ceph-mon[34138]: pgmap v1192: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.5 MiB/s wr, 280 op/s 2024-09-10T22:28:28.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:27 smithi090 ceph-mon[30350]: pgmap v1192: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.5 MiB/s wr, 280 op/s 2024-09-10T22:28:28.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:27 smithi154 ceph-mon[34495]: pgmap v1192: 129 pgs: 129 active+clean; 1.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.5 MiB/s wr, 280 op/s 2024-09-10T22:28:28.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 59619 4.77 MB/sec execute 559 sec latency 96.724 ms 2024-09-10T22:28:29.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 59722 4.77 MB/sec execute 560 sec latency 196.647 ms 2024-09-10T22:28:30.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:29 smithi171 ceph-mon[34138]: pgmap v1193: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 229 op/s 2024-09-10T22:28:30.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:29 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:28:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:29 smithi090 ceph-mon[30350]: pgmap v1193: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 229 op/s 2024-09-10T22:28:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:29 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:28:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:29 smithi154 ceph-mon[34495]: pgmap v1193: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 229 op/s 2024-09-10T22:28:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:29 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:28:30.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 59817 4.78 MB/sec execute 561 sec latency 232.295 ms 2024-09-10T22:28:31.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 59918 4.78 MB/sec execute 562 sec latency 83.051 ms 2024-09-10T22:28:32.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:31 smithi171 ceph-mon[34138]: pgmap v1194: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.8 MiB/s wr, 287 op/s 2024-09-10T22:28:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:31 smithi090 ceph-mon[30350]: pgmap v1194: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.8 MiB/s wr, 287 op/s 2024-09-10T22:28:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:31 smithi154 ceph-mon[34495]: pgmap v1194: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.8 MiB/s wr, 287 op/s 2024-09-10T22:28:32.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60029 4.78 MB/sec execute 563 sec latency 73.933 ms 2024-09-10T22:28:33.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60165 4.78 MB/sec execute 564 sec latency 94.685 ms 2024-09-10T22:28:34.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:33 smithi171 ceph-mon[34138]: pgmap v1195: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2024-09-10T22:28:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:33 smithi090 ceph-mon[30350]: pgmap v1195: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2024-09-10T22:28:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:33 smithi154 ceph-mon[34495]: pgmap v1195: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 232 op/s 2024-09-10T22:28:34.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60271 4.78 MB/sec execute 565 sec latency 97.217 ms 2024-09-10T22:28:35.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60373 4.78 MB/sec execute 566 sec latency 82.568 ms 2024-09-10T22:28:36.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:35 smithi171 ceph-mon[34138]: pgmap v1196: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 231 op/s 2024-09-10T22:28:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:35 smithi090 ceph-mon[30350]: pgmap v1196: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 231 op/s 2024-09-10T22:28:36.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:35 smithi154 ceph-mon[34495]: pgmap v1196: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 231 op/s 2024-09-10T22:28:36.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60464 4.78 MB/sec execute 567 sec latency 284.035 ms 2024-09-10T22:28:37.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60541 4.78 MB/sec execute 568 sec latency 649.011 ms 2024-09-10T22:28:38.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:37 smithi171 ceph-mon[34138]: pgmap v1197: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.4 MiB/s wr, 245 op/s 2024-09-10T22:28:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:37 smithi090 ceph-mon[30350]: pgmap v1197: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.4 MiB/s wr, 245 op/s 2024-09-10T22:28:38.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:37 smithi154 ceph-mon[34495]: pgmap v1197: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.4 MiB/s wr, 245 op/s 2024-09-10T22:28:38.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60600 4.78 MB/sec execute 569 sec latency 636.745 ms 2024-09-10T22:28:39.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60739 4.79 MB/sec execute 570 sec latency 83.317 ms 2024-09-10T22:28:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:39 smithi171 ceph-mon[34138]: pgmap v1198: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 207 op/s 2024-09-10T22:28:40.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:39 smithi090 ceph-mon[30350]: pgmap v1198: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 207 op/s 2024-09-10T22:28:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:39 smithi154 ceph-mon[34495]: pgmap v1198: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 207 op/s 2024-09-10T22:28:40.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60868 4.80 MB/sec execute 571 sec latency 85.311 ms 2024-09-10T22:28:41.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60922 4.80 MB/sec execute 572 sec latency 600.178 ms 2024-09-10T22:28:42.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:41 smithi171 ceph-mon[34138]: pgmap v1199: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-09-10T22:28:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:41 smithi090 ceph-mon[30350]: pgmap v1199: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-09-10T22:28:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:41 smithi154 ceph-mon[34495]: pgmap v1199: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 254 op/s 2024-09-10T22:28:42.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 60979 4.79 MB/sec execute 573 sec latency 514.990 ms 2024-09-10T22:28:43.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61062 4.79 MB/sec execute 574 sec latency 226.296 ms 2024-09-10T22:28:44.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:43 smithi171 ceph-mon[34138]: pgmap v1200: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 192 op/s 2024-09-10T22:28:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:43 smithi090 ceph-mon[30350]: pgmap v1200: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 192 op/s 2024-09-10T22:28:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:43 smithi154 ceph-mon[34495]: pgmap v1200: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 192 op/s 2024-09-10T22:28:44.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61128 4.78 MB/sec execute 575 sec latency 604.885 ms 2024-09-10T22:28:45.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61199 4.79 MB/sec execute 576 sec latency 597.852 ms 2024-09-10T22:28:46.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:45 smithi171 ceph-mon[34138]: pgmap v1201: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 227 op/s 2024-09-10T22:28:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:45 smithi090 ceph-mon[30350]: pgmap v1201: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 227 op/s 2024-09-10T22:28:46.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:45 smithi154 ceph-mon[34495]: pgmap v1201: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 227 op/s 2024-09-10T22:28:46.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61323 4.78 MB/sec execute 577 sec latency 121.416 ms 2024-09-10T22:28:47.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61448 4.78 MB/sec execute 578 sec latency 106.018 ms 2024-09-10T22:28:48.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:47 smithi171 ceph-mon[34138]: pgmap v1202: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 239 op/s 2024-09-10T22:28:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:47 smithi090 ceph-mon[30350]: pgmap v1202: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 239 op/s 2024-09-10T22:28:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:47 smithi154 ceph-mon[34495]: pgmap v1202: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 239 op/s 2024-09-10T22:28:48.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61553 4.78 MB/sec execute 579 sec latency 84.211 ms 2024-09-10T22:28:49.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61661 4.77 MB/sec execute 580 sec latency 94.958 ms 2024-09-10T22:28:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:49 smithi171 ceph-mon[34138]: pgmap v1203: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-09-10T22:28:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:49 smithi090 ceph-mon[30350]: pgmap v1203: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-09-10T22:28:50.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:49 smithi154 ceph-mon[34495]: pgmap v1203: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 199 op/s 2024-09-10T22:28:50.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61731 4.77 MB/sec execute 581 sec latency 451.788 ms 2024-09-10T22:28:51.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61789 4.77 MB/sec execute 582 sec latency 479.563 ms 2024-09-10T22:28:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:51 smithi171 ceph-mon[34138]: pgmap v1204: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-09-10T22:28:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:51 smithi090 ceph-mon[30350]: pgmap v1204: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-09-10T22:28:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:51 smithi154 ceph-mon[34495]: pgmap v1204: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-09-10T22:28:52.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61853 4.77 MB/sec execute 583 sec latency 476.349 ms 2024-09-10T22:28:53.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 61949 4.77 MB/sec execute 584 sec latency 654.814 ms 2024-09-10T22:28:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:53 smithi171 ceph-mon[34138]: pgmap v1205: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 182 op/s 2024-09-10T22:28:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:53 smithi090 ceph-mon[30350]: pgmap v1205: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 182 op/s 2024-09-10T22:28:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:53 smithi154 ceph-mon[34495]: pgmap v1205: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 182 op/s 2024-09-10T22:28:54.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 62095 4.78 MB/sec execute 585 sec latency 157.192 ms 2024-09-10T22:28:55.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 62228 4.79 MB/sec execute 586 sec latency 684.023 ms 2024-09-10T22:28:56.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:55 smithi090 ceph-mon[30350]: pgmap v1206: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 219 op/s 2024-09-10T22:28:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:55 smithi154 ceph-mon[34495]: pgmap v1206: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 219 op/s 2024-09-10T22:28:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:55 smithi171 ceph-mon[34138]: pgmap v1206: 129 pgs: 129 active+clean; 1.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.6 MiB/s wr, 219 op/s 2024-09-10T22:28:56.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 62389 4.80 MB/sec execute 587 sec latency 81.418 ms 2024-09-10T22:28:57.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 62489 4.80 MB/sec execute 588 sec latency 78.968 ms 2024-09-10T22:28:58.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:28:58 smithi090 ceph-mon[30350]: pgmap v1207: 129 pgs: 129 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 224 op/s 2024-09-10T22:28:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:28:58 smithi154 ceph-mon[34495]: pgmap v1207: 129 pgs: 129 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 224 op/s 2024-09-10T22:28:58.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:28:58 smithi171 ceph-mon[34138]: pgmap v1207: 129 pgs: 129 active+clean; 1.4 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.9 MiB/s wr, 224 op/s 2024-09-10T22:28:58.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 62599 4.81 MB/sec execute 589 sec latency 238.243 ms 2024-09-10T22:28:59.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 62717 4.81 MB/sec execute 590 sec latency 76.927 ms 2024-09-10T22:29:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:00 smithi090 ceph-mon[30350]: pgmap v1208: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 195 op/s 2024-09-10T22:29:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:00 smithi154 ceph-mon[34495]: pgmap v1208: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 195 op/s 2024-09-10T22:29:00.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:00 smithi171 ceph-mon[34138]: pgmap v1208: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.5 MiB/s wr, 195 op/s 2024-09-10T22:29:00.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 62811 4.81 MB/sec execute 591 sec latency 221.895 ms 2024-09-10T22:29:01.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 62913 4.81 MB/sec execute 592 sec latency 141.152 ms 2024-09-10T22:29:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:02 smithi090 ceph-mon[30350]: pgmap v1209: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 249 op/s 2024-09-10T22:29:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:02 smithi154 ceph-mon[34495]: pgmap v1209: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 249 op/s 2024-09-10T22:29:02.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:02 smithi171 ceph-mon[34138]: pgmap v1209: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 249 op/s 2024-09-10T22:29:02.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 62950 4.80 MB/sec execute 593 sec latency 678.491 ms 2024-09-10T22:29:03.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:03 smithi090 ceph-mon[30350]: pgmap v1210: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-09-10T22:29:03.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:03 smithi154 ceph-mon[34495]: pgmap v1210: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-09-10T22:29:03.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:03 smithi171 ceph-mon[34138]: pgmap v1210: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 198 op/s 2024-09-10T22:29:03.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 63044 4.80 MB/sec execute 594 sec latency 856.408 ms 2024-09-10T22:29:04.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 63128 4.80 MB/sec execute 595 sec latency 436.583 ms 2024-09-10T22:29:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:29:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:29:05.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:29:05.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:29:05.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:29:05.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:05 smithi090 ceph-mon[30350]: osdmap e129: 12 total, 12 up, 12 in 2024-09-10T22:29:05.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:05 smithi090 ceph-mon[30350]: pgmap v1212: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 224 op/s 2024-09-10T22:29:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:29:05.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:29:05.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:29:05.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:29:05.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:29:05.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:05 smithi154 ceph-mon[34495]: osdmap e129: 12 total, 12 up, 12 in 2024-09-10T22:29:05.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:05 smithi154 ceph-mon[34495]: pgmap v1212: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 224 op/s 2024-09-10T22:29:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:29:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:29:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:29:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:29:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:29:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:05 smithi171 ceph-mon[34138]: osdmap e129: 12 total, 12 up, 12 in 2024-09-10T22:29:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:05 smithi171 ceph-mon[34138]: pgmap v1212: 129 pgs: 129 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.6 MiB/s wr, 224 op/s 2024-09-10T22:29:05.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 63217 4.80 MB/sec execute 596 sec latency 312.899 ms 2024-09-10T22:29:06.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 63335 4.81 MB/sec execute 597 sec latency 376.103 ms 2024-09-10T22:29:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:07 smithi154 ceph-mon[34495]: pgmap v1213: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 222 op/s 2024-09-10T22:29:07.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:07 smithi171 ceph-mon[34138]: pgmap v1213: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 222 op/s 2024-09-10T22:29:07.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:07 smithi090 ceph-mon[30350]: pgmap v1213: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 222 op/s 2024-09-10T22:29:07.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 63452 4.80 MB/sec execute 598 sec latency 92.983 ms 2024-09-10T22:29:08.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 63581 4.81 MB/sec execute 599 sec latency 147.243 ms 2024-09-10T22:29:09.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:09 smithi154 ceph-mon[34495]: pgmap v1214: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 219 op/s 2024-09-10T22:29:09.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:09 smithi171 ceph-mon[34138]: pgmap v1214: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 219 op/s 2024-09-10T22:29:09.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:09 smithi090 ceph-mon[30350]: pgmap v1214: 129 pgs: 5 active+clean+snaptrim_wait, 1 active+clean+snaptrim, 123 active+clean; 1.4 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.4 MiB/s wr, 219 op/s 2024-09-10T22:29:09.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 600 sec 2024-09-10T22:29:10.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 601 sec 2024-09-10T22:29:11.468 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T22:29:11.468 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T22:29:11.469 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:29:11.923 INFO:tasks.workunit.client.0.smithi090.stdout: 6 cleanup 602 sec 2024-09-10T22:29:12.028 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:29:12.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:11 smithi154 ceph-mon[34495]: pgmap v1215: 129 pgs: 129 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 210 op/s 2024-09-10T22:29:12.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:11 smithi154 ceph-mon[34495]: osdmap e130: 12 total, 12 up, 12 in 2024-09-10T22:29:12.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:11 smithi090 ceph-mon[30350]: pgmap v1215: 129 pgs: 129 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 210 op/s 2024-09-10T22:29:12.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:11 smithi090 ceph-mon[30350]: osdmap e130: 12 total, 12 up, 12 in 2024-09-10T22:29:12.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:11 smithi171 ceph-mon[34138]: pgmap v1215: 129 pgs: 129 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.4 MiB/s wr, 210 op/s 2024-09-10T22:29:12.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:11 smithi171 ceph-mon[34138]: osdmap e130: 12 total, 12 up, 12 in 2024-09-10T22:29:12.923 INFO:tasks.workunit.client.0.smithi090.stdout: 4 cleanup 603 sec 2024-09-10T22:29:13.923 INFO:tasks.workunit.client.0.smithi090.stdout: 4 cleanup 604 sec 2024-09-10T22:29:14.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:13 smithi154 ceph-mon[34495]: pgmap v1217: 129 pgs: 129 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 212 op/s 2024-09-10T22:29:14.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:13 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:29:14.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:13 smithi171 ceph-mon[34138]: pgmap v1217: 129 pgs: 129 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 212 op/s 2024-09-10T22:29:14.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:13 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:29:14.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:13 smithi090 ceph-mon[30350]: pgmap v1217: 129 pgs: 129 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.6 MiB/s wr, 212 op/s 2024-09-10T22:29:14.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:13 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:29:14.608 INFO:tasks.workunit.client.0.smithi090.stdout: 0 cleanup 605 sec 2024-09-10T22:29:14.608 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-09-10T22:29:14.608 INFO:tasks.workunit.client.0.smithi090.stdout: Operation Count AvgLat MaxLat 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: ---------------------------------------- 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: NTCreateX 92547 24.644 33685.178 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: Close 67821 0.279 290.453 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: Rename 3919 65.773 33915.485 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: Unlink 18802 26.235 33943.242 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: Qpathinfo 83731 10.244 3734.245 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: Qfileinfo 14592 0.577 370.360 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: Qfsinfo 15426 2.935 2403.576 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: Sfileinfo 7527 18.559 28567.631 2024-09-10T22:29:14.609 INFO:tasks.workunit.client.0.smithi090.stdout: Find 32404 50.384 3083.171 2024-09-10T22:29:14.610 INFO:tasks.workunit.client.0.smithi090.stdout: WriteX 45619 4.237 33747.216 2024-09-10T22:29:14.610 INFO:tasks.workunit.client.0.smithi090.stdout: ReadX 145333 0.028 308.526 2024-09-10T22:29:14.610 INFO:tasks.workunit.client.0.smithi090.stdout: LockX 305 1.956 7.550 2024-09-10T22:29:14.610 INFO:tasks.workunit.client.0.smithi090.stdout: UnlockX 305 1.892 9.110 2024-09-10T22:29:14.610 INFO:tasks.workunit.client.0.smithi090.stdout: Flush 6507 9.842 867.227 2024-09-10T22:29:14.610 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-09-10T22:29:14.610 INFO:tasks.workunit.client.0.smithi090.stdout:Throughput 4.81276 MB/sec 10 clients 10 procs max_latency=33943.252 ms 2024-09-10T22:29:15.001 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T22:29:15.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T22:29:15.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:14 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:29:15.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:14 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:29:15.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:14 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:29:16.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.056+0000 7fc647e00640 1 -- 172.21.15.90:0/1543627123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc648104430 msgr2=0x7fc64810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:16.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.056+0000 7fc647e00640 1 --2- 172.21.15.90:0/1543627123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc648104430 0x7fc64810c920 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc634009900 tx=0x7fc63402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.057+0000 7fc647e00640 1 -- 172.21.15.90:0/1543627123 shutdown_connections 2024-09-10T22:29:16.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.057+0000 7fc647e00640 1 --2- 172.21.15.90:0/1543627123 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64810cf70 0x7fc64810f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.057+0000 7fc647e00640 1 --2- 172.21.15.90:0/1543627123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc648104430 0x7fc64810c920 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.057+0000 7fc647e00640 1 --2- 172.21.15.90:0/1543627123 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc648103b10 0x7fc648103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.057+0000 7fc647e00640 1 -- 172.21.15.90:0/1543627123 >> 172.21.15.90:0/1543627123 conn(0x7fc6480fd5b0 msgr2=0x7fc6480ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:16.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.057+0000 7fc647e00640 1 -- 172.21.15.90:0/1543627123 shutdown_connections 2024-09-10T22:29:16.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.057+0000 7fc647e00640 1 -- 172.21.15.90:0/1543627123 wait complete. 2024-09-10T22:29:16.060 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:15 smithi090 ceph-mon[30350]: pgmap v1218: 129 pgs: 129 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 248 op/s 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.058+0000 7fc647e00640 1 Processor -- start 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.058+0000 7fc647e00640 1 -- start start 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.058+0000 7fc647e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc648103b10 0x7fc64810bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.058+0000 7fc647e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc648104430 0x7fc64810c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.058+0000 7fc647e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64810cf70 0x7fc6481a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.058+0000 7fc647e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc648112ae0 con 0x7fc648103b10 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.058+0000 7fc647e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc648112960 con 0x7fc64810cf70 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.058+0000 7fc647e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc648112c60 con 0x7fc648104430 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc646a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc648103b10 0x7fc64810bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc646a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc648103b10 0x7fc64810bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44276/0 (socket says 172.21.15.90:44276) 2024-09-10T22:29:16.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc647400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64810cf70 0x7fc6481a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc646a00640 1 -- 172.21.15.90:0/1863590320 learned_addr learned my addr 172.21.15.90:0/1863590320 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc646000640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc648104430 0x7fc64810c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc647400640 1 -- 172.21.15.90:0/1863590320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc648104430 msgr2=0x7fc64810c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc647400640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc648104430 0x7fc64810c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc647400640 1 -- 172.21.15.90:0/1863590320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc648103b10 msgr2=0x7fc64810bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc647400640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc648103b10 0x7fc64810bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.059+0000 7fc647400640 1 -- 172.21.15.90:0/1863590320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6481a23f0 con 0x7fc64810cf70 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.060+0000 7fc647400640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64810cf70 0x7fc6481a1c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc640009fd0 tx=0x7fc64000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.060+0000 7fc63f400640 1 -- 172.21.15.90:0/1863590320 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64000ce20 con 0x7fc64810cf70 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.060+0000 7fc63f400640 1 -- 172.21.15.90:0/1863590320 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc640005020 con 0x7fc64810cf70 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.060+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6481a26e0 con 0x7fc64810cf70 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.060+0000 7fc63f400640 1 -- 172.21.15.90:0/1863590320 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640010690 con 0x7fc64810cf70 2024-09-10T22:29:16.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.060+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6481ac050 con 0x7fc64810cf70 2024-09-10T22:29:16.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.061+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc64806b1b0 con 0x7fc64810cf70 2024-09-10T22:29:16.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.064+0000 7fc63f400640 1 -- 172.21.15.90:0/1863590320 <== mon.2 v2:172.21.15.154:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc640004a40 con 0x7fc64810cf70 2024-09-10T22:29:16.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.066+0000 7fc63f400640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc62c0b5500 0x7fc62c0b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:16.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.066+0000 7fc63f400640 1 -- 172.21.15.90:0/1863590320 <== mon.2 v2:172.21.15.154:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7fc6400e6ab0 con 0x7fc64810cf70 2024-09-10T22:29:16.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.066+0000 7fc646a00640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc62c0b5500 0x7fc62c0b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:16.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.067+0000 7fc646a00640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc62c0b5500 0x7fc62c0b79c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc638009c90 tx=0x7fc638005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:16.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.069+0000 7fc63f400640 1 -- 172.21.15.90:0/1863590320 <== mon.2 v2:172.21.15.154:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc6400ad200 con 0x7fc64810cf70 2024-09-10T22:29:16.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:15 smithi154 ceph-mon[34495]: pgmap v1218: 129 pgs: 129 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 248 op/s 2024-09-10T22:29:16.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:15 smithi171 ceph-mon[34138]: pgmap v1218: 129 pgs: 129 active+clean; 1.4 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.1 MiB/s wr, 248 op/s 2024-09-10T22:29:16.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.364+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6481ac280 con 0x7fc64810cf70 2024-09-10T22:29:16.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.365+0000 7fc63f400640 1 -- 172.21.15.90:0/1863590320 <== mon.2 v2:172.21.15.154:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+12011 (secure 0 0 0) 0x7fc6400acb00 con 0x7fc64810cf70 2024-09-10T22:29:16.367 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:29:16.367 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":102,"btime":"2024-09-10T22:29:13:891041+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":102,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:29:13.800471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34703,34670,34640,34646,34607]},"id":1}]} 2024-09-10T22:29:16.368 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 102 2024-09-10T22:29:16.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc62c0b5500 msgr2=0x7fc62c0b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:16.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc62c0b5500 0x7fc62c0b79c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc638009c90 tx=0x7fc638005f90 comp rx=0 tx=0).stop 2024-09-10T22:29:16.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64810cf70 msgr2=0x7fc6481a1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:16.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64810cf70 0x7fc6481a1c70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc640009fd0 tx=0x7fc64000ef10 comp rx=0 tx=0).stop 2024-09-10T22:29:16.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 shutdown_connections 2024-09-10T22:29:16.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc62c0b5500 0x7fc62c0b79c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc64810cf70 0x7fc6481a1c70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc648104430 0x7fc64810c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 --2- 172.21.15.90:0/1863590320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc648103b10 0x7fc64810bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:16.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 >> 172.21.15.90:0/1863590320 conn(0x7fc6480fd5b0 msgr2=0x7fc64810f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:16.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 shutdown_connections 2024-09-10T22:29:16.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:16.368+0000 7fc647e00640 1 -- 172.21.15.90:0/1863590320 wait complete. 2024-09-10T22:29:17.021 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-10T22:29:17.021 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T22:29:17.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:16 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1863590320' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:17.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:16 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1863590320' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:16 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1863590320' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:17.461 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T22:29:17.461 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T22:29:17.841 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:29:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:17 smithi154 ceph-mon[34495]: pgmap v1219: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 293 op/s 2024-09-10T22:29:18.226 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:17 smithi090 ceph-mon[30350]: pgmap v1219: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 293 op/s 2024-09-10T22:29:18.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:17 smithi171 ceph-mon[34138]: pgmap v1219: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.0 MiB/s wr, 293 op/s 2024-09-10T22:29:18.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.508+0000 7fe21f000640 1 -- 172.21.15.90:0/3312898467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810a9b0 msgr2=0x7fe21810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:18.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.508+0000 7fe21f000640 1 --2- 172.21.15.90:0/3312898467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810a9b0 0x7fe21810adf0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe204009900 tx=0x7fe20402a410 comp rx=0 tx=0).stop 2024-09-10T22:29:18.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.509+0000 7fe21f000640 1 -- 172.21.15.90:0/3312898467 shutdown_connections 2024-09-10T22:29:18.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.509+0000 7fe21f000640 1 --2- 172.21.15.90:0/3312898467 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe21810b330 0x7fe218111b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.509+0000 7fe21f000640 1 --2- 172.21.15.90:0/3312898467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810a9b0 0x7fe21810adf0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.509+0000 7fe21f000640 1 --2- 172.21.15.90:0/3312898467 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe218104fa0 0x7fe218105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.509+0000 7fe21f000640 1 -- 172.21.15.90:0/3312898467 >> 172.21.15.90:0/3312898467 conn(0x7fe218100a50 msgr2=0x7fe218102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.509+0000 7fe21f000640 1 -- 172.21.15.90:0/3312898467 shutdown_connections 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.509+0000 7fe21f000640 1 -- 172.21.15.90:0/3312898467 wait complete. 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.510+0000 7fe21f000640 1 Processor -- start 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.510+0000 7fe21f000640 1 -- start start 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.510+0000 7fe21f000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe218104fa0 0x7fe21819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.510+0000 7fe21f000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21810a9b0 0x7fe21819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.510+0000 7fe21f000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810b330 0x7fe2181a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.510+0000 7fe21f000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe218114e10 con 0x7fe21810b330 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.510+0000 7fe21f000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe218114c90 con 0x7fe21810a9b0 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.510+0000 7fe21f000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe218114f90 con 0x7fe218104fa0 2024-09-10T22:29:18.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe21d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810b330 0x7fe2181a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe21d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810b330 0x7fe2181a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47780/0 (socket says 172.21.15.90:47780) 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe21d000640 1 -- 172.21.15.90:0/2674278803 learned_addr learned my addr 172.21.15.90:0/2674278803 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe217400640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21810a9b0 0x7fe21819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe217e00640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe218104fa0 0x7fe21819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe21d000640 1 -- 172.21.15.90:0/2674278803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe218104fa0 msgr2=0x7fe21819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe21d000640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe218104fa0 0x7fe21819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe21d000640 1 -- 172.21.15.90:0/2674278803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21810a9b0 msgr2=0x7fe21819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe21d000640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21810a9b0 0x7fe21819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe21d000640 1 -- 172.21.15.90:0/2674278803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2181a13f0 con 0x7fe21810b330 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe217e00640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe218104fa0 0x7fe21819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.511+0000 7fe21d000640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810b330 0x7fe2181a0c20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe21000eaa0 tx=0x7fe21000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.512+0000 7fe214c00640 1 -- 172.21.15.90:0/2674278803 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21000cdb0 con 0x7fe21810b330 2024-09-10T22:29:18.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.512+0000 7fe214c00640 1 -- 172.21.15.90:0/2674278803 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe210005020 con 0x7fe21810b330 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.512+0000 7fe214c00640 1 -- 172.21.15.90:0/2674278803 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2100116d0 con 0x7fe21810b330 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.512+0000 7fe21f000640 1 -- 172.21.15.90:0/2674278803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2181a16e0 con 0x7fe21810b330 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.512+0000 7fe21f000640 1 -- 172.21.15.90:0/2674278803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2181a7f70 con 0x7fe21810b330 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.515+0000 7fe214c00640 1 -- 172.21.15.90:0/2674278803 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fe21000f040 con 0x7fe21810b330 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.517+0000 7fe214c00640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe1fc0b54d0 0x7fe1fc0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.517+0000 7fe214c00640 1 -- 172.21.15.90:0/2674278803 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7fe2100e6200 con 0x7fe21810b330 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.517+0000 7fe217e00640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe1fc0b54d0 0x7fe1fc0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.517+0000 7fe217e00640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe1fc0b54d0 0x7fe1fc0b7990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe208009c20 tx=0x7fe208005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.575+0000 7fe221a8f740 1 Processor -- start 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.575+0000 7fe221a8f740 1 -- start start 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 0x561681aaa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe221a8f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561681a1fca0 con 0x561681ab6a30 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe221a8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x561681a0f0d0 con 0x561681aaa180 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe221a8f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561681a12ee0 con 0x561681aaaaa0 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe21d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe217e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 0x561681aaa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe21d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47796/0 (socket says 172.21.15.90:47796) 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe21d000640 1 -- 172.21.15.90:0/4088251037 learned_addr learned my addr 172.21.15.90:0/4088251037 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe217400640 1 --2- 172.21.15.90:0/4088251037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe21d000640 1 -- 172.21.15.90:0/4088251037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 msgr2=0x561681ab64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe21d000640 1 --2- 172.21.15.90:0/4088251037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe21d000640 1 -- 172.21.15.90:0/4088251037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 msgr2=0x561681aaa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe21d000640 1 --2- 172.21.15.90:0/4088251037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 0x561681aaa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.576+0000 7fe21d000640 1 -- 172.21.15.90:0/4088251037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616819c1400 con 0x561681ab6a30 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.577+0000 7fe21d000640 1 --2- 172.21.15.90:0/4088251037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8e20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe2100a8820 tx=0x7fe2100ac130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ce8a0bc05b849b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.577+0000 7fe20d600640 1 -- 172.21.15.90:0/4088251037 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21001f070 con 0x561681ab6a30 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.577+0000 7fe20d600640 1 -- 172.21.15.90:0/4088251037 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2100ac250 con 0x561681ab6a30 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.577+0000 7fe20d600640 1 -- 172.21.15.90:0/4088251037 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2100e3930 con 0x561681ab6a30 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 -- 172.21.15.90:0/4088251037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 msgr2=0x561681ab8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4088251037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8e20 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe2100a8820 tx=0x7fe2100ac130 comp rx=0 tx=0).stop 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 -- 172.21.15.90:0/4088251037 shutdown_connections 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4088251037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8e20 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4088251037 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4088251037 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 0x561681aaa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 -- 172.21.15.90:0/4088251037 >> 172.21.15.90:0/4088251037 conn(0x561681aa7fb0 msgr2=0x561681aa9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 -- 172.21.15.90:0/4088251037 shutdown_connections 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 -- 172.21.15.90:0/4088251037 wait complete. 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 -- start start 2024-09-10T22:29:18.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.578+0000 7fe221a8f740 1 Processor -- start 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681ab63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 0x561681ab3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe221a8f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5616818707a0 con 0x561681aaa180 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe221a8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x5616819b4a10 con 0x561681ab6a30 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe221a8f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x561681ab95f0 con 0x561681aaaaa0 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe221a8f740 10 client.0 _ll_register_callbacks cb 0x5616819c4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe217e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681ab63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe217e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681ab63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47798/0 (socket says 172.21.15.90:47798) 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe217e00640 1 -- 172.21.15.90:0/4159415029 learned_addr learned my addr 172.21.15.90:0/4159415029 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.579+0000 7fe217400640 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe21d000640 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 0x561681ab3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe217e00640 1 -- 172.21.15.90:0/4159415029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 msgr2=0x561681ab37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe217e00640 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe217e00640 1 -- 172.21.15.90:0/4159415029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 msgr2=0x561681ab3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe217e00640 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 0x561681ab3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe217e00640 1 -- 172.21.15.90:0/4159415029 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168177cb90 con 0x561681aaa180 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe217e00640 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681ab63a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe2080038a0 tx=0x7fe208004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a7c750677e27a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe202000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe202000640 1 -- 172.21.15.90:0/4159415029 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe208012650 con 0x561681aaa180 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe221a8f740 1 -- 172.21.15.90:0/4159415029 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2100ac250 con 0x561681aaa180 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe202000640 1 -- 172.21.15.90:0/4159415029 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7fe2080127f0 con 0x561681aaa180 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.580+0000 7fe202000640 1 -- 172.21.15.90:0/4159415029 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7fe208032050 con 0x561681aaa180 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe221a8f740 10 client.16587 fetch_fsmap learned FSMap version 103 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe221a8f740 1 -- 172.21.15.90:0/4159415029 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x561681ac5050 con 0x561681aaa180 2024-09-10T22:29:18.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe202000640 1 -- 172.21.15.90:0/4159415029 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2100ac250 con 0x561681aaa180 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe202000640 1 -- 172.21.15.90:0/4159415029 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 103) ==== 8434+0+0 (secure 0 0 0) 0x7fe20802aad0 con 0x561681aaa180 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe221a8f740 10 client.16587 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe221a8f740 10 client.16587 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe221a8f740 20 client.16587 populate_metadata read hostname 'smithi090' 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561681ac6550 0x561681ac8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe221a8f740 4 client.16587 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.581+0000 7fe221a8f740 1 -- 172.21.15.90:0/4159415029 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561681a21bf0 con 0x561681ac6550 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.582+0000 7fe217400640 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561681ac6550 0x561681ac8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.582+0000 7fe217400640 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561681ac6550 0x561681ac8990 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec46ace07721099 server_cookie=1d81f82c8990e248 in_seq=0 out_seq=0 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.582+0000 7fe202000640 10 client.16587 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe202000640 1 -- 172.21.15.90:0/4159415029 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561681a21bf0 con 0x561681ac6550 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe202000640 10 client.16587 handle_command_reply: tid=0 2024-09-10T22:29:18.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 client.16587 shutdown 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 -- 172.21.15.90:0/4159415029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 msgr2=0x561681ab63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681ab63a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe2080038a0 tx=0x7fe208004730 comp rx=0 tx=0).stop 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 -- 172.21.15.90:0/4159415029 shutdown_connections 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561681ac6550 0x561681ac8990 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 0x561681ab3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 --2- 172.21.15.90:0/4159415029 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681ab63a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe202000640 0 client.16587 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 -- 172.21.15.90:0/4159415029 >> 172.21.15.90:0/4159415029 conn(0x561681aa7fb0 msgr2=0x561681aadb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.583+0000 7fe221a8f740 1 -- 172.21.15.90:0/4159415029 shutdown_connections 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.584+0000 7fe221a8f740 1 -- 172.21.15.90:0/4159415029 wait complete. 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.584+0000 7fe221a8f740 20 client.16587 trim_cache size 0 max 16384 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.601+0000 7fe221a8f740 1 Processor -- start 2024-09-10T22:29:18.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.601+0000 7fe221a8f740 1 -- start start 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.601+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 0x561681ab2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.601+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.601+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.601+0000 7fe221a8f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181b2060 con 0x561681ab6a30 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.601+0000 7fe221a8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181b3e40 con 0x561681aaa180 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.601+0000 7fe221a8f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181b27f0 con 0x561681aaaaa0 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe21d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe21d000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47806/0 (socket says 172.21.15.90:47806) 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe21d000640 1 -- 172.21.15.90:0/135602022 learned_addr learned my addr 172.21.15.90:0/135602022 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe217e00640 1 --2- 172.21.15.90:0/135602022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 0x561681ab2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe217400640 1 --2- 172.21.15.90:0/135602022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe21d000640 1 -- 172.21.15.90:0/135602022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 msgr2=0x561681ab7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:18.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe21d000640 1 --2- 172.21.15.90:0/135602022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe21d000640 1 -- 172.21.15.90:0/135602022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 msgr2=0x561681ab2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe21d000640 1 --2- 172.21.15.90:0/135602022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 0x561681ab2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe21d000640 1 -- 172.21.15.90:0/135602022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561681ab4740 con 0x561681ab6a30 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.602+0000 7fe21d000640 1 --2- 172.21.15.90:0/135602022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8360 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe2100a94c0 tx=0x7fe2100a80f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf531d7576314be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe20cc00640 1 -- 172.21.15.90:0/135602022 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe21001f040 con 0x561681ab6a30 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe20cc00640 1 -- 172.21.15.90:0/135602022 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe2100b0d30 con 0x561681ab6a30 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe20cc00640 1 -- 172.21.15.90:0/135602022 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2100cf0b0 con 0x561681ab6a30 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe221a8f740 1 -- 172.21.15.90:0/135602022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 msgr2=0x561681ab8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe221a8f740 1 --2- 172.21.15.90:0/135602022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8360 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe2100a94c0 tx=0x7fe2100a80f0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe221a8f740 1 -- 172.21.15.90:0/135602022 shutdown_connections 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe221a8f740 1 --2- 172.21.15.90:0/135602022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681ab6a30 0x561681ab8360 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe221a8f740 1 --2- 172.21.15.90:0/135602022 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe221a8f740 1 --2- 172.21.15.90:0/135602022 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681aaa180 0x561681ab2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe221a8f740 1 -- 172.21.15.90:0/135602022 >> 172.21.15.90:0/135602022 conn(0x561681aa7fb0 msgr2=0x561681ab5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe221a8f740 1 -- 172.21.15.90:0/135602022 shutdown_connections 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.603+0000 7fe221a8f740 1 -- 172.21.15.90:0/135602022 wait complete. 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.604+0000 7fe221a8f740 1 -- start start 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.604+0000 7fe221a8f740 1 Processor -- start 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.604+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681abeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.604+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.604+0000 7fe221a8f740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 0x561681ab9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.604+0000 7fe221a8f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181b36b0 con 0x561681aaa180 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.604+0000 7fe221a8f740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181b2f80 con 0x561681ab6a30 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.604+0000 7fe221a8f740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fe2181b0060 con 0x561681aaaaa0 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe221a8f740 10 client.0 _ll_register_callbacks cb 0x7fe208009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681abeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681abeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47820/0 (socket says 172.21.15.90:47820) 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217e00640 1 -- 172.21.15.90:0/1552974461 learned_addr learned my addr 172.21.15.90:0/1552974461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe21d000640 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 0x561681ab9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217400640 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217e00640 1 -- 172.21.15.90:0/1552974461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 msgr2=0x561681ab9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217e00640 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217e00640 1 -- 172.21.15.90:0/1552974461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 msgr2=0x561681ab9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217e00640 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 0x561681ab9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217e00640 1 -- 172.21.15.90:0/1552974461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561681aba420 con 0x561681aaa180 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217400640 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe217e00640 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681abeee0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe208003870 tx=0x7fe208003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c7cdf7ec2b3cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.605+0000 7fe202000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe202000640 1 -- 172.21.15.90:0/1552974461 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20802a620 con 0x561681aaa180 2024-09-10T22:29:18.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe202000640 1 -- 172.21.15.90:0/1552974461 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7fe20802a7c0 con 0x561681aaa180 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe221a8f740 1 -- 172.21.15.90:0/1552974461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561681a1d6a0 con 0x561681aaa180 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe202000640 1 -- 172.21.15.90:0/1552974461 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7fe20804a050 con 0x561681aaa180 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe221a8f740 10 client.16593 fetch_fsmap learned FSMap version 103 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe221a8f740 1 -- 172.21.15.90:0/1552974461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x561681b34310 con 0x561681aaa180 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe202000640 1 -- 172.21.15.90:0/1552974461 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561681a1d6a0 con 0x561681aaa180 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe202000640 1 -- 172.21.15.90:0/1552974461 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 103) ==== 8434+0+0 (secure 0 0 0) 0x7fe208021bf0 con 0x561681aaa180 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe221a8f740 10 client.16593 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe221a8f740 10 client.16593 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe221a8f740 20 client.16593 populate_metadata read hostname 'smithi090' 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe221a8f740 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561681b34b30 0x561681b36f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe221a8f740 4 client.16593 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T22:29:18.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.606+0000 7fe221a8f740 1 -- 172.21.15.90:0/1552974461 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x561681a1aad0 con 0x561681b34b30 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.607+0000 7fe217400640 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561681b34b30 0x561681b36f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.607+0000 7fe217400640 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561681b34b30 0x561681b36f50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23a850ea700fc983 server_cookie=34606beefb1b391c in_seq=0 out_seq=0 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.607+0000 7fe202000640 10 client.16593 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.617+0000 7fe202000640 1 -- 172.21.15.90:0/1552974461 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+592 (crc 0 0 0) 0x561681a1aad0 con 0x561681b34b30 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.617+0000 7fe202000640 10 client.16593 handle_command_reply: tid=0 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.617+0000 7fe221a8f740 1 client.16593 shutdown 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 -- 172.21.15.90:0/1552974461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 msgr2=0x561681abeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681abeee0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe208003870 tx=0x7fe208003d60 comp rx=0 tx=0).stop 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 -- 172.21.15.90:0/1552974461 shutdown_connections 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x561681b34b30 0x561681b36f50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x561681ab6a30 0x561681ab9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x561681aaaaa0 0x561681ab9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 --2- 172.21.15.90:0/1552974461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561681aaa180 0x561681abeee0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe202000640 0 client.16593 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 -- 172.21.15.90:0/1552974461 >> 172.21.15.90:0/1552974461 conn(0x561681aa7fb0 msgr2=0x561681aa6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 -- 172.21.15.90:0/1552974461 shutdown_connections 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 1 -- 172.21.15.90:0/1552974461 wait complete. 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.618+0000 7fe221a8f740 20 client.16593 trim_cache size 0 max 16384 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 103, 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 78, 2024-09-10T22:29:18.637 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "3fdd8644": { 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 78, 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:29:18.638 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:29:18.639 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:29:18.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:18.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:18.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:18.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:18.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:18.639 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:29:18.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 -- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe1fc0b54d0 msgr2=0x7fe1fc0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe1fc0b54d0 0x7fe1fc0b7990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe208009c20 tx=0x7fe208005f90 comp rx=0 tx=0).stop 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 -- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810b330 msgr2=0x7fe2181a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810b330 0x7fe2181a0c20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fe21000eaa0 tx=0x7fe21000ef60 comp rx=0 tx=0).stop 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 -- 172.21.15.90:0/2674278803 shutdown_connections 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fe1fc0b54d0 0x7fe1fc0b7990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe21810b330 0x7fe2181a0c20 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fe21810a9b0 0x7fe21819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 --2- 172.21.15.90:0/2674278803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fe218104fa0 0x7fe21819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 -- 172.21.15.90:0/2674278803 >> 172.21.15.90:0/2674278803 conn(0x7fe218100a50 msgr2=0x7fe218102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 -- 172.21.15.90:0/2674278803 shutdown_connections 2024-09-10T22:29:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:18.619+0000 7fe21f000640 1 -- 172.21.15.90:0/2674278803 wait complete. 2024-09-10T22:29:18.828 DEBUG:teuthology.parallel:result is None 2024-09-10T22:29:18.829 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T22:29:19.083 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:18 smithi090 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:29:19.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:18 smithi171 ceph-mon[34138]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:29:19.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:18 smithi154 ceph-mon[34495]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T22:29:19.658 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3fdd8644', quiesce took 8.2 seconds. Will release after: 21.8 2024-09-10T22:29:20.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:19 smithi171 ceph-mon[34138]: pgmap v1220: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 309 op/s 2024-09-10T22:29:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:19 smithi090 ceph-mon[30350]: pgmap v1220: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 309 op/s 2024-09-10T22:29:20.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:19 smithi154 ceph-mon[34495]: pgmap v1220: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.9 MiB/s wr, 309 op/s 2024-09-10T22:29:22.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:21 smithi171 ceph-mon[34138]: pgmap v1221: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-09-10T22:29:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:21 smithi090 ceph-mon[30350]: pgmap v1221: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-09-10T22:29:22.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:21 smithi154 ceph-mon[34495]: pgmap v1221: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 276 op/s 2024-09-10T22:29:24.221 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:23 smithi154 ceph-mon[34495]: pgmap v1222: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-09-10T22:29:24.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:23 smithi171 ceph-mon[34138]: pgmap v1222: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-09-10T22:29:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:23 smithi090 ceph-mon[30350]: pgmap v1222: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-09-10T22:29:26.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:25 smithi171 ceph-mon[34138]: pgmap v1223: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-09-10T22:29:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:25 smithi090 ceph-mon[30350]: pgmap v1223: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-09-10T22:29:26.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:25 smithi154 ceph-mon[34495]: pgmap v1223: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-09-10T22:29:28.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:27 smithi171 ceph-mon[34138]: pgmap v1224: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-09-10T22:29:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:27 smithi090 ceph-mon[30350]: pgmap v1224: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-09-10T22:29:28.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:27 smithi154 ceph-mon[34495]: pgmap v1224: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-09-10T22:29:30.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:29 smithi171 ceph-mon[34138]: pgmap v1225: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 640 KiB/s wr, 76 op/s 2024-09-10T22:29:30.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:29 smithi090 ceph-mon[30350]: pgmap v1225: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 640 KiB/s wr, 76 op/s 2024-09-10T22:29:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:29 smithi154 ceph-mon[34495]: pgmap v1225: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 640 KiB/s wr, 76 op/s 2024-09-10T22:29:32.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:31 smithi171 ceph-mon[34138]: pgmap v1226: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 460 KiB/s wr, 61 op/s 2024-09-10T22:29:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:31 smithi090 ceph-mon[30350]: pgmap v1226: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 460 KiB/s wr, 61 op/s 2024-09-10T22:29:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:31 smithi154 ceph-mon[34495]: pgmap v1226: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 460 KiB/s wr, 61 op/s 2024-09-10T22:29:34.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:33 smithi171 ceph-mon[34138]: pgmap v1227: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-09-10T22:29:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:33 smithi090 ceph-mon[30350]: pgmap v1227: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-09-10T22:29:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:33 smithi154 ceph-mon[34495]: pgmap v1227: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 68 KiB/s wr, 19 op/s 2024-09-10T22:29:36.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:35 smithi171 ceph-mon[34138]: pgmap v1228: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 68 KiB/s wr, 20 op/s 2024-09-10T22:29:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:35 smithi090 ceph-mon[30350]: pgmap v1228: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 68 KiB/s wr, 20 op/s 2024-09-10T22:29:36.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:35 smithi154 ceph-mon[34495]: pgmap v1228: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 68 KiB/s wr, 20 op/s 2024-09-10T22:29:38.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:37 smithi171 ceph-mon[34138]: pgmap v1229: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-09-10T22:29:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:37 smithi090 ceph-mon[30350]: pgmap v1229: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-09-10T22:29:38.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:37 smithi154 ceph-mon[34495]: pgmap v1229: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-09-10T22:29:40.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:39 smithi171 ceph-mon[34138]: pgmap v1230: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-09-10T22:29:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:39 smithi090 ceph-mon[30350]: pgmap v1230: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-09-10T22:29:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:39 smithi154 ceph-mon[34495]: pgmap v1230: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-09-10T22:29:41.459 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:29:41.838 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:29:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:41 smithi171 ceph-mon[34138]: pgmap v1231: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-10T22:29:42.253 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:41 smithi090 ceph-mon[30350]: pgmap v1231: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-10T22:29:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:41 smithi154 ceph-mon[34495]: pgmap v1231: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 14 op/s 2024-09-10T22:29:42.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.572+0000 7f2f34200640 1 -- 172.21.15.90:0/1945254302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 msgr2=0x7f2f2c10ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.572+0000 7f2f34200640 1 --2- 172.21.15.90:0/1945254302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 0x7f2f2c10ee50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009900 tx=0x7f2f1802a410 comp rx=0 tx=0).stop 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.572+0000 7f2f34200640 1 -- 172.21.15.90:0/1945254302 shutdown_connections 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.572+0000 7f2f34200640 1 --2- 172.21.15.90:0/1945254302 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c10f390 0x7f2f2c111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.572+0000 7f2f34200640 1 --2- 172.21.15.90:0/1945254302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 0x7f2f2c10ee50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.572+0000 7f2f34200640 1 --2- 172.21.15.90:0/1945254302 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c0754e0 0x7f2f2c0758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.572+0000 7f2f34200640 1 -- 172.21.15.90:0/1945254302 >> 172.21.15.90:0/1945254302 conn(0x7f2f2c0fe810 msgr2=0x7f2f2c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.572+0000 7f2f34200640 1 -- 172.21.15.90:0/1945254302 shutdown_connections 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.572+0000 7f2f34200640 1 -- 172.21.15.90:0/1945254302 wait complete. 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.573+0000 7f2f34200640 1 Processor -- start 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f34200640 1 -- start start 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f34200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c0754e0 0x7f2f2c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:42.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f34200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 0x7f2f2c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f34200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c10f390 0x7f2f2c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f34200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c114e00 con 0x7f2f2c075e00 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f34200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c114c80 con 0x7f2f2c0754e0 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f34200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c114f80 con 0x7f2f2c10f390 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f30e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 0x7f2f2c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f31800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c0754e0 0x7f2f2c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f30e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 0x7f2f2c19b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53952/0 (socket says 172.21.15.90:53952) 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f30e00640 1 -- 172.21.15.90:0/2796723631 learned_addr learned my addr 172.21.15.90:0/2796723631 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.574+0000 7f2f32200640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c10f390 0x7f2f2c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.575+0000 7f2f30e00640 1 -- 172.21.15.90:0/2796723631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c10f390 msgr2=0x7f2f2c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.575+0000 7f2f30e00640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c10f390 0x7f2f2c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.575+0000 7f2f30e00640 1 -- 172.21.15.90:0/2796723631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c0754e0 msgr2=0x7f2f2c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.575+0000 7f2f30e00640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c0754e0 0x7f2f2c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:42.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.575+0000 7f2f30e00640 1 -- 172.21.15.90:0/2796723631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c1a1460 con 0x7f2f2c075e00 2024-09-10T22:29:42.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.575+0000 7f2f31800640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c0754e0 0x7f2f2c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:29:42.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.575+0000 7f2f32200640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c10f390 0x7f2f2c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:29:42.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.575+0000 7f2f30e00640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 0x7f2f2c19b8f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009230 tx=0x7f2f18004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:42.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.576+0000 7f2f2a000640 1 -- 172.21.15.90:0/2796723631 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1803f030 con 0x7f2f2c075e00 2024-09-10T22:29:42.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.576+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c1a16f0 con 0x7f2f2c075e00 2024-09-10T22:29:42.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.576+0000 7f2f2a000640 1 -- 172.21.15.90:0/2796723631 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f18003a80 con 0x7f2f2c075e00 2024-09-10T22:29:42.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.576+0000 7f2f2a000640 1 -- 172.21.15.90:0/2796723631 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18035070 con 0x7f2f2c075e00 2024-09-10T22:29:42.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.576+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c1a7e10 con 0x7f2f2c075e00 2024-09-10T22:29:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.577+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2c0758c0 con 0x7f2f2c075e00 2024-09-10T22:29:42.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.579+0000 7f2f2a000640 1 -- 172.21.15.90:0/2796723631 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2f1804c020 con 0x7f2f2c075e00 2024-09-10T22:29:42.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.581+0000 7f2f2a000640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f140b5450 0x7f2f140b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:42.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.581+0000 7f2f2a000640 1 -- 172.21.15.90:0/2796723631 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f2f18106940 con 0x7f2f2c075e00 2024-09-10T22:29:42.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.582+0000 7f2f31800640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f140b5450 0x7f2f140b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:42.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.582+0000 7f2f31800640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f140b5450 0x7f2f140b7910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f20005d00 tx=0x7f2f2000b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:42.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.584+0000 7f2f2a000640 1 -- 172.21.15.90:0/2796723631 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2f180ccaa0 con 0x7f2f2c075e00 2024-09-10T22:29:42.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.826+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f2c1a7fe0 con 0x7f2f2c075e00 2024-09-10T22:29:42.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.827+0000 7f2f2a000640 1 -- 172.21.15.90:0/2796723631 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+12015 (secure 0 0 0) 0x7f2f180d14d0 con 0x7f2f2c075e00 2024-09-10T22:29:42.828 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:29:42.829 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":103,"btime":"2024-09-10T22:29:17:913064+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":103,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:29:17.776682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:29:42.829 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 103 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f140b5450 msgr2=0x7f2f140b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f140b5450 0x7f2f140b7910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f20005d00 tx=0x7f2f2000b410 comp rx=0 tx=0).stop 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 msgr2=0x7f2f2c19b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 0x7f2f2c19b8f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009230 tx=0x7f2f18004270 comp rx=0 tx=0).stop 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 shutdown_connections 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f140b5450 0x7f2f140b7910 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f2c10f390 0x7f2f2c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f2c075e00 0x7f2f2c19b8f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 --2- 172.21.15.90:0/2796723631 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f2c0754e0 0x7f2f2c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:42.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 >> 172.21.15.90:0/2796723631 conn(0x7f2f2c0fe810 msgr2=0x7f2f2c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 shutdown_connections 2024-09-10T22:29:42.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:42.830+0000 7f2f34200640 1 -- 172.21.15.90:0/2796723631 wait complete. 2024-09-10T22:29:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:42 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2796723631' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:43.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:42 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2796723631' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:43.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:42 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2796723631' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:43.781 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --set-id 3fdd8644 --release --await' 2024-09-10T22:29:43.781 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --set-id 3fdd8644 --release --await 2024-09-10T22:29:44.176 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:29:44.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:43 smithi171 ceph-mon[34138]: pgmap v1232: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T22:29:44.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:43 smithi090 ceph-mon[30350]: pgmap v1232: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T22:29:44.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:43 smithi154 ceph-mon[34495]: pgmap v1232: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-10T22:29:44.888 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T22:29:44.888 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-10T22:29:44.901 INFO:tasks.quiescer:joining Quiescers 2024-09-10T22:29:44.902 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-10T22:29:44.902 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:29:44.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.834+0000 7f0951e00640 1 -- 172.21.15.90:0/284869326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 msgr2=0x7f094c10f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:44.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.834+0000 7f0951e00640 1 --2- 172.21.15.90:0/284869326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 0x7f094c10f7a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0944009ca0 tx=0x7f094402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.834+0000 7f0951e00640 1 -- 172.21.15.90:0/284869326 shutdown_connections 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.834+0000 7f0951e00640 1 --2- 172.21.15.90:0/284869326 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 0x7f094c10f7a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.834+0000 7f0951e00640 1 --2- 172.21.15.90:0/284869326 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f094c108620 0x7f094c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.834+0000 7f0951e00640 1 --2- 172.21.15.90:0/284869326 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f094c102c10 0x7f094c102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.834+0000 7f0951e00640 1 -- 172.21.15.90:0/284869326 >> 172.21.15.90:0/284869326 conn(0x7f094c0fe6c0 msgr2=0x7f094c100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.834+0000 7f0951e00640 1 -- 172.21.15.90:0/284869326 shutdown_connections 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.834+0000 7f0951e00640 1 -- 172.21.15.90:0/284869326 wait complete. 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.835+0000 7f0951e00640 1 Processor -- start 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.835+0000 7f0951e00640 1 -- start start 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0951e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f094c102c10 0x7f094c19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0951e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f094c108620 0x7f094c19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0951e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 0x7f094c1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0951e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f094c112b20 con 0x7f094c108fa0 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0951e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f094c1129a0 con 0x7f094c102c10 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0951e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f094c112ca0 con 0x7f094c108620 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0951400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 0x7f094c1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0951400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 0x7f094c1a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53976/0 (socket says 172.21.15.90:53976) 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0951400640 1 -- 172.21.15.90:0/2839145237 learned_addr learned my addr 172.21.15.90:0/2839145237 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0943e00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f094c108620 0x7f094c19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.836+0000 7f0950a00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f094c102c10 0x7f094c19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f0951400640 1 -- 172.21.15.90:0/2839145237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f094c108620 msgr2=0x7f094c19b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f0951400640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f094c108620 0x7f094c19b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f0951400640 1 -- 172.21.15.90:0/2839145237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f094c102c10 msgr2=0x7f094c19b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f0951400640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f094c102c10 0x7f094c19b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f0951400640 1 -- 172.21.15.90:0/2839145237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094c1a11d0 con 0x7f094c108fa0 2024-09-10T22:29:44.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f0951400640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 0x7f094c1a0a00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0944005e30 tx=0x7f09440093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f094a000640 1 -- 172.21.15.90:0/2839145237 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944032d10 con 0x7f094c108fa0 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f094a000640 1 -- 172.21.15.90:0/2839145237 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0944032eb0 con 0x7f094c108fa0 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f094a000640 1 -- 172.21.15.90:0/2839145237 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944030b20 con 0x7f094c108fa0 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.837+0000 7f0951e00640 1 -- 172.21.15.90:0/2839145237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094c1a1460 con 0x7f094c108fa0 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.838+0000 7f0951e00640 1 -- 172.21.15.90:0/2839145237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f094c1a7b80 con 0x7f094c108fa0 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.840+0000 7f0955df0740 1 Processor -- start 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.841+0000 7f094a000640 1 -- 172.21.15.90:0/2839145237 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f0944030cc0 con 0x7f094c108fa0 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.841+0000 7f0955df0740 1 -- start start 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.841+0000 7f0955df0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d3d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.841+0000 7f0955df0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 0x55ed55d49540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.841+0000 7f0955df0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d49a80 0x55ed55d4be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.841+0000 7f0955df0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ed55cb4290 con 0x55ed55d49a80 2024-09-10T22:29:44.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.841+0000 7f0955df0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ed55ca36c0 con 0x55ed55d3d1d0 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.841+0000 7f0955df0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ed55ca74d0 con 0x55ed55d3daf0 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d49a80 0x55ed55d4be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0950a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 0x55ed55d49540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d49a80 0x55ed55d4be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53988/0 (socket says 172.21.15.90:53988) 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 -- 172.21.15.90:0/2409745221 learned_addr learned my addr 172.21.15.90:0/2409745221 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 --2- 172.21.15.90:0/2409745221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d3d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 -- 172.21.15.90:0/2409745221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 msgr2=0x55ed55d49540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 --2- 172.21.15.90:0/2409745221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 0x55ed55d49540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 -- 172.21.15.90:0/2409745221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 msgr2=0x55ed55d3d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 --2- 172.21.15.90:0/2409745221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d3d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 -- 172.21.15.90:0/2409745221 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed55c30460 con 0x55ed55d49a80 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f094a000640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f09300b5580 0x7f09300b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f0943e00640 1 --2- 172.21.15.90:0/2409745221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d49a80 0x55ed55d4be70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0934011840 tx=0x7f09340097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab35a255efb9928 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.842+0000 7f094a000640 1 -- 172.21.15.90:0/2839145237 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f0944107500 con 0x7f094c108fa0 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.843+0000 7f0950a00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f09300b5580 0x7f09300b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.843+0000 7f0941c00640 1 -- 172.21.15.90:0/2409745221 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934004030 con 0x55ed55d49a80 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.843+0000 7f0941c00640 1 -- 172.21.15.90:0/2409745221 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f093400f7c0 con 0x55ed55d49a80 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.843+0000 7f0941c00640 1 -- 172.21.15.90:0/2409745221 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934003850 con 0x55ed55d49a80 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 -- 172.21.15.90:0/2409745221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d49a80 msgr2=0x55ed55d4be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 --2- 172.21.15.90:0/2409745221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d49a80 0x55ed55d4be70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f0934011840 tx=0x7f09340097d0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0950a00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f09300b5580 0x7f09300b7a40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55ed55d40350 tx=0x7f093c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 -- 172.21.15.90:0/2409745221 shutdown_connections 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 --2- 172.21.15.90:0/2409745221 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d49a80 0x55ed55d4be70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 --2- 172.21.15.90:0/2409745221 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 0x55ed55d49540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 --2- 172.21.15.90:0/2409745221 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d3d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 -- 172.21.15.90:0/2409745221 >> 172.21.15.90:0/2409745221 conn(0x55ed55d3b000 msgr2=0x55ed55d3c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 -- 172.21.15.90:0/2409745221 shutdown_connections 2024-09-10T22:29:44.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 -- 172.21.15.90:0/2409745221 wait complete. 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 -- start start 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.844+0000 7f0955df0740 1 Processor -- start 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0955df0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d46b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0955df0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d47040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0955df0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d49a80 0x55ed55d47580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0955df0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ed55b04bb0 con 0x55ed55d3daf0 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0955df0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ed55c3e6c0 con 0x55ed55d3d1d0 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0955df0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ed55d4c730 con 0x55ed55d49a80 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0955df0740 10 client.0 _ll_register_callbacks cb 0x55ed55c59490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0943e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d47040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0943e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d47040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53996/0 (socket says 172.21.15.90:53996) 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0943e00640 1 -- 172.21.15.90:0/3458384780 learned_addr learned my addr 172.21.15.90:0/3458384780 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:44.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0951400640 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d49a80 0x55ed55d47580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.845+0000 7f0950a00640 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d46b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0943e00640 1 -- 172.21.15.90:0/3458384780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d49a80 msgr2=0x55ed55d47580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0943e00640 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d49a80 0x55ed55d47580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0943e00640 1 -- 172.21.15.90:0/3458384780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 msgr2=0x55ed55d46b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0943e00640 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d46b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0943e00640 1 -- 172.21.15.90:0/3458384780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed55a01b00 con 0x55ed55d3daf0 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0950a00640 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d46b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0943e00640 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d47040 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f093400fdd0 tx=0x7f093400fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f495f40b6ff033 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f093a000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f093a000640 1 -- 172.21.15.90:0/3458384780 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09340051d0 con 0x55ed55d3daf0 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f093a000640 1 -- 172.21.15.90:0/3458384780 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f0934005370 con 0x55ed55d3daf0 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0955df0740 1 -- 172.21.15.90:0/3458384780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f093400f7c0 con 0x55ed55d3daf0 2024-09-10T22:29:44.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f093a000640 1 -- 172.21.15.90:0/3458384780 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f0934031050 con 0x55ed55d3daf0 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0955df0740 10 client.16611 fetch_fsmap learned FSMap version 103 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.846+0000 7f0955df0740 1 -- 172.21.15.90:0/3458384780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x55ed55a78960 con 0x55ed55d3daf0 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.847+0000 7f093a000640 1 -- 172.21.15.90:0/3458384780 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f093400f7c0 con 0x55ed55d3daf0 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.847+0000 7f093a000640 1 -- 172.21.15.90:0/3458384780 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 103) ==== 8434+0+0 (secure 0 0 0) 0x7f093401f3f0 con 0x55ed55d3daf0 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.847+0000 7f0955df0740 10 client.16611 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.847+0000 7f0955df0740 10 client.16611 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.847+0000 7f0955df0740 20 client.16611 populate_metadata read hostname 'smithi090' 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.847+0000 7f0955df0740 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ed55d59520 0x55ed55d5b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.847+0000 7f0955df0740 4 client.16611 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.847+0000 7f0955df0740 1 -- 172.21.15.90:0/3458384780 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed55cb61e0 con 0x55ed55d59520 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.848+0000 7f0950a00640 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ed55d59520 0x55ed55d5b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.848+0000 7f0950a00640 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ed55d59520 0x55ed55d5b910 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc95861886da1b36 server_cookie=fe6437f1519c9763 in_seq=0 out_seq=0 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.848+0000 7f093a000640 10 client.16611 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:44.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.849+0000 7f093a000640 1 -- 172.21.15.90:0/3458384780 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ed55cb61e0 con 0x55ed55d59520 2024-09-10T22:29:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.849+0000 7f093a000640 10 client.16611 handle_command_reply: tid=0 2024-09-10T22:29:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.849+0000 7f0955df0740 1 client.16611 shutdown 2024-09-10T22:29:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 -- 172.21.15.90:0/3458384780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 msgr2=0x55ed55d47040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d47040 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f093400fdd0 tx=0x7f093400fe00 comp rx=0 tx=0).stop 2024-09-10T22:29:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 -- 172.21.15.90:0/3458384780 shutdown_connections 2024-09-10T22:29:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ed55d59520 0x55ed55d5b910 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d49a80 0x55ed55d47580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d47040 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f093a000640 0 client.16611 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 --2- 172.21.15.90:0/3458384780 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d46b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 -- 172.21.15.90:0/3458384780 >> 172.21.15.90:0/3458384780 conn(0x55ed55d3b000 msgr2=0x55ed55d40ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 -- 172.21.15.90:0/3458384780 shutdown_connections 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 1 -- 172.21.15.90:0/3458384780 wait complete. 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.850+0000 7f0955df0740 20 client.16611 trim_cache size 0 max 16384 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.868+0000 7f0955df0740 1 Processor -- start 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0955df0740 1 -- start start 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0955df0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3b000 0x55ed55d41390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0955df0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3d1d0 0x55ed55d43d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0955df0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d44250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0955df0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f094c1b1da0 con 0x55ed55d3daf0 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0955df0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f094c1b3b80 con 0x55ed55d3b000 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0955df0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f094c1b2530 con 0x55ed55d3d1d0 2024-09-10T22:29:44.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0951400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d44250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0951400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d44250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54010/0 (socket says 172.21.15.90:54010) 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0951400640 1 -- 172.21.15.90:0/3911040697 learned_addr learned my addr 172.21.15.90:0/3911040697 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0943e00640 1 --2- 172.21.15.90:0/3911040697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3d1d0 0x55ed55d43d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.869+0000 7f0950a00640 1 --2- 172.21.15.90:0/3911040697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3b000 0x55ed55d41390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.870+0000 7f0951400640 1 -- 172.21.15.90:0/3911040697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3d1d0 msgr2=0x55ed55d43d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.870+0000 7f0951400640 1 --2- 172.21.15.90:0/3911040697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3d1d0 0x55ed55d43d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.870+0000 7f0951400640 1 -- 172.21.15.90:0/3911040697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3b000 msgr2=0x55ed55d41390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.870+0000 7f0951400640 1 --2- 172.21.15.90:0/3911040697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3b000 0x55ed55d41390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.870+0000 7f0951400640 1 -- 172.21.15.90:0/3911040697 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed55d5bef0 con 0x55ed55d3daf0 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.870+0000 7f0951400640 1 --2- 172.21.15.90:0/3911040697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d44250 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0944031040 tx=0x7f09440ccf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991d1b1ba74b92b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.870+0000 7f0941200640 1 -- 172.21.15.90:0/3911040697 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440cdc50 con 0x55ed55d3daf0 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.870+0000 7f0941200640 1 -- 172.21.15.90:0/3911040697 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0944036070 con 0x55ed55d3daf0 2024-09-10T22:29:44.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.870+0000 7f0941200640 1 -- 172.21.15.90:0/3911040697 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09440d6ae0 con 0x55ed55d3daf0 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 -- 172.21.15.90:0/3911040697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 msgr2=0x55ed55d44250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 --2- 172.21.15.90:0/3911040697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d44250 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0944031040 tx=0x7f09440ccf60 comp rx=0 tx=0).stop 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 -- 172.21.15.90:0/3911040697 shutdown_connections 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 --2- 172.21.15.90:0/3911040697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3daf0 0x55ed55d44250 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 --2- 172.21.15.90:0/3911040697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3d1d0 0x55ed55d43d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 --2- 172.21.15.90:0/3911040697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3b000 0x55ed55d41390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 -- 172.21.15.90:0/3911040697 >> 172.21.15.90:0/3911040697 conn(0x55ed55d3c5d0 msgr2=0x55ed55d48130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 -- 172.21.15.90:0/3911040697 shutdown_connections 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 -- 172.21.15.90:0/3911040697 wait complete. 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.871+0000 7f0955df0740 1 -- start start 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0955df0740 1 Processor -- start 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0955df0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3b000 0x55ed55d522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0955df0740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d4e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0955df0740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 0x55ed55d4ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0955df0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f094c1b33f0 con 0x55ed55d3b000 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0955df0740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f094c1b2cc0 con 0x55ed55d3d1d0 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0955df0740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f094c1afda0 con 0x55ed55d3daf0 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0955df0740 10 client.0 _ll_register_callbacks cb 0x7f0934009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0950a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3b000 0x55ed55d522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0950a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3b000 0x55ed55d522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54018/0 (socket says 172.21.15.90:54018) 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0950a00640 1 -- 172.21.15.90:0/1519895216 learned_addr learned my addr 172.21.15.90:0/1519895216 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0951400640 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 0x55ed55d4ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.872+0000 7f0943e00640 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d4e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.873+0000 7f0950a00640 1 -- 172.21.15.90:0/1519895216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 msgr2=0x55ed55d4ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.873+0000 7f0950a00640 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 0x55ed55d4ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.873+0000 7f0950a00640 1 -- 172.21.15.90:0/1519895216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 msgr2=0x55ed55d4e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:44.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.873+0000 7f0950a00640 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d4e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.873+0000 7f0950a00640 1 -- 172.21.15.90:0/1519895216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed55dc6390 con 0x55ed55d3b000 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.873+0000 7f0950a00640 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3b000 0x55ed55d522d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f093c003d00 tx=0x7f093c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee99488968a9028 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.873+0000 7f093a000640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.873+0000 7f093a000640 1 -- 172.21.15.90:0/1519895216 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c02a470 con 0x55ed55d3b000 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.873+0000 7f093a000640 1 -- 172.21.15.90:0/1519895216 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f093c017030 con 0x55ed55d3b000 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f0955df0740 1 -- 172.21.15.90:0/1519895216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0944036070 con 0x55ed55d3b000 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f093a000640 1 -- 172.21.15.90:0/1519895216 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f093c02a610 con 0x55ed55d3b000 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f0955df0740 10 client.16617 fetch_fsmap learned FSMap version 103 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f0955df0740 1 -- 172.21.15.90:0/1519895216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x55ed55dc75f0 con 0x55ed55d3b000 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f093a000640 1 -- 172.21.15.90:0/1519895216 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0944036070 con 0x55ed55d3b000 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f093a000640 1 -- 172.21.15.90:0/1519895216 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 103) ==== 8434+0+0 (secure 0 0 0) 0x7f093c01c780 con 0x55ed55d3b000 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f0955df0740 10 client.16617 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T22:29:44.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f0955df0740 10 client.16617 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f0955df0740 20 client.16617 populate_metadata read hostname 'smithi090' 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f0955df0740 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ed55d49a80 0x55ed55dc9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f0955df0740 4 client.16617 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3fdd8644", "release": true, "await": true}] 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.874+0000 7f0955df0740 1 -- 172.21.15.90:0/1519895216 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3fdd8644", "release": true, "await": true}) -- 0x55ed55cb1c90 con 0x55ed55d49a80 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.875+0000 7f0943e00640 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ed55d49a80 0x55ed55dc9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.875+0000 7f0943e00640 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ed55d49a80 0x55ed55dc9ea0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3763d66c1995fb55 server_cookie=1a569926ed6796f1 in_seq=0 out_seq=0 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.875+0000 7f093a000640 10 client.16617 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f093a000640 1 -- 172.21.15.90:0/1519895216 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+592 (crc 0 0 0) 0x55ed55cb1c90 con 0x55ed55d49a80 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f093a000640 10 client.16617 handle_command_reply: tid=0 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 client.16617 shutdown 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 -- 172.21.15.90:0/1519895216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3b000 msgr2=0x55ed55d522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3b000 0x55ed55d522d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f093c003d00 tx=0x7f093c0046c0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 -- 172.21.15.90:0/1519895216 shutdown_connections 2024-09-10T22:29:44.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ed55d49a80 0x55ed55dc9ea0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ed55d3daf0 0x55ed55d4ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ed55d3d1d0 0x55ed55d4e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 --2- 172.21.15.90:0/1519895216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ed55d3b000 0x55ed55d522d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 -- 172.21.15.90:0/1519895216 >> 172.21.15.90:0/1519895216 conn(0x55ed55d3c5d0 msgr2=0x55ed55d54950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:44.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f093a000640 0 client.16617 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:44.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 -- 172.21.15.90:0/1519895216 shutdown_connections 2024-09-10T22:29:44.974 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:29:44.974 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 103, 2024-09-10T22:29:44.974 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 80, 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "3fdd8644": { 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 80, 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:29:44.975 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:44.976 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 1 -- 172.21.15.90:0/1519895216 wait complete. 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.879+0000 7f0955df0740 20 client.16617 trim_cache size 0 max 16384 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 -- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f09300b5580 msgr2=0x7f09300b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f09300b5580 0x7f09300b7a40 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55ed55d40350 tx=0x7f093c00b390 comp rx=0 tx=0).stop 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 -- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 msgr2=0x7f094c1a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 0x7f094c1a0a00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f0944005e30 tx=0x7f09440093d0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 -- 172.21.15.90:0/2839145237 shutdown_connections 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f09300b5580 0x7f09300b7a40 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f094c108fa0 0x7f094c1a0a00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f094c108620 0x7f094c19b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 --2- 172.21.15.90:0/2839145237 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f094c102c10 0x7f094c19b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:44.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 -- 172.21.15.90:0/2839145237 >> 172.21.15.90:0/2839145237 conn(0x7f094c0fe6c0 msgr2=0x7f094c106c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:44.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 -- 172.21.15.90:0/2839145237 shutdown_connections 2024-09-10T22:29:44.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:44.880+0000 7f0951e00640 1 -- 172.21.15.90:0/2839145237 wait complete. 2024-09-10T22:29:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:45 smithi090 ceph-mon[30350]: pgmap v1233: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-10T22:29:46.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:45 smithi171 ceph-mon[34138]: pgmap v1233: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-10T22:29:46.288 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3fdd8644', total seconds elapsed: 34.8 2024-09-10T22:29:46.288 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T22:29:46.288 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:29:46.320 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:29:46.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:45 smithi154 ceph-mon[34495]: pgmap v1233: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-09-10T22:29:46.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.914+0000 7f6449800640 1 -- 172.21.15.90:0/453661220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444102c10 msgr2=0x7f6444102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.914+0000 7f6449800640 1 --2- 172.21.15.90:0/453661220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444102c10 0x7f6444102ff0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6434009900 tx=0x7f643402a410 comp rx=0 tx=0).stop 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.914+0000 7f6449800640 1 -- 172.21.15.90:0/453661220 shutdown_connections 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.914+0000 7f6449800640 1 --2- 172.21.15.90:0/453661220 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6444108fa0 0x7f644410f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.914+0000 7f6449800640 1 --2- 172.21.15.90:0/453661220 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6444108620 0x7f6444108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.914+0000 7f6449800640 1 --2- 172.21.15.90:0/453661220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444102c10 0x7f6444102ff0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.914+0000 7f6449800640 1 -- 172.21.15.90:0/453661220 >> 172.21.15.90:0/453661220 conn(0x7f64440fe6c0 msgr2=0x7f6444100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.914+0000 7f6449800640 1 -- 172.21.15.90:0/453661220 shutdown_connections 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.914+0000 7f6449800640 1 -- 172.21.15.90:0/453661220 wait complete. 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.915+0000 7f6449800640 1 Processor -- start 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.915+0000 7f6449800640 1 -- start start 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.915+0000 7f6449800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6444102c10 0x7f6444109790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:46.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.915+0000 7f6449800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6444108620 0x7f6444109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.915+0000 7f6449800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444108fa0 0x7f644410a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.915+0000 7f6449800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6444112b10 con 0x7f6444108fa0 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.915+0000 7f6449800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6444112990 con 0x7f6444102c10 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.915+0000 7f6449800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6444112c90 con 0x7f6444108620 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6448e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444108fa0 0x7f644410a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6448e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444108fa0 0x7f644410a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54042/0 (socket says 172.21.15.90:54042) 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6448e00640 1 -- 172.21.15.90:0/2134240786 learned_addr learned my addr 172.21.15.90:0/2134240786 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6443e00640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6444102c10 0x7f6444109790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6443400640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6444108620 0x7f6444109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6448e00640 1 -- 172.21.15.90:0/2134240786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6444108620 msgr2=0x7f6444109cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6448e00640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6444108620 0x7f6444109cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6448e00640 1 -- 172.21.15.90:0/2134240786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6444102c10 msgr2=0x7f6444109790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6448e00640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6444102c10 0x7f6444109790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:46.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.916+0000 7f6448e00640 1 -- 172.21.15.90:0/2134240786 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64441ab760 con 0x7f6444108fa0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.917+0000 7f6448e00640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444108fa0 0x7f644410a210 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f643c00eb10 tx=0x7f643c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.917+0000 7f6440c00640 1 -- 172.21.15.90:0/2134240786 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c004b20 con 0x7f6444108fa0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.917+0000 7f6440c00640 1 -- 172.21.15.90:0/2134240786 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f643c004cc0 con 0x7f6444108fa0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.917+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64441aba50 con 0x7f6444108fa0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.917+0000 7f6440c00640 1 -- 172.21.15.90:0/2134240786 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c010690 con 0x7f6444108fa0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.917+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64441abf40 con 0x7f6444108fa0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.918+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6444102ff0 con 0x7f6444108fa0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.920+0000 7f6440c00640 1 -- 172.21.15.90:0/2134240786 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f643c005000 con 0x7f6444108fa0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.922+0000 7f6440c00640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f64280b5500 0x7f64280b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.922+0000 7f6440c00640 1 -- 172.21.15.90:0/2134240786 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f643c0e6c20 con 0x7f6444108fa0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.922+0000 7f6443e00640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f64280b5500 0x7f64280b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.923+0000 7f6443e00640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f64280b5500 0x7f64280b79c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f64340057c0 tx=0x7f64340058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:46.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:46.923+0000 7f6440c00640 1 -- 172.21.15.90:0/2134240786 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f643c056220 con 0x7f6444108fa0 2024-09-10T22:29:47.045 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:29:47.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.154+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f644410aa90 con 0x7f6444108fa0 2024-09-10T22:29:47.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.155+0000 7f6440c00640 1 -- 172.21.15.90:0/2134240786 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+12015 (secure 0 0 0) 0x7f644410aa90 con 0x7f6444108fa0 2024-09-10T22:29:47.157 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:29:47.157 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":103,"btime":"2024-09-10T22:29:17:913064+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":103,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:29:17.776682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:29:47.157 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 103 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f64280b5500 msgr2=0x7f64280b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f64280b5500 0x7f64280b79c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f64340057c0 tx=0x7f64340058d0 comp rx=0 tx=0).stop 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444108fa0 msgr2=0x7f644410a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444108fa0 0x7f644410a210 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f643c00eb10 tx=0x7f643c00c2d0 comp rx=0 tx=0).stop 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 shutdown_connections 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f64280b5500 0x7f64280b79c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6444108fa0 0x7f644410a210 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6444108620 0x7f6444109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 --2- 172.21.15.90:0/2134240786 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6444102c10 0x7f6444109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:47.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.157+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 >> 172.21.15.90:0/2134240786 conn(0x7f64440fe6c0 msgr2=0x7f6444104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.158+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 shutdown_connections 2024-09-10T22:29:47.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:47.158+0000 7f6449800640 1 -- 172.21.15.90:0/2134240786 wait complete. 2024-09-10T22:29:48.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:47 smithi171 ceph-mon[34138]: pgmap v1234: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T22:29:48.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:47 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2134240786' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:48.255 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:47 smithi090 ceph-mon[30350]: pgmap v1234: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T22:29:48.255 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:47 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2134240786' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:48.258 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db --cancel --all' 2024-09-10T22:29:48.258 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db --cancel --all 2024-09-10T22:29:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:47 smithi154 ceph-mon[34495]: pgmap v1234: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T22:29:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:47 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2134240786' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:48.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.507+0000 7f3cb1200640 1 -- 172.21.15.90:0/1628351697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac077ff0 msgr2=0x7f3cac113470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:48.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.507+0000 7f3cb1200640 1 --2- 172.21.15.90:0/1628351697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac077ff0 0x7f3cac113470 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009900 tx=0x7f3c9802a410 comp rx=0 tx=0).stop 2024-09-10T22:29:48.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.507+0000 7f3cb1200640 1 -- 172.21.15.90:0/1628351697 shutdown_connections 2024-09-10T22:29:48.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.507+0000 7f3cb1200640 1 --2- 172.21.15.90:0/1628351697 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cac1139b0 0x7f3cac115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.507+0000 7f3cb1200640 1 --2- 172.21.15.90:0/1628351697 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac077ff0 0x7f3cac113470 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.507+0000 7f3cb1200640 1 --2- 172.21.15.90:0/1628351697 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cac0776d0 0x7f3cac077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.507+0000 7f3cb1200640 1 -- 172.21.15.90:0/1628351697 >> 172.21.15.90:0/1628351697 conn(0x7f3cac1009f0 msgr2=0x7f3cac102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:48.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.507+0000 7f3cb1200640 1 -- 172.21.15.90:0/1628351697 shutdown_connections 2024-09-10T22:29:48.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.507+0000 7f3cb1200640 1 -- 172.21.15.90:0/1628351697 wait complete. 2024-09-10T22:29:48.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.508+0000 7f3cb1200640 1 Processor -- start 2024-09-10T22:29:48.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3cb1200640 1 -- start start 2024-09-10T22:29:48.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3cb1200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cac0776d0 0x7f3cac19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:48.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3cb1200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cac077ff0 0x7f3cac19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:48.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3cb1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac1139b0 0x7f3cac1a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:48.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3cb1200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1192a0 con 0x7f3cac1139b0 2024-09-10T22:29:48.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3cb1200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac119120 con 0x7f3cac077ff0 2024-09-10T22:29:48.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3cb1200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac119420 con 0x7f3cac0776d0 2024-09-10T22:29:48.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3caa800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac1139b0 0x7f3cac1a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:48.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3caa800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac1139b0 0x7f3cac1a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41076/0 (socket says 172.21.15.90:41076) 2024-09-10T22:29:48.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3caa800640 1 -- 172.21.15.90:0/316331820 learned_addr learned my addr 172.21.15.90:0/316331820 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:48.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.509+0000 7f3ca9e00640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cac0776d0 0x7f3cac19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:48.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.510+0000 7f3ca9400640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cac077ff0 0x7f3cac19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:48.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.510+0000 7f3caa800640 1 -- 172.21.15.90:0/316331820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cac0776d0 msgr2=0x7f3cac19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:48.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.510+0000 7f3caa800640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cac0776d0 0x7f3cac19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.510+0000 7f3caa800640 1 -- 172.21.15.90:0/316331820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cac077ff0 msgr2=0x7f3cac19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:48.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.510+0000 7f3caa800640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cac077ff0 0x7f3cac19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.510+0000 7f3caa800640 1 -- 172.21.15.90:0/316331820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac1a5850 con 0x7f3cac1139b0 2024-09-10T22:29:48.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.510+0000 7f3caa800640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac1139b0 0x7f3cac1a5080 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400eb40 tx=0x7f3ca400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:48.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.511+0000 7f3ca2a00640 1 -- 172.21.15.90:0/316331820 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4004b20 con 0x7f3cac1139b0 2024-09-10T22:29:48.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.511+0000 7f3ca2a00640 1 -- 172.21.15.90:0/316331820 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3ca4004cc0 con 0x7f3cac1139b0 2024-09-10T22:29:48.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.511+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cac1a5b40 con 0x7f3cac1139b0 2024-09-10T22:29:48.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.511+0000 7f3ca2a00640 1 -- 172.21.15.90:0/316331820 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4010690 con 0x7f3cac1139b0 2024-09-10T22:29:48.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.511+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cac1ac200 con 0x7f3cac1139b0 2024-09-10T22:29:48.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.512+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cac077ab0 con 0x7f3cac1139b0 2024-09-10T22:29:48.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.514+0000 7f3ca2a00640 1 -- 172.21.15.90:0/316331820 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f3ca4005540 con 0x7f3cac1139b0 2024-09-10T22:29:48.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.516+0000 7f3ca2a00640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c900b5450 0x7f3c900b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:48.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.516+0000 7f3ca2a00640 1 -- 172.21.15.90:0/316331820 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f3ca40e6b20 con 0x7f3cac1139b0 2024-09-10T22:29:48.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.516+0000 7f3ca9e00640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c900b5450 0x7f3c900b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:48.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.517+0000 7f3ca9e00640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c900b5450 0x7f3c900b7910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005d00 tx=0x7f3c9c00b410 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:48.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.520+0000 7f3ca2a00640 1 -- 172.21.15.90:0/316331820 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3ca40ad2d0 con 0x7f3cac1139b0 2024-09-10T22:29:48.677 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:29:48.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.754+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cac1ac560 con 0x7f3cac1139b0 2024-09-10T22:29:48.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.755+0000 7f3ca2a00640 1 -- 172.21.15.90:0/316331820 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+12015 (secure 0 0 0) 0x7f3ca40acbd0 con 0x7f3cac1139b0 2024-09-10T22:29:48.757 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:29:48.757 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":103,"btime":"2024-09-10T22:29:17:913064+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":103,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:29:17.776682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:29:48.757 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 103 2024-09-10T22:29:48.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.757+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c900b5450 msgr2=0x7f3c900b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:48.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.757+0000 7f3cb1200640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c900b5450 0x7f3c900b7910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c005d00 tx=0x7f3c9c00b410 comp rx=0 tx=0).stop 2024-09-10T22:29:48.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.757+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac1139b0 msgr2=0x7f3cac1a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:48.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.757+0000 7f3cb1200640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac1139b0 0x7f3cac1a5080 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400eb40 tx=0x7f3ca400c2d0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.758+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 shutdown_connections 2024-09-10T22:29:48.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.758+0000 7f3cb1200640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f3c900b5450 0x7f3c900b7910 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.758+0000 7f3cb1200640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cac1139b0 0x7f3cac1a5080 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.758+0000 7f3cb1200640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f3cac077ff0 0x7f3cac19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.758+0000 7f3cb1200640 1 --2- 172.21.15.90:0/316331820 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f3cac0776d0 0x7f3cac19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:48.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.758+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 >> 172.21.15.90:0/316331820 conn(0x7f3cac1009f0 msgr2=0x7f3cac102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:48.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.758+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 shutdown_connections 2024-09-10T22:29:48.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:48.758+0000 7f3cb1200640 1 -- 172.21.15.90:0/316331820 wait complete. 2024-09-10T22:29:49.065 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:48 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/316331820' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:48 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/316331820' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:48 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/316331820' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:29:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:49 smithi171 ceph-mon[34138]: pgmap v1235: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T22:29:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:49 smithi090 ceph-mon[30350]: pgmap v1235: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T22:29:50.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:49 smithi154 ceph-mon[34495]: pgmap v1235: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T22:29:50.774 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T22:29:50.775 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.34703 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T22:29:50.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.861+0000 7f96ea800640 1 -- 172.21.15.90:0/298389474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e4075540 msgr2=0x7f96e4075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:50.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.861+0000 7f96ea800640 1 --2- 172.21.15.90:0/298389474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e4075540 0x7f96e4075920 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009900 tx=0x7f96d402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.862+0000 7f96ea800640 1 -- 172.21.15.90:0/298389474 shutdown_connections 2024-09-10T22:29:50.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.862+0000 7f96ea800640 1 --2- 172.21.15.90:0/298389474 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e410f3e0 0x7f96e41117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.862+0000 7f96ea800640 1 --2- 172.21.15.90:0/298389474 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e4075f40 0x7f96e410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.862+0000 7f96ea800640 1 --2- 172.21.15.90:0/298389474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e4075540 0x7f96e4075920 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.862+0000 7f96ea800640 1 -- 172.21.15.90:0/298389474 >> 172.21.15.90:0/298389474 conn(0x7f96e40fe830 msgr2=0x7f96e4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.862+0000 7f96ea800640 1 -- 172.21.15.90:0/298389474 shutdown_connections 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.862+0000 7f96ea800640 1 -- 172.21.15.90:0/298389474 wait complete. 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96ea800640 1 Processor -- start 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96ea800640 1 -- start start 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96ea800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e4075540 0x7f96e419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96ea800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e4075f40 0x7f96e419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96ea800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e410f3e0 0x7f96e41a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96ea800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4114e20 con 0x7f96e410f3e0 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96ea800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4114ca0 con 0x7f96e4075f40 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96ea800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96e4114fa0 con 0x7f96e4075540 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e410f3e0 0x7f96e41a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e410f3e0 0x7f96e41a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41092/0 (socket says 172.21.15.90:41092) 2024-09-10T22:29:50.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.863+0000 7f96e3e00640 1 -- 172.21.15.90:0/2282564625 learned_addr learned my addr 172.21.15.90:0/2282564625 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.864+0000 7f96e3400640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e4075540 0x7f96e419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.864+0000 7f96daa00640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e4075f40 0x7f96e419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.864+0000 7f96e3e00640 1 -- 172.21.15.90:0/2282564625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e4075540 msgr2=0x7f96e419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.864+0000 7f96e3e00640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e4075540 0x7f96e419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.864+0000 7f96e3e00640 1 -- 172.21.15.90:0/2282564625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e4075f40 msgr2=0x7f96e419b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.864+0000 7f96e3e00640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e4075f40 0x7f96e419b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.864+0000 7f96e3e00640 1 -- 172.21.15.90:0/2282564625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e41a1480 con 0x7f96e410f3e0 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.864+0000 7f96e3e00640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e410f3e0 0x7f96e41a0cb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00dac0 tx=0x7f96dc00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.865+0000 7f96e0c00640 1 -- 172.21.15.90:0/2282564625 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc017070 con 0x7f96e410f3e0 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.865+0000 7f96ea800640 1 -- 172.21.15.90:0/2282564625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e41a1770 con 0x7f96e410f3e0 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.865+0000 7f96e0c00640 1 -- 172.21.15.90:0/2282564625 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96dc00be30 con 0x7f96e410f3e0 2024-09-10T22:29:50.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.865+0000 7f96e0c00640 1 -- 172.21.15.90:0/2282564625 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc010460 con 0x7f96e410f3e0 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.865+0000 7f96ea800640 1 -- 172.21.15.90:0/2282564625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41a7e30 con 0x7f96e410f3e0 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.868+0000 7f96ed1d5740 1 Processor -- start 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.868+0000 7f96e0c00640 1 -- 172.21.15.90:0/2282564625 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f96dc023060 con 0x7f96e410f3e0 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.868+0000 7f96ed1d5740 1 -- start start 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.868+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a0a0 0x564280f9a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.868+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a9c0 0x564280fa6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.868+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.868+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564280f11ff0 con 0x564280fa6950 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.868+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564280f01420 con 0x564280f9a0a0 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.868+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564280f05230 con 0x564280f9a9c0 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96daa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96daa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41094/0 (socket says 172.21.15.90:41094) 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96daa00640 1 -- 172.21.15.90:0/1604391198 learned_addr learned my addr 172.21.15.90:0/1604391198 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:50.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96daa00640 1 --2- 172.21.15.90:0/1604391198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a0a0 0x564280f9a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96e3400640 1 --2- 172.21.15.90:0/1604391198 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a9c0 0x564280fa6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96daa00640 1 -- 172.21.15.90:0/1604391198 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a9c0 msgr2=0x564280fa6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96daa00640 1 --2- 172.21.15.90:0/1604391198 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a9c0 0x564280fa6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96daa00640 1 -- 172.21.15.90:0/1604391198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a0a0 msgr2=0x564280f9a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96daa00640 1 --2- 172.21.15.90:0/1604391198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a0a0 0x564280f9a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.869+0000 7f96daa00640 1 -- 172.21.15.90:0/1604391198 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564280c5f8b0 con 0x564280fa6950 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.870+0000 7f96daa00640 1 --2- 172.21.15.90:0/1604391198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa8d40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0127f0 tx=0x7f96cc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81cd585b99cb8e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.870+0000 7f96e0c00640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f96c80b5450 0x7f96c80b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.870+0000 7f96d8c00640 1 -- 172.21.15.90:0/1604391198 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc012e30 con 0x564280fa6950 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.870+0000 7f96e0c00640 1 -- 172.21.15.90:0/2282564625 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f96dc0e59f0 con 0x7f96e410f3e0 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.870+0000 7f96d8c00640 1 -- 172.21.15.90:0/1604391198 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96cc010c00 con 0x564280fa6950 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.870+0000 7f96d8c00640 1 -- 172.21.15.90:0/1604391198 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc01e970 con 0x564280fa6950 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.870+0000 7f96e3400640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f96c80b5450 0x7f96c80b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1604391198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 msgr2=0x564280fa8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1604391198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa8d40 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0127f0 tx=0x7f96cc00b7c0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96e3400640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f96c80b5450 0x7f96c80b7910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f96d40057c0 tx=0x7f96d40058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1604391198 shutdown_connections 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1604391198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa8d40 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1604391198 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a9c0 0x564280fa6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1604391198 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a0a0 0x564280f9a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1604391198 >> 172.21.15.90:0/1604391198 conn(0x564280f97ed0 msgr2=0x564280f99070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1604391198 shutdown_connections 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1604391198 wait complete. 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.871+0000 7f96ed1d5740 1 -- start start 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96ed1d5740 1 Processor -- start 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280f9a0a0 0x564280fa62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fa3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280fa6950 0x564280fa3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564280d62810 con 0x564280f9a0a0 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x564280e9c550 con 0x564280f9a9c0 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x564280fa9600 con 0x564280fa6950 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96ed1d5740 10 client.0 _ll_register_callbacks cb 0x564280eac040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96e3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280f9a0a0 0x564280fa62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96e3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280f9a0a0 0x564280fa62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41108/0 (socket says 172.21.15.90:41108) 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.872+0000 7f96e3400640 1 -- 172.21.15.90:0/1675051872 learned_addr learned my addr 172.21.15.90:0/1675051872 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96e3400640 1 -- 172.21.15.90:0/1675051872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280fa6950 msgr2=0x564280fa3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96e3400640 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280fa6950 0x564280fa3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96e3400640 1 -- 172.21.15.90:0/1675051872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 msgr2=0x564280fa3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96e3400640 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fa3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96e3400640 1 -- 172.21.15.90:0/1675051872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564280e8e200 con 0x564280f9a0a0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96e3400640 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280f9a0a0 0x564280fa62c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f96d4005ec0 tx=0x7f96d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11412766d3ff6c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96d1600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96d1600640 1 -- 172.21.15.90:0/1675051872 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4035040 con 0x564280f9a0a0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96d1600640 1 -- 172.21.15.90:0/1675051872 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f96d4003a00 con 0x564280f9a0a0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.873+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1675051872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96cc010c00 con 0x564280f9a0a0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.874+0000 7f96d1600640 1 -- 172.21.15.90:0/1675051872 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f96d4003c00 con 0x564280f9a0a0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.874+0000 7f96ed1d5740 10 client.16641 fetch_fsmap learned FSMap version 103 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.874+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1675051872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x564280fb4fb0 con 0x564280f9a0a0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.874+0000 7f96d1600640 1 -- 172.21.15.90:0/1675051872 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96cc010c00 con 0x564280f9a0a0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.875+0000 7f96d1600640 1 -- 172.21.15.90:0/1675051872 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 103) ==== 8434+0+0 (secure 0 0 0) 0x7f96d4044030 con 0x564280f9a0a0 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.875+0000 7f96ed1d5740 10 client.16641 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.875+0000 7f96ed1d5740 10 client.16641 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.875+0000 7f96ed1d5740 20 client.16641 populate_metadata read hostname 'smithi090' 2024-09-10T22:29:50.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.875+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564280fb6510 0x564280fb89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.875+0000 7f96ed1d5740 4 client.16641 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.875+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1675051872 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564280f13f40 con 0x564280fb6510 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.875+0000 7f96daa00640 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564280fb6510 0x564280fb89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.876+0000 7f96daa00640 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564280fb6510 0x564280fb89d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=399324096ffa059a server_cookie=e1719ab24d0386d6 in_seq=0 out_seq=0 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.876+0000 7f96d1600640 10 client.16641 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96d1600640 1 -- 172.21.15.90:0/1675051872 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564280f13f40 con 0x564280fb6510 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96d1600640 10 client.16641 handle_command_reply: tid=0 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96ed1d5740 1 client.16641 shutdown 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1675051872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280f9a0a0 msgr2=0x564280fa62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280f9a0a0 0x564280fa62c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f96d4005ec0 tx=0x7f96d4009260 comp rx=0 tx=0).stop 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1675051872 shutdown_connections 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564280fb6510 0x564280fb89d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280fa6950 0x564280fa3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fa3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1675051872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280f9a0a0 0x564280fa62c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1675051872 >> 172.21.15.90:0/1675051872 conn(0x564280f97ed0 msgr2=0x564280f9da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.877+0000 7f96d1600640 0 client.16641 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.878+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1675051872 shutdown_connections 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.878+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1675051872 wait complete. 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.878+0000 7f96ed1d5740 20 client.16641 trim_cache size 0 max 16384 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.896+0000 7f96ed1d5740 1 Processor -- start 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.896+0000 7f96ed1d5740 1 -- start start 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.896+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 0x564280fa2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fb2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b2050 con 0x564280fa6950 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b3e30 con 0x564280f9a9c0 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b27e0 con 0x564280f9a0a0 2024-09-10T22:29:50.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41112/0 (socket says 172.21.15.90:41112) 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96e3e00640 1 -- 172.21.15.90:0/1841422230 learned_addr learned my addr 172.21.15.90:0/1841422230 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96e3400640 1 --2- 172.21.15.90:0/1841422230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 0x564280fa2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96daa00640 1 --2- 172.21.15.90:0/1841422230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fb2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96e3e00640 1 -- 172.21.15.90:0/1841422230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 msgr2=0x564280fa2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96e3e00640 1 --2- 172.21.15.90:0/1841422230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 0x564280fa2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96e3e00640 1 -- 172.21.15.90:0/1841422230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 msgr2=0x564280fb2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96e3e00640 1 --2- 172.21.15.90:0/1841422230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fb2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.897+0000 7f96e3e00640 1 -- 172.21.15.90:0/1841422230 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564280fa1c10 con 0x564280fa6950 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.898+0000 7f96e3e00640 1 --2- 172.21.15.90:0/1841422230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa5450 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0a8d60 tx=0x7f96dc0ab260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20896fa23bdacf80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.898+0000 7f96d3e00640 1 -- 172.21.15.90:0/1841422230 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc020070 con 0x564280fa6950 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.898+0000 7f96d3e00640 1 -- 172.21.15.90:0/1841422230 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96dc0b0520 con 0x564280fa6950 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.898+0000 7f96d3e00640 1 -- 172.21.15.90:0/1841422230 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0a7d60 con 0x564280fa6950 2024-09-10T22:29:50.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1841422230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 msgr2=0x564280fa5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1841422230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa5450 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0a8d60 tx=0x7f96dc0ab260 comp rx=0 tx=0).stop 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1841422230 shutdown_connections 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1841422230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280fa5450 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1841422230 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fb2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1841422230 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 0x564280fa2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1841422230 >> 172.21.15.90:0/1841422230 conn(0x564280f97ed0 msgr2=0x564280fa4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1841422230 shutdown_connections 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1841422230 wait complete. 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 -- start start 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.899+0000 7f96ed1d5740 1 Processor -- start 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 0x564280fa9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fa9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96ed1d5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280faa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b36a0 con 0x564280fa6950 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b2f70 con 0x564280f9a9c0 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96ed1d5740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f96e41b0050 con 0x564280f9a0a0 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96ed1d5740 10 client.0 _ll_register_callbacks cb 0x7f96d4031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280faa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280faa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41116/0 (socket says 172.21.15.90:41116) 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.900+0000 7f96e3e00640 1 -- 172.21.15.90:0/1900600210 learned_addr learned my addr 172.21.15.90:0/1900600210 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96e3e00640 1 -- 172.21.15.90:0/1900600210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 msgr2=0x564280fa9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96e3400640 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 0x564280fa9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96e3e00640 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 0x564280fa9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96daa00640 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fa9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96e3e00640 1 -- 172.21.15.90:0/1900600210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 msgr2=0x564280fa9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96e3e00640 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fa9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96e3e00640 1 -- 172.21.15.90:0/1900600210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642810235b0 con 0x564280fa6950 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96e3400640 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 0x564280fa9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96e3e00640 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280faa300 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0a80c0 tx=0x7f96dc0a7270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d61d3d0adab9d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96d1600640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96d1600640 1 -- 172.21.15.90:0/1900600210 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc020030 con 0x564280fa6950 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96d1600640 1 -- 172.21.15.90:0/1900600210 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f96dc0a7d60 con 0x564280fa6950 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1900600210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96dc0b0520 con 0x564280fa6950 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96d1600640 1 -- 172.21.15.90:0/1900600210 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f96dc0aad50 con 0x564280fa6950 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96ed1d5740 10 client.16647 fetch_fsmap learned FSMap version 103 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.901+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1900600210 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x5642810248d0 con 0x564280fa6950 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.902+0000 7f96d1600640 1 -- 172.21.15.90:0/1900600210 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f96dc0b0520 con 0x564280fa6950 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.902+0000 7f96d1600640 1 -- 172.21.15.90:0/1900600210 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 103) ==== 8434+0+0 (secure 0 0 0) 0x7f96dc0aa9c0 con 0x564280fa6950 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.902+0000 7f96ed1d5740 10 client.16647 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T22:29:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.902+0000 7f96ed1d5740 10 client.16647 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.902+0000 7f96ed1d5740 20 client.16647 populate_metadata read hostname 'smithi090' 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.902+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564281025130 0x564281027550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.902+0000 7f96ed1d5740 4 client.16647 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.902+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1900600210 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x564280f0f9f0 con 0x564281025130 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.902+0000 7f96e3400640 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564281025130 0x564281027550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.903+0000 7f96e3400640 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564281025130 0x564281027550 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a918fee8af87e9 server_cookie=80c4a46c58fd1ad8 in_seq=0 out_seq=0 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.903+0000 7f96d1600640 10 client.16647 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.903+0000 7f96d1600640 1 -- 172.21.15.90:0/1900600210 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+81 (crc 0 0 0) 0x564280f0f9f0 con 0x564281025130 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.903+0000 7f96d1600640 10 client.16647 handle_command_reply: tid=0 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.903+0000 7f96ed1d5740 1 client.16647 shutdown 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1900600210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 msgr2=0x564280faa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280faa300 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0a80c0 tx=0x7f96dc0a7270 comp rx=0 tx=0).stop 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1900600210 shutdown_connections 2024-09-10T22:29:50.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x564281025130 0x564281027550 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564280fa6950 0x564280faa300 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x564280f9a9c0 0x564280fa9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96d1600640 0 client.16647 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 --2- 172.21.15.90:0/1900600210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x564280f9a0a0 0x564280fa9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1900600210 >> 172.21.15.90:0/1900600210 conn(0x564280f97ed0 msgr2=0x564280fa5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1900600210 shutdown_connections 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 1 -- 172.21.15.90:0/1900600210 wait complete. 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.904+0000 7f96ed1d5740 20 client.16647 trim_cache size 0 max 16384 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 103, 2024-09-10T22:29:50.946 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 80, 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": {} 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 -- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f96c80b5450 msgr2=0x7f96c80b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f96c80b5450 0x7f96c80b7910 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f96d40057c0 tx=0x7f96d40058d0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 -- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e410f3e0 msgr2=0x7f96e41a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e410f3e0 0x7f96e41a0cb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00dac0 tx=0x7f96dc00df80 comp rx=0 tx=0).stop 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 -- 172.21.15.90:0/2282564625 shutdown_connections 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f96c80b5450 0x7f96c80b7910 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96e410f3e0 0x7f96e41a0cb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f96e4075f40 0x7f96e419b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 --2- 172.21.15.90:0/2282564625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f96e4075540 0x7f96e419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:50.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 -- 172.21.15.90:0/2282564625 >> 172.21.15.90:0/2282564625 conn(0x7f96e40fe830 msgr2=0x7f96e40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:50.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 -- 172.21.15.90:0/2282564625 shutdown_connections 2024-09-10T22:29:50.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:49.905+0000 7f96ea800640 1 -- 172.21.15.90:0/2282564625 wait complete. 2024-09-10T22:29:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:51 smithi090 ceph-mon[30350]: pgmap v1236: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2024-09-10T22:29:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:52 smithi154 ceph-mon[34495]: pgmap v1236: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2024-09-10T22:29:52.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:52 smithi171 ceph-mon[34138]: pgmap v1236: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2024-09-10T22:29:53.392 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:29:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:54 smithi090 ceph-mon[30350]: pgmap v1237: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T22:29:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.248+0000 7f4524e00640 1 -- 172.21.15.90:0/3979126076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f452010d140 msgr2=0x7f452010f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.248+0000 7f4524e00640 1 --2- 172.21.15.90:0/3979126076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f452010d140 0x7f452010f570 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f4518009900 tx=0x7f451802a3f0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.248+0000 7f4524e00640 1 -- 172.21.15.90:0/3979126076 shutdown_connections 2024-09-10T22:29:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.248+0000 7f4524e00640 1 --2- 172.21.15.90:0/3979126076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f452010d140 0x7f452010f570 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.248+0000 7f4524e00640 1 --2- 172.21.15.90:0/3979126076 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4520104590 0x7f452010cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.248+0000 7f4524e00640 1 --2- 172.21.15.90:0/3979126076 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4520103b90 0x7f4520103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.248+0000 7f4524e00640 1 -- 172.21.15.90:0/3979126076 >> 172.21.15.90:0/3979126076 conn(0x7f45200fd700 msgr2=0x7f45200ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.248+0000 7f4524e00640 1 -- 172.21.15.90:0/3979126076 shutdown_connections 2024-09-10T22:29:54.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.248+0000 7f4524e00640 1 -- 172.21.15.90:0/3979126076 wait complete. 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.249+0000 7f4524e00640 1 Processor -- start 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.249+0000 7f4524e00640 1 -- start start 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.249+0000 7f4524e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4520103b90 0x7f452019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.249+0000 7f4524e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4520104590 0x7f452019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.249+0000 7f4524e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452010d140 0x7f45201a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.249+0000 7f4524e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4520112c80 con 0x7f4520104590 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.249+0000 7f4524e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4520112b00 con 0x7f452010d140 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.249+0000 7f4524e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4520112e00 con 0x7f4520103b90 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.249+0000 7f451d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4520104590 0x7f452019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4520104590 0x7f452019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41134/0 (socket says 172.21.15.90:41134) 2024-09-10T22:29:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451d400640 1 -- 172.21.15.90:0/904951803 learned_addr learned my addr 172.21.15.90:0/904951803 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451de00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4520103b90 0x7f452019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451e800640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452010d140 0x7f45201a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451d400640 1 -- 172.21.15.90:0/904951803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4520103b90 msgr2=0x7f452019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451d400640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4520103b90 0x7f452019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451d400640 1 -- 172.21.15.90:0/904951803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452010d140 msgr2=0x7f45201a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451d400640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452010d140 0x7f45201a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451d400640 1 -- 172.21.15.90:0/904951803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45201a1430 con 0x7f4520104590 2024-09-10T22:29:54.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.250+0000 7f451d400640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4520104590 0x7f452019b8c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f450c00d950 tx=0x7f450c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:54.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.251+0000 7f4516a00640 1 -- 172.21.15.90:0/904951803 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c017070 con 0x7f4520104590 2024-09-10T22:29:54.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.251+0000 7f4516a00640 1 -- 172.21.15.90:0/904951803 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f450c009c30 con 0x7f4520104590 2024-09-10T22:29:54.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.251+0000 7f4524e00640 1 -- 172.21.15.90:0/904951803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45201a1720 con 0x7f4520104590 2024-09-10T22:29:54.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.251+0000 7f4516a00640 1 -- 172.21.15.90:0/904951803 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00f620 con 0x7f4520104590 2024-09-10T22:29:54.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.251+0000 7f4524e00640 1 -- 172.21.15.90:0/904951803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45201a7fb0 con 0x7f4520104590 2024-09-10T22:29:54.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.255+0000 7f45278d8740 1 Processor -- start 2024-09-10T22:29:54.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.255+0000 7f45278d8740 1 -- start start 2024-09-10T22:29:54.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.255+0000 7f45278d8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.255+0000 7f45278d8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 0x55ba468bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.255+0000 7f45278d8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468bfa30 0x55ba468c1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f45278d8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ba46828ca0 con 0x55ba468bfa30 2024-09-10T22:29:54.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f45278d8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ba468180d0 con 0x55ba468b3180 2024-09-10T22:29:54.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f45278d8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ba4681bee0 con 0x55ba468b3aa0 2024-09-10T22:29:54.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451de00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 0x55ba468bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451e800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.154:3300/0 says I am v2:172.21.15.90:43914/0 (socket says 172.21.15.90:43914) 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451e800640 1 -- 172.21.15.90:0/4067702206 learned_addr learned my addr 172.21.15.90:0/4067702206 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f4516a00640 1 -- 172.21.15.90:0/904951803 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f450c010460 con 0x7f4520104590 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451e800640 1 -- 172.21.15.90:0/4067702206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 msgr2=0x55ba468bf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451e800640 1 --2- 172.21.15.90:0/4067702206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 0x55ba468bf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451e800640 1 -- 172.21.15.90:0/4067702206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468bfa30 msgr2=0x55ba468c1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451e800640 1 --2- 172.21.15.90:0/4067702206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468bfa30 0x55ba468c1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451e800640 1 -- 172.21.15.90:0/4067702206 --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba467ca400 con 0x55ba468b3180 2024-09-10T22:29:54.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.256+0000 7f451de00640 1 --2- 172.21.15.90:0/4067702206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 0x55ba468bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:29:54.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.257+0000 7f451e800640 1 --2- 172.21.15.90:0/4067702206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b3560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f45180057c0 tx=0x7f4518004310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94ff5b6d3d9bda88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:54.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.257+0000 7f450b400640 1 -- 172.21.15.90:0/4067702206 <== mon.2 v2:172.21.15.154:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518035030 con 0x55ba468b3180 2024-09-10T22:29:54.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.257+0000 7f450b400640 1 -- 172.21.15.90:0/4067702206 <== mon.2 v2:172.21.15.154:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4518040070 con 0x55ba468b3180 2024-09-10T22:29:54.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.257+0000 7f450b400640 1 -- 172.21.15.90:0/4067702206 <== mon.2 v2:172.21.15.154:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451803c950 con 0x55ba468b3180 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f4516a00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45040b54d0 0x7f45040b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f45278d8740 1 -- 172.21.15.90:0/4067702206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 msgr2=0x55ba468b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f45278d8740 1 --2- 172.21.15.90:0/4067702206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b3560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f45180057c0 tx=0x7f4518004310 comp rx=0 tx=0).stop 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f45278d8740 1 -- 172.21.15.90:0/4067702206 shutdown_connections 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f45278d8740 1 --2- 172.21.15.90:0/4067702206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468bfa30 0x55ba468c1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f45278d8740 1 --2- 172.21.15.90:0/4067702206 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 0x55ba468bf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f45278d8740 1 --2- 172.21.15.90:0/4067702206 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b3560 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f45278d8740 1 -- 172.21.15.90:0/4067702206 >> 172.21.15.90:0/4067702206 conn(0x55ba468b0fb0 msgr2=0x55ba468b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f4516a00640 1 -- 172.21.15.90:0/904951803 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f450c0e5f20 con 0x7f4520104590 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f45278d8740 1 -- 172.21.15.90:0/4067702206 shutdown_connections 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.258+0000 7f451de00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45040b54d0 0x7f45040b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.259+0000 7f45278d8740 1 -- 172.21.15.90:0/4067702206 wait complete. 2024-09-10T22:29:54.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.259+0000 7f45278d8740 1 -- start start 2024-09-10T22:29:54.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.259+0000 7f451de00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45040b54d0 0x7f45040b7990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55ba468b1e90 tx=0x7f451000a570 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:54.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.259+0000 7f45278d8740 1 Processor -- start 2024-09-10T22:29:54.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f45278d8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3180 0x55ba468bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f45278d8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468bc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f45278d8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 0x55ba468bcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f45278d8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ba466797a0 con 0x55ba468b3aa0 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f45278d8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x55ba467bda10 con 0x55ba468bfa30 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f45278d8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x55ba468c2650 con 0x55ba468b3180 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f45278d8740 10 client.0 _ll_register_callbacks cb 0x55ba467cdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f451d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468bc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f451d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468bc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41146/0 (socket says 172.21.15.90:41146) 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f451d400640 1 -- 172.21.15.90:0/3583806971 learned_addr learned my addr 172.21.15.90:0/3583806971 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f451de00640 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3180 0x55ba468bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.260+0000 7f451e800640 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 0x55ba468bcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f451d400640 1 -- 172.21.15.90:0/3583806971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3180 msgr2=0x55ba468bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f451d400640 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3180 0x55ba468bf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f451d400640 1 -- 172.21.15.90:0/3583806971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 msgr2=0x55ba468bcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:54.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f451d400640 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 0x55ba468bcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f451d400640 1 -- 172.21.15.90:0/3583806971 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba46585b90 con 0x55ba468b3aa0 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f451d400640 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468bc830 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f450c0abfb0 tx=0x7f450c0ab850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df218c15aa6535ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f44ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f44ffe00640 1 -- 172.21.15.90:0/3583806971 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c020070 con 0x55ba468b3aa0 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f44ffe00640 1 -- 172.21.15.90:0/3583806971 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f450c0ab280 con 0x55ba468b3aa0 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f45278d8740 1 -- 172.21.15.90:0/3583806971 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4518040070 con 0x55ba468b3aa0 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f44ffe00640 1 -- 172.21.15.90:0/3583806971 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f450c0e36c0 con 0x55ba468b3aa0 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f45278d8740 10 client.16656 fetch_fsmap learned FSMap version 103 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.261+0000 7f45278d8740 1 -- 172.21.15.90:0/3583806971 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x55ba468ce0c0 con 0x55ba468b3aa0 2024-09-10T22:29:54.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.262+0000 7f44ffe00640 1 -- 172.21.15.90:0/3583806971 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4518040070 con 0x55ba468b3aa0 2024-09-10T22:29:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.262+0000 7f44ffe00640 1 -- 172.21.15.90:0/3583806971 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 103) ==== 8434+0+0 (secure 0 0 0) 0x7f450c0af060 con 0x55ba468b3aa0 2024-09-10T22:29:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.262+0000 7f45278d8740 10 client.16656 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T22:29:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.262+0000 7f45278d8740 10 client.16656 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:29:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.262+0000 7f45278d8740 20 client.16656 populate_metadata read hostname 'smithi090' 2024-09-10T22:29:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.262+0000 7f45278d8740 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ba468cf610 0x55ba468d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.262+0000 7f45278d8740 4 client.16656 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T22:29:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.262+0000 7f45278d8740 1 -- 172.21.15.90:0/3583806971 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba4682abf0 con 0x55ba468cf610 2024-09-10T22:29:54.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.263+0000 7f451de00640 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ba468cf610 0x55ba468d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.263+0000 7f451de00640 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ba468cf610 0x55ba468d1a50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a2e4960a0eeea15 server_cookie=22b0edd290da6c2 in_seq=0 out_seq=0 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.263+0000 7f44ffe00640 10 client.16656 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.264+0000 7f44ffe00640 1 -- 172.21.15.90:0/3583806971 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ba4682abf0 con 0x55ba468cf610 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.264+0000 7f44ffe00640 10 client.16656 handle_command_reply: tid=0 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.264+0000 7f45278d8740 1 client.16656 shutdown 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.264+0000 7f45278d8740 1 -- 172.21.15.90:0/3583806971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 msgr2=0x55ba468bc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.264+0000 7f45278d8740 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468bc830 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f450c0abfb0 tx=0x7f450c0ab850 comp rx=0 tx=0).stop 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f45278d8740 1 -- 172.21.15.90:0/3583806971 shutdown_connections 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f45278d8740 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ba468cf610 0x55ba468d1a50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f45278d8740 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 0x55ba468bcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f45278d8740 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468bc830 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f45278d8740 1 --2- 172.21.15.90:0/3583806971 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3180 0x55ba468bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f45278d8740 1 -- 172.21.15.90:0/3583806971 >> 172.21.15.90:0/3583806971 conn(0x55ba468b0fb0 msgr2=0x55ba468b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:54.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f44ffe00640 0 client.16656 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f45278d8740 1 -- 172.21.15.90:0/3583806971 shutdown_connections 2024-09-10T22:29:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f45278d8740 1 -- 172.21.15.90:0/3583806971 wait complete. 2024-09-10T22:29:54.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.265+0000 7f45278d8740 20 client.16656 trim_cache size 0 max 16384 2024-09-10T22:29:54.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.282+0000 7f45278d8740 1 Processor -- start 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.282+0000 7f45278d8740 1 -- start start 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f45278d8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f45278d8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468b9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f45278d8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468bfa30 0x55ba468b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f45278d8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45201b1fd0 con 0x55ba468b3aa0 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f45278d8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f45201b3db0 con 0x55ba468b3180 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f45278d8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f45201b2760 con 0x55ba468bfa30 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468b9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468b9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41158/0 (socket says 172.21.15.90:41158) 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451d400640 1 -- 172.21.15.90:0/1541944477 learned_addr learned my addr 172.21.15.90:0/1541944477 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451e800640 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468bfa30 0x55ba468b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451de00640 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451d400640 1 -- 172.21.15.90:0/1541944477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468bfa30 msgr2=0x55ba468b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451d400640 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468bfa30 0x55ba468b9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451d400640 1 -- 172.21.15.90:0/1541944477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 msgr2=0x55ba468b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451d400640 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451d400640 1 -- 172.21.15.90:0/1541944477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba468b1a30 con 0x55ba468b3aa0 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.283+0000 7f451e800640 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468bfa30 0x55ba468b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.284+0000 7f451d400640 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468b9a60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f450c0058c0 tx=0x7f450c0ab8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6c35cb00d1f1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.284+0000 7f450aa00640 1 -- 172.21.15.90:0/1541944477 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c0a8670 con 0x55ba468b3aa0 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.284+0000 7f450aa00640 1 -- 172.21.15.90:0/1541944477 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f450c0b5500 con 0x55ba468b3aa0 2024-09-10T22:29:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.284+0000 7f450aa00640 1 -- 172.21.15.90:0/1541944477 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c0e34c0 con 0x55ba468b3aa0 2024-09-10T22:29:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 -- 172.21.15.90:0/1541944477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 msgr2=0x55ba468b9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468b9a60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f450c0058c0 tx=0x7f450c0ab8c0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 -- 172.21.15.90:0/1541944477 shutdown_connections 2024-09-10T22:29:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468bfa30 0x55ba468b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3aa0 0x55ba468b9a60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 --2- 172.21.15.90:0/1541944477 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468b3180 0x55ba468b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 -- 172.21.15.90:0/1541944477 >> 172.21.15.90:0/1541944477 conn(0x55ba468b0fb0 msgr2=0x55ba468be680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 -- 172.21.15.90:0/1541944477 shutdown_connections 2024-09-10T22:29:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 -- 172.21.15.90:0/1541944477 wait complete. 2024-09-10T22:29:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.285+0000 7f45278d8740 1 -- start start 2024-09-10T22:29:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f45278d8740 1 Processor -- start 2024-09-10T22:29:54.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f45278d8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3180 0x55ba468c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f45278d8740 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 0x55ba468c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f45278d8740 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 0x55ba4693ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f45278d8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f45201b3620 con 0x55ba468b3180 2024-09-10T22:29:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f45278d8740 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f45201b2ef0 con 0x55ba468bfa30 2024-09-10T22:29:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f45278d8740 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f45201affd0 con 0x55ba468b3aa0 2024-09-10T22:29:54.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f45278d8740 10 client.0 _ll_register_callbacks cb 0x7f450c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T22:29:54.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f451de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3180 0x55ba468c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f451de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3180 0x55ba468c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41164/0 (socket says 172.21.15.90:41164) 2024-09-10T22:29:54.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f451de00640 1 -- 172.21.15.90:0/1989793349 learned_addr learned my addr 172.21.15.90:0/1989793349 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:54.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f451e800640 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 0x55ba4693ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.286+0000 7f451d400640 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 0x55ba468c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f451de00640 1 -- 172.21.15.90:0/1989793349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 msgr2=0x55ba468c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:54.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f451de00640 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 0x55ba468c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f451de00640 1 -- 172.21.15.90:0/1989793349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 msgr2=0x55ba4693ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T22:29:54.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f451de00640 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 0x55ba4693ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f451de00640 1 -- 172.21.15.90:0/1989793349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba4693f490 con 0x55ba468b3180 2024-09-10T22:29:54.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f451de00640 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3180 0x55ba468c5230 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55ba468d2290 tx=0x7f4510004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad344ef25587a874 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:54.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f44ffe00640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-09-10T22:29:54.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f44ffe00640 1 -- 172.21.15.90:0/1989793349 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451002a040 con 0x55ba468b3180 2024-09-10T22:29:54.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f44ffe00640 1 -- 172.21.15.90:0/1989793349 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f4510004b20 con 0x55ba468b3180 2024-09-10T22:29:54.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.287+0000 7f45278d8740 1 -- 172.21.15.90:0/1989793349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f450c0b5500 con 0x55ba468b3180 2024-09-10T22:29:54.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f44ffe00640 1 -- 172.21.15.90:0/1989793349 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) ==== 24+0+0 (secure 0 0 0) 0x7f4510039050 con 0x55ba468b3180 2024-09-10T22:29:54.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f45278d8740 10 client.16662 fetch_fsmap learned FSMap version 103 2024-09-10T22:29:54.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f45278d8740 1 -- 172.21.15.90:0/1989793349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=4+}) -- 0x55ba46946870 con 0x55ba468b3180 2024-09-10T22:29:54.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f44ffe00640 1 -- 172.21.15.90:0/1989793349 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f450c0b5500 con 0x55ba468b3180 2024-09-10T22:29:54.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f44ffe00640 1 -- 172.21.15.90:0/1989793349 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 103) ==== 8434+0+0 (secure 0 0 0) 0x7f4510004600 con 0x55ba468b3180 2024-09-10T22:29:54.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f45278d8740 10 client.16662 fetch_fsmap finished waiting for FSMap version 103 2024-09-10T22:29:54.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f45278d8740 10 client.16662 resolve_mds: validated gid 34703 aka daemon mds.b 2024-09-10T22:29:54.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f45278d8740 20 client.16662 populate_metadata read hostname 'smithi090' 2024-09-10T22:29:54.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f45278d8740 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ba46947150 0x55ba46949570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:54.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f45278d8740 4 client.16662 mds_command: new command op to 34703 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T22:29:54.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.288+0000 7f45278d8740 1 -- 172.21.15.90:0/1989793349 --> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55ba468266a0 con 0x55ba46947150 2024-09-10T22:29:54.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.289+0000 7f451d400640 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ba46947150 0x55ba46949570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:54.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.289+0000 7f451d400640 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ba46947150 0x55ba46949570 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9398cad3dde1ec server_cookie=76765360cae07ecc in_seq=0 out_seq=0 2024-09-10T22:29:54.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.289+0000 7f44ffe00640 10 client.16662 ms_handle_connect on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:54.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f44ffe00640 1 -- 172.21.15.90:0/1989793349 <== mds.0 v2:172.21.15.154:6838/703390433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+592 (crc 0 0 0) 0x55ba468266a0 con 0x55ba46947150 2024-09-10T22:29:54.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f44ffe00640 10 client.16662 handle_command_reply: tid=0 2024-09-10T22:29:54.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 client.16662 shutdown 2024-09-10T22:29:54.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 -- 172.21.15.90:0/1989793349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3180 msgr2=0x55ba468c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:54.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3180 0x55ba468c5230 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55ba468d2290 tx=0x7f4510004a40 comp rx=0 tx=0).stop 2024-09-10T22:29:54.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 -- 172.21.15.90:0/1989793349 shutdown_connections 2024-09-10T22:29:54.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.154:6838/703390433,v1:172.21.15.154:6839/703390433] conn(0x55ba46947150 0x55ba46949570 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x55ba468bfa30 0x55ba4693ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x55ba468b3aa0 0x55ba468c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 --2- 172.21.15.90:0/1989793349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ba468b3180 0x55ba468c5230 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 -- 172.21.15.90:0/1989793349 >> 172.21.15.90:0/1989793349 conn(0x55ba468b0fb0 msgr2=0x55ba468b60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:54.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f44ffe00640 0 client.16662 ms_handle_reset on v2:172.21.15.154:6838/703390433 2024-09-10T22:29:54.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 -- 172.21.15.90:0/1989793349 shutdown_connections 2024-09-10T22:29:54.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 1 -- 172.21.15.90:0/1989793349 wait complete. 2024-09-10T22:29:54.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.299+0000 7f45278d8740 20 client.16662 trim_cache size 0 max 16384 2024-09-10T22:29:54.305 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:29:54.305 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 103, 2024-09-10T22:29:54.306 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 34703, 2024-09-10T22:29:54.306 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 82, 2024-09-10T22:29:54.306 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-09-10T22:29:54.306 INFO:teuthology.orchestra.run.smithi090.stdout: "15153d4c": { 2024-09-10T22:29:54.306 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 82, 2024-09-10T22:29:54.306 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-09-10T22:29:54.307 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:29:54.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:29:54.307 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:29:54.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:54.307 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 90.0, 2024-09-10T22:29:54.307 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 150.0, 2024-09-10T22:29:54.307 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-09-10T22:29:54.308 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-09-10T22:29:54.308 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-09-10T22:29:54.308 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-09-10T22:29:54.308 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-09-10T22:29:54.308 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-09-10T22:29:54.308 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:54.308 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:54.309 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:54.309 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:54.309 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:54.309 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:29:54.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 -- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45040b54d0 msgr2=0x7f45040b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:54.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45040b54d0 0x7f45040b7990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55ba468b1e90 tx=0x7f451000a570 comp rx=0 tx=0).stop 2024-09-10T22:29:54.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 -- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4520104590 msgr2=0x7f452019b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:54.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4520104590 0x7f452019b8c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f450c00d950 tx=0x7f450c00de10 comp rx=0 tx=0).stop 2024-09-10T22:29:54.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 -- 172.21.15.90:0/904951803 shutdown_connections 2024-09-10T22:29:54.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f45040b54d0 0x7f45040b7990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f452010d140 0x7f45201a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4520104590 0x7f452019b8c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 --2- 172.21.15.90:0/904951803 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4520103b90 0x7f452019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:54.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 -- 172.21.15.90:0/904951803 >> 172.21.15.90:0/904951803 conn(0x7f45200fd700 msgr2=0x7f452010f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:54.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 -- 172.21.15.90:0/904951803 shutdown_connections 2024-09-10T22:29:54.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:54.300+0000 7f4524e00640 1 -- 172.21.15.90:0/904951803 wait complete. 2024-09-10T22:29:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:54 smithi154 ceph-mon[34495]: pgmap v1237: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T22:29:54.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:54 smithi171 ceph-mon[34138]: pgmap v1237: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T22:29:55.430 INFO:tasks.quiescer:done joining Quiescers 2024-09-10T22:29:55.430 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-10T22:29:55.440 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-10T22:29:55.440 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T22:29:55.441 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-09-10T22:29:55.441 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-10T22:29:55.517 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-10T22:29:55.582 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-10T22:29:56.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.130+0000 7f9fc6000640 1 -- 172.21.15.90:0/491543869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0102cf0 msgr2=0x7f9fc01030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.130+0000 7f9fc6000640 1 --2- 172.21.15.90:0/491543869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0102cf0 0x7f9fc01030d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009900 tx=0x7f9fb002a410 comp rx=0 tx=0).stop 2024-09-10T22:29:56.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.130+0000 7f9fc6000640 1 -- 172.21.15.90:0/491543869 shutdown_connections 2024-09-10T22:29:56.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.130+0000 7f9fc6000640 1 --2- 172.21.15.90:0/491543869 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fc0109070 0x7f9fc010f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.130+0000 7f9fc6000640 1 --2- 172.21.15.90:0/491543869 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc01086f0 0x7f9fc0108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.130+0000 7f9fc6000640 1 --2- 172.21.15.90:0/491543869 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0102cf0 0x7f9fc01030d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.130+0000 7f9fc6000640 1 -- 172.21.15.90:0/491543869 >> 172.21.15.90:0/491543869 conn(0x7f9fc00fe730 msgr2=0x7f9fc0100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:56.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.131+0000 7f9fc6000640 1 -- 172.21.15.90:0/491543869 shutdown_connections 2024-09-10T22:29:56.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.131+0000 7f9fc6000640 1 -- 172.21.15.90:0/491543869 wait complete. 2024-09-10T22:29:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.131+0000 7f9fc6000640 1 Processor -- start 2024-09-10T22:29:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fc6000640 1 -- start start 2024-09-10T22:29:56.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fc6000640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc0102cf0 0x7f9fc019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fc6000640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fc01086f0 0x7f9fc019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fc6000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0109070 0x7f9fc01a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fc6000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0112be0 con 0x7f9fc0109070 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fc6000640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0112a60 con 0x7f9fc01086f0 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fc6000640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0112d60 con 0x7f9fc0102cf0 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0109070 0x7f9fc01a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0109070 0x7f9fc01a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41176/0 (socket says 172.21.15.90:41176) 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fbfe00640 1 -- 172.21.15.90:0/2522559648 learned_addr learned my addr 172.21.15.90:0/2522559648 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fbf400640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc0102cf0 0x7f9fc019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fb7e00640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fc01086f0 0x7f9fc019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.132+0000 7f9fbfe00640 1 -- 172.21.15.90:0/2522559648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc0102cf0 msgr2=0x7f9fc019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fbfe00640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc0102cf0 0x7f9fc019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fbfe00640 1 -- 172.21.15.90:0/2522559648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fc01086f0 msgr2=0x7f9fc019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fbfe00640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fc01086f0 0x7f9fc019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fbfe00640 1 -- 172.21.15.90:0/2522559648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc01a14c0 con 0x7f9fc0109070 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fbfe00640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0109070 0x7f9fc01a0cf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800b970 tx=0x7f9fb800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fbcc00640 1 -- 172.21.15.90:0/2522559648 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8019070 con 0x7f9fc0109070 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fbcc00640 1 -- 172.21.15.90:0/2522559648 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9fb8014440 con 0x7f9fc0109070 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc01a17b0 con 0x7f9fc0109070 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fbcc00640 1 -- 172.21.15.90:0/2522559648 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80075d0 con 0x7f9fc0109070 2024-09-10T22:29:56.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.133+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc01a7e70 con 0x7f9fc0109070 2024-09-10T22:29:56.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.134+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc01030d0 con 0x7f9fc0109070 2024-09-10T22:29:56.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.136+0000 7f9fbcc00640 1 -- 172.21.15.90:0/2522559648 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9fb80145f0 con 0x7f9fc0109070 2024-09-10T22:29:56.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.139+0000 7f9fbcc00640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9fa40b5450 0x7f9fa40b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.139+0000 7f9fbcc00640 1 -- 172.21.15.90:0/2522559648 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f9fb80e5c30 con 0x7f9fc0109070 2024-09-10T22:29:56.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.139+0000 7f9fbf400640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9fa40b5450 0x7f9fa40b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.139+0000 7f9fbf400640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9fa40b5450 0x7f9fa40b7910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0000c00 tx=0x7f9fb0005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:56.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.143+0000 7f9fbcc00640 1 -- 172.21.15.90:0/2522559648 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9fb80ac380 con 0x7f9fc0109070 2024-09-10T22:29:56.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.345+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9fc0063590 con 0x7f9fa40b5450 2024-09-10T22:29:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:56 smithi090 ceph-mon[30350]: pgmap v1238: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-09-10T22:29:56.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.357+0000 7f9fbcc00640 1 -- 172.21.15.90:0/2522559648 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f9fc0063590 con 0x7f9fa40b5450 2024-09-10T22:29:56.359 INFO:teuthology.orchestra.run.smithi090.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-10T00:00:00", "created": "2024-09-10T22:03:48", "first": "2024-09-10T22:04:00", "last": "2024-09-10T22:29:00", "last_pruned": "2024-09-10T22:29:00", "created_count": 26, "pruned_count": 20, "active": true} 2024-09-10T22:29:56.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.359+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9fa40b5450 msgr2=0x7f9fa40b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.359+0000 7f9fc6000640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9fa40b5450 0x7f9fa40b7910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0000c00 tx=0x7f9fb0005a20 comp rx=0 tx=0).stop 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.359+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0109070 msgr2=0x7f9fc01a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.359+0000 7f9fc6000640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0109070 0x7f9fc01a0cf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9fb800b970 tx=0x7f9fb800be30 comp rx=0 tx=0).stop 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.360+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 shutdown_connections 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.360+0000 7f9fc6000640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9fa40b5450 0x7f9fa40b7910 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.360+0000 7f9fc6000640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9fc0109070 0x7f9fc01a0cf0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.360+0000 7f9fc6000640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9fc01086f0 0x7f9fc019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.360+0000 7f9fc6000640 1 --2- 172.21.15.90:0/2522559648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9fc0102cf0 0x7f9fc019b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.360+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 >> 172.21.15.90:0/2522559648 conn(0x7f9fc00fe730 msgr2=0x7f9fc0106d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.360+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 shutdown_connections 2024-09-10T22:29:56.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.360+0000 7f9fc6000640 1 -- 172.21.15.90:0/2522559648 wait complete. 2024-09-10T22:29:56.377 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-10T22:29:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:56 smithi154 ceph-mon[34495]: pgmap v1238: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-09-10T22:29:56.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:56 smithi171 ceph-mon[34138]: pgmap v1238: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 11 op/s 2024-09-10T22:29:56.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.529+0000 7f769c200640 1 -- 172.21.15.90:0/2080061569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 msgr2=0x7f769410f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.529+0000 7f769c200640 1 --2- 172.21.15.90:0/2080061569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 0x7f769410f870 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f768c009ca0 tx=0x7f768c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.529+0000 7f769c200640 1 -- 172.21.15.90:0/2080061569 shutdown_connections 2024-09-10T22:29:56.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.529+0000 7f769c200640 1 --2- 172.21.15.90:0/2080061569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 0x7f769410f870 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.529+0000 7f769c200640 1 --2- 172.21.15.90:0/2080061569 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76941086f0 0x7f7694108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.529+0000 7f769c200640 1 --2- 172.21.15.90:0/2080061569 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7694102cf0 0x7f76941030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.529+0000 7f769c200640 1 -- 172.21.15.90:0/2080061569 >> 172.21.15.90:0/2080061569 conn(0x7f76940fe730 msgr2=0x7f7694100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.529+0000 7f769c200640 1 -- 172.21.15.90:0/2080061569 shutdown_connections 2024-09-10T22:29:56.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.529+0000 7f769c200640 1 -- 172.21.15.90:0/2080061569 wait complete. 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.530+0000 7f769c200640 1 Processor -- start 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.530+0000 7f769c200640 1 -- start start 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769c200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7694102cf0 0x7f7694199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769c200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76941086f0 0x7f7694199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769c200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 0x7f769419ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769c200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7694112c30 con 0x7f7694109070 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769c200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f7694112ab0 con 0x7f7694102cf0 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769c200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f7694112db0 con 0x7f76941086f0 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 0x7f769419ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 0x7f769419ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41192/0 (socket says 172.21.15.90:41192) 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769a200640 1 -- 172.21.15.90:0/3785874898 learned_addr learned my addr 172.21.15.90:0/3785874898 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f7699800640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7694102cf0 0x7f7694199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f7698e00640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76941086f0 0x7f7694199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769a200640 1 -- 172.21.15.90:0/3785874898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76941086f0 msgr2=0x7f7694199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769a200640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76941086f0 0x7f7694199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769a200640 1 -- 172.21.15.90:0/3785874898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7694102cf0 msgr2=0x7f7694199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.531+0000 7f769a200640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7694102cf0 0x7f7694199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.532+0000 7f769a200640 1 -- 172.21.15.90:0/3785874898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769419f1d0 con 0x7f7694109070 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.532+0000 7f7699800640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7694102cf0 0x7f7694199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.532+0000 7f769a200640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 0x7f769419ea00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f768c005cc0 tx=0x7f768c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.532+0000 7f7692000640 1 -- 172.21.15.90:0/3785874898 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c032e00 con 0x7f7694109070 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.532+0000 7f7692000640 1 -- 172.21.15.90:0/3785874898 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f768c0044d0 con 0x7f7694109070 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.532+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769419f460 con 0x7f7694109070 2024-09-10T22:29:56.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.532+0000 7f7692000640 1 -- 172.21.15.90:0/3785874898 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768c030b70 con 0x7f7694109070 2024-09-10T22:29:56.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.532+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76941a5b80 con 0x7f7694109070 2024-09-10T22:29:56.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.533+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76941030d0 con 0x7f7694109070 2024-09-10T22:29:56.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.535+0000 7f7692000640 1 -- 172.21.15.90:0/3785874898 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f768c004680 con 0x7f7694109070 2024-09-10T22:29:56.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.538+0000 7f7692000640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f767c0b5450 0x7f767c0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.538+0000 7f7692000640 1 -- 172.21.15.90:0/3785874898 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f768c107790 con 0x7f7694109070 2024-09-10T22:29:56.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.538+0000 7f7699800640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f767c0b5450 0x7f767c0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.538+0000 7f7699800640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f767c0b5450 0x7f767c0b7910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7688005d00 tx=0x7f7688005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:56.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.541+0000 7f7692000640 1 -- 172.21.15.90:0/3785874898 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f768c0cdee0 con 0x7f7694109070 2024-09-10T22:29:56.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.748+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 --> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7694063590 con 0x7f767c0b5450 2024-09-10T22:29:56.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.758+0000 7f7692000640 1 -- 172.21.15.90:0/3785874898 <== mgr.14223 v2:172.21.15.90:6800/3093492804 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7694063590 con 0x7f767c0b5450 2024-09-10T22:29:56.760 INFO:teuthology.orchestra.run.smithi090.stdout:/ 1m 6m3h 2024-09-10T22:29:56.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f767c0b5450 msgr2=0x7f767c0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f767c0b5450 0x7f767c0b7910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7688005d00 tx=0x7f7688005c70 comp rx=0 tx=0).stop 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 msgr2=0x7f769419ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 0x7f769419ea00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f768c005cc0 tx=0x7f768c032a10 comp rx=0 tx=0).stop 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 shutdown_connections 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f767c0b5450 0x7f767c0b7910 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7694109070 0x7f769419ea00 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f76941086f0 0x7f7694199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 --2- 172.21.15.90:0/3785874898 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f7694102cf0 0x7f7694199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.760+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 >> 172.21.15.90:0/3785874898 conn(0x7f76940fe730 msgr2=0x7f7694100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.761+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 shutdown_connections 2024-09-10T22:29:56.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.761+0000 7f769c200640 1 -- 172.21.15.90:0/3785874898 wait complete. 2024-09-10T22:29:56.778 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T22:29:56.790 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T22:29:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.949+0000 7fb6fa400640 1 -- 172.21.15.90:0/994010035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f410b0a0 msgr2=0x7fb6f41118a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.949+0000 7fb6fa400640 1 --2- 172.21.15.90:0/994010035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f410b0a0 0x7fb6f41118a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009ca0 tx=0x7fb6ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.950+0000 7fb6fa400640 1 -- 172.21.15.90:0/994010035 shutdown_connections 2024-09-10T22:29:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.950+0000 7fb6fa400640 1 --2- 172.21.15.90:0/994010035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f410b0a0 0x7fb6f41118a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.950+0000 7fb6fa400640 1 --2- 172.21.15.90:0/994010035 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f410a720 0x7fb6f410ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.950+0000 7fb6fa400640 1 --2- 172.21.15.90:0/994010035 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6f4104d20 0x7fb6f4105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.950+0000 7fb6fa400640 1 -- 172.21.15.90:0/994010035 >> 172.21.15.90:0/994010035 conn(0x7fb6f4100720 msgr2=0x7fb6f4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.950+0000 7fb6fa400640 1 -- 172.21.15.90:0/994010035 shutdown_connections 2024-09-10T22:29:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.950+0000 7fb6fa400640 1 -- 172.21.15.90:0/994010035 wait complete. 2024-09-10T22:29:56.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.950+0000 7fb6fa400640 1 Processor -- start 2024-09-10T22:29:56.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6fa400640 1 -- start start 2024-09-10T22:29:56.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6fa400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f4104d20 0x7fb6f419b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6fa400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6f410a720 0x7fb6f419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6fa400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f410b0a0 0x7fb6f41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6fa400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f4114c60 con 0x7fb6f4104d20 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6fa400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f4114ae0 con 0x7fb6f410b0a0 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6fa400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f4114de0 con 0x7fb6f410a720 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6f9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f4104d20 0x7fb6f419b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6f9000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f4104d20 0x7fb6f419b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41202/0 (socket says 172.21.15.90:41202) 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6f9000640 1 -- 172.21.15.90:0/2588186024 learned_addr learned my addr 172.21.15.90:0/2588186024 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.951+0000 7fb6f3e00640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6f410a720 0x7fb6f419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f9a00640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f410b0a0 0x7fb6f41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f9000640 1 -- 172.21.15.90:0/2588186024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6f410a720 msgr2=0x7fb6f419b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f9000640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6f410a720 0x7fb6f419b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f9000640 1 -- 172.21.15.90:0/2588186024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f410b0a0 msgr2=0x7fb6f41a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f9000640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f410b0a0 0x7fb6f41a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:56.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f9000640 1 -- 172.21.15.90:0/2588186024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f41a1200 con 0x7fb6f4104d20 2024-09-10T22:29:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f3e00640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6f410a720 0x7fb6f419b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:29:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f9000640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f4104d20 0x7fb6f419b160 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9f0 tx=0x7fb6e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f1600640 1 -- 172.21.15.90:0/2588186024 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800cd80 con 0x7fb6f4104d20 2024-09-10T22:29:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f1600640 1 -- 172.21.15.90:0/2588186024 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb6e80054b0 con 0x7fb6f4104d20 2024-09-10T22:29:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f41a14f0 con 0x7fb6f4104d20 2024-09-10T22:29:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6f1600640 1 -- 172.21.15.90:0/2588186024 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8010430 con 0x7fb6f4104d20 2024-09-10T22:29:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.952+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f41a7bb0 con 0x7fb6f4104d20 2024-09-10T22:29:56.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.953+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f4105100 con 0x7fb6f4104d20 2024-09-10T22:29:56.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.956+0000 7fb6f1600640 1 -- 172.21.15.90:0/2588186024 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb6e8005020 con 0x7fb6f4104d20 2024-09-10T22:29:56.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.958+0000 7fb6f1600640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6dc0b5450 0x7fb6dc0b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:56.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.959+0000 7fb6f1600640 1 -- 172.21.15.90:0/2588186024 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7fb6e80e6950 con 0x7fb6f4104d20 2024-09-10T22:29:56.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.959+0000 7fb6f3e00640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6dc0b5450 0x7fb6dc0b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:56.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.959+0000 7fb6f3e00640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6dc0b5450 0x7fb6dc0b7910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0005e00 tx=0x7fb6e0005c70 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:56.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:56.962+0000 7fb6f1600640 1 -- 172.21.15.90:0/2588186024 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb6e80ad0a0 con 0x7fb6f4104d20 2024-09-10T22:29:57.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.150+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6f419c280 con 0x7fb6f4104d20 2024-09-10T22:29:57.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.151+0000 7fb6f1600640 1 -- 172.21.15.90:0/2588186024 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) ==== 74+0+25415 (secure 0 0 0) 0x7fb6e80ac9a0 con 0x7fb6f4104d20 2024-09-10T22:29:57.153 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:29:57.153 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":130,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:29:10.877712+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":47,"snap_epoch":129,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]},{"pool":4,"snaps":[{"begin":34,"length":1},{"begin":47,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:29:57.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6dc0b5450 msgr2=0x7fb6dc0b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6dc0b5450 0x7fb6dc0b7910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0005e00 tx=0x7fb6e0005c70 comp rx=0 tx=0).stop 2024-09-10T22:29:57.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f4104d20 msgr2=0x7fb6f419b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f4104d20 0x7fb6f419b160 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9f0 tx=0x7fb6e800edf0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 shutdown_connections 2024-09-10T22:29:57.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6dc0b5450 0x7fb6dc0b7910 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb6f410b0a0 0x7fb6f41a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb6f410a720 0x7fb6f419b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 --2- 172.21.15.90:0/2588186024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6f4104d20 0x7fb6f419b160 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.153+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 >> 172.21.15.90:0/2588186024 conn(0x7fb6f4100720 msgr2=0x7fb6f4102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:57.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.154+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 shutdown_connections 2024-09-10T22:29:57.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.154+0000 7fb6fa400640 1 -- 172.21.15.90:0/2588186024 wait complete. 2024-09-10T22:29:57.171 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:54:47.794295+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T22:00:28.928263+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T22:00:32.528935+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 129, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T22:00:41.695408+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 47, 'snap_epoch': 129, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T22:29:57.171 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T22:29:57.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.327+0000 7fd844e00640 1 -- 172.21.15.90:0/109083791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 msgr2=0x7fd84010ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.327+0000 7fd844e00640 1 --2- 172.21.15.90:0/109083791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 0x7fd84010ad20 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009900 tx=0x7fd82c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.328+0000 7fd844e00640 1 -- 172.21.15.90:0/109083791 shutdown_connections 2024-09-10T22:29:57.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.328+0000 7fd844e00640 1 --2- 172.21.15.90:0/109083791 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd84010b260 0x7fd840111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.328+0000 7fd844e00640 1 --2- 172.21.15.90:0/109083791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 0x7fd84010ad20 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.328+0000 7fd844e00640 1 --2- 172.21.15.90:0/109083791 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd840104ee0 0x7fd8401052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.328+0000 7fd844e00640 1 -- 172.21.15.90:0/109083791 >> 172.21.15.90:0/109083791 conn(0x7fd840100900 msgr2=0x7fd840102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:57.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.328+0000 7fd844e00640 1 -- 172.21.15.90:0/109083791 shutdown_connections 2024-09-10T22:29:57.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.328+0000 7fd844e00640 1 -- 172.21.15.90:0/109083791 wait complete. 2024-09-10T22:29:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.328+0000 7fd844e00640 1 Processor -- start 2024-09-10T22:29:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd844e00640 1 -- start start 2024-09-10T22:29:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd844e00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd840104ee0 0x7fd84019b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:57.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd844e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 0x7fd84019b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd844e00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84010b260 0x7fd8401a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd844e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd840114d90 con 0x7fd84010a8e0 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd844e00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fd840114c10 con 0x7fd84010b260 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd844e00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fd840114f10 con 0x7fd840104ee0 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd83d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 0x7fd84019b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd83d400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 0x7fd84019b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41216/0 (socket says 172.21.15.90:41216) 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd83d400640 1 -- 172.21.15.90:0/3293380983 learned_addr learned my addr 172.21.15.90:0/3293380983 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd83de00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd840104ee0 0x7fd84019b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.329+0000 7fd83e800640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84010b260 0x7fd8401a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd83d400640 1 -- 172.21.15.90:0/3293380983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd840104ee0 msgr2=0x7fd84019b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd83d400640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd840104ee0 0x7fd84019b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd83d400640 1 -- 172.21.15.90:0/3293380983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84010b260 msgr2=0x7fd8401a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd83d400640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84010b260 0x7fd8401a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd83d400640 1 -- 172.21.15.90:0/3293380983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8401a1330 con 0x7fd84010a8e0 2024-09-10T22:29:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd83d400640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 0x7fd84019b7d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009230 tx=0x7fd82c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd836a00640 1 -- 172.21.15.90:0/3293380983 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c03f030 con 0x7fd84010a8e0 2024-09-10T22:29:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8401a15c0 con 0x7fd84010a8e0 2024-09-10T22:29:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd836a00640 1 -- 172.21.15.90:0/3293380983 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd82c003a80 con 0x7fd84010a8e0 2024-09-10T22:29:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.330+0000 7fd836a00640 1 -- 172.21.15.90:0/3293380983 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c035070 con 0x7fd84010a8e0 2024-09-10T22:29:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.331+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8401a7ce0 con 0x7fd84010a8e0 2024-09-10T22:29:57.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.332+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8401052c0 con 0x7fd84010a8e0 2024-09-10T22:29:57.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.334+0000 7fd836a00640 1 -- 172.21.15.90:0/3293380983 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fd82c003c30 con 0x7fd84010a8e0 2024-09-10T22:29:57.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.335+0000 7fd836a00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8240b5450 0x7fd8240b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:57.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.335+0000 7fd836a00640 1 -- 172.21.15.90:0/3293380983 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7fd82c107020 con 0x7fd84010a8e0 2024-09-10T22:29:57.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.335+0000 7fd83de00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8240b5450 0x7fd8240b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:57.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.336+0000 7fd83de00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8240b5450 0x7fd8240b7910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd830007ec0 tx=0x7fd8300075b0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:57.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.339+0000 7fd836a00640 1 -- 172.21.15.90:0/3293380983 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd82c0cd180 con 0x7fd84010a8e0 2024-09-10T22:29:57.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.522+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd84002e5c0 con 0x7fd84010a8e0 2024-09-10T22:29:57.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.522+0000 7fd836a00640 1 -- 172.21.15.90:0/3293380983 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v130) ==== 93+0+10 (secure 0 0 0) 0x7fd82c0d1bb0 con 0x7fd84010a8e0 2024-09-10T22:29:57.524 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-09-10T22:29:57.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.524+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8240b5450 msgr2=0x7fd8240b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.524+0000 7fd844e00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8240b5450 0x7fd8240b7910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd830007ec0 tx=0x7fd8300075b0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.524+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 msgr2=0x7fd84019b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.524+0000 7fd844e00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 0x7fd84019b7d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009230 tx=0x7fd82c004270 comp rx=0 tx=0).stop 2024-09-10T22:29:57.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.525+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 shutdown_connections 2024-09-10T22:29:57.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.525+0000 7fd844e00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fd8240b5450 0x7fd8240b7910 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.525+0000 7fd844e00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fd84010b260 0x7fd8401a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.525+0000 7fd844e00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd84010a8e0 0x7fd84019b7d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.525+0000 7fd844e00640 1 --2- 172.21.15.90:0/3293380983 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fd840104ee0 0x7fd84019b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.525+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 >> 172.21.15.90:0/3293380983 conn(0x7fd840100900 msgr2=0x7fd840103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:57.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.525+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 shutdown_connections 2024-09-10T22:29:57.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.525+0000 7fd844e00640 1 -- 172.21.15.90:0/3293380983 wait complete. 2024-09-10T22:29:57.543 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T22:29:57.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.700+0000 7f902ac00640 1 -- 172.21.15.90:0/143311558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9024102cd0 msgr2=0x7f90241030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.701+0000 7f902ac00640 1 --2- 172.21.15.90:0/143311558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9024102cd0 0x7f90241030b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9014009900 tx=0x7f901402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.701+0000 7f902ac00640 1 -- 172.21.15.90:0/143311558 shutdown_connections 2024-09-10T22:29:57.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.701+0000 7f902ac00640 1 --2- 172.21.15.90:0/143311558 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9024109050 0x7f902410f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.701+0000 7f902ac00640 1 --2- 172.21.15.90:0/143311558 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f90241086d0 0x7f9024108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.701+0000 7f902ac00640 1 --2- 172.21.15.90:0/143311558 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9024102cd0 0x7f90241030b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.701+0000 7f902ac00640 1 -- 172.21.15.90:0/143311558 >> 172.21.15.90:0/143311558 conn(0x7f90240fe710 msgr2=0x7f9024100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:57.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.701+0000 7f902ac00640 1 -- 172.21.15.90:0/143311558 shutdown_connections 2024-09-10T22:29:57.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.701+0000 7f902ac00640 1 -- 172.21.15.90:0/143311558 wait complete. 2024-09-10T22:29:57.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f902ac00640 1 Processor -- start 2024-09-10T22:29:57.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f902ac00640 1 -- start start 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f902ac00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9024102cd0 0x7f902419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f902ac00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90241086d0 0x7f902419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f902ac00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9024109050 0x7f90241a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f902ac00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9024112c10 con 0x7f90241086d0 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f902ac00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9024112a90 con 0x7f9024102cd0 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f902ac00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9024112d90 con 0x7f9024109050 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f9023400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90241086d0 0x7f902419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f9023400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90241086d0 0x7f902419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41242/0 (socket says 172.21.15.90:41242) 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.702+0000 7f9023400640 1 -- 172.21.15.90:0/287747784 learned_addr learned my addr 172.21.15.90:0/287747784 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9023400640 1 -- 172.21.15.90:0/287747784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9024109050 msgr2=0x7f90241a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9023400640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9024109050 0x7f90241a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9023400640 1 -- 172.21.15.90:0/287747784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9024102cd0 msgr2=0x7f902419b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9023e00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9024102cd0 0x7f902419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:57.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9023400640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9024102cd0 0x7f902419b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9023400640 1 -- 172.21.15.90:0/287747784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90241a1440 con 0x7f90241086d0 2024-09-10T22:29:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9023e00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9024102cd0 0x7f902419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:29:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9023400640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90241086d0 0x7f902419b8e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f901000e9f0 tx=0x7f901000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9020c00640 1 -- 172.21.15.90:0/287747784 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901000cd80 con 0x7f90241086d0 2024-09-10T22:29:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9020c00640 1 -- 172.21.15.90:0/287747784 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f90100054b0 con 0x7f90241086d0 2024-09-10T22:29:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90241a1730 con 0x7f90241086d0 2024-09-10T22:29:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f9020c00640 1 -- 172.21.15.90:0/287747784 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9010010430 con 0x7f90241086d0 2024-09-10T22:29:57.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.703+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90241a7df0 con 0x7f90241086d0 2024-09-10T22:29:57.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.705+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90241030b0 con 0x7f90241086d0 2024-09-10T22:29:57.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.707+0000 7f9020c00640 1 -- 172.21.15.90:0/287747784 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9010005020 con 0x7f90241086d0 2024-09-10T22:29:57.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.708+0000 7f9020c00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90080b5450 0x7f90080b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:57.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.708+0000 7f9020c00640 1 -- 172.21.15.90:0/287747784 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f90100e6950 con 0x7f90241086d0 2024-09-10T22:29:57.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.708+0000 7f9023e00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90080b5450 0x7f90080b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:57.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.709+0000 7f9023e00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90080b5450 0x7f90080b7910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f901402a910 tx=0x7f9014005fd0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:57.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.712+0000 7f9020c00640 1 -- 172.21.15.90:0/287747784 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f90100ad0a0 con 0x7f90241086d0 2024-09-10T22:29:57.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.896+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f902402e5f0 con 0x7f90241086d0 2024-09-10T22:29:57.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.897+0000 7f9020c00640 1 -- 172.21.15.90:0/287747784 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v130) ==== 104+0+11 (secure 0 0 0) 0x7f90100ac9a0 con 0x7f90241086d0 2024-09-10T22:29:57.899 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-09-10T22:29:57.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90080b5450 msgr2=0x7f90080b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90080b5450 0x7f90080b7910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f901402a910 tx=0x7f9014005fd0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90241086d0 msgr2=0x7f902419b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:57.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90241086d0 0x7f902419b8e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f901000e9f0 tx=0x7f901000edf0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 shutdown_connections 2024-09-10T22:29:57.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f90080b5450 0x7f90080b7910 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9024109050 0x7f90241a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90241086d0 0x7f902419b8e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 --2- 172.21.15.90:0/287747784 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9024102cd0 0x7f902419b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:57.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.899+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 >> 172.21.15.90:0/287747784 conn(0x7f90240fe710 msgr2=0x7f9024104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:57.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.900+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 shutdown_connections 2024-09-10T22:29:57.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:57.900+0000 7f902ac00640 1 -- 172.21.15.90:0/287747784 wait complete. 2024-09-10T22:29:57.917 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T22:29:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.075+0000 7f6f40a00640 1 -- 172.21.15.90:0/767711263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c108eb0 msgr2=0x7f6f3c10f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.075+0000 7f6f40a00640 1 --2- 172.21.15.90:0/767711263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c10f6b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009900 tx=0x7f6f3402a3f0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.075+0000 7f6f40a00640 1 -- 172.21.15.90:0/767711263 shutdown_connections 2024-09-10T22:29:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.075+0000 7f6f40a00640 1 --2- 172.21.15.90:0/767711263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c10f6b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.075+0000 7f6f40a00640 1 --2- 172.21.15.90:0/767711263 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c108530 0x7f6f3c108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.075+0000 7f6f40a00640 1 --2- 172.21.15.90:0/767711263 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f3c102b30 0x7f6f3c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.075+0000 7f6f40a00640 1 -- 172.21.15.90:0/767711263 >> 172.21.15.90:0/767711263 conn(0x7f6f3c0fe550 msgr2=0x7f6f3c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.075+0000 7f6f40a00640 1 -- 172.21.15.90:0/767711263 shutdown_connections 2024-09-10T22:29:58.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.075+0000 7f6f40a00640 1 -- 172.21.15.90:0/767711263 wait complete. 2024-09-10T22:29:58.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.076+0000 7f6f40a00640 1 Processor -- start 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.076+0000 7f6f40a00640 1 -- start start 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.076+0000 7f6f40a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c102b30 0x7f6f3c19b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.076+0000 7f6f40a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c108530 0x7f6f3c19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.076+0000 7f6f40a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c1a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.076+0000 7f6f40a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1129e0 con 0x7f6f3c102b30 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.076+0000 7f6f40a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c112860 con 0x7f6f3c108eb0 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.076+0000 7f6f40a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c112b60 con 0x7f6f3c108530 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c102b30 0x7f6f3c19b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3b400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c102b30 0x7f6f3c19b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53818/0 (socket says 172.21.15.90:53818) 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3b400640 1 -- 172.21.15.90:0/4123624625 learned_addr learned my addr 172.21.15.90:0/4123624625 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f32a00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c108530 0x7f6f3c19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3b400640 1 -- 172.21.15.90:0/4123624625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c108530 msgr2=0x7f6f3c19b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3be00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c1a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3b400640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c108530 0x7f6f3c19b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3b400640 1 -- 172.21.15.90:0/4123624625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f3c108eb0 msgr2=0x7f6f3c1a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3b400640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c1a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3b400640 1 -- 172.21.15.90:0/4123624625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c1a1290 con 0x7f6f3c102b30 2024-09-10T22:29:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3be00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c1a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:29:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f3b400640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c102b30 0x7f6f3c19b1f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00eab0 tx=0x7f6f2c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.077+0000 7f6f38c00640 1 -- 172.21.15.90:0/4123624625 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00ce40 con 0x7f6f3c102b30 2024-09-10T22:29:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.078+0000 7f6f38c00640 1 -- 172.21.15.90:0/4123624625 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6f2c011900 con 0x7f6f3c102b30 2024-09-10T22:29:58.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.078+0000 7f6f38c00640 1 -- 172.21.15.90:0/4123624625 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c010990 con 0x7f6f3c102b30 2024-09-10T22:29:58.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.078+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c1a1580 con 0x7f6f3c102b30 2024-09-10T22:29:58.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.078+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c1a7c40 con 0x7f6f3c102b30 2024-09-10T22:29:58.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.079+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f3c102f10 con 0x7f6f3c102b30 2024-09-10T22:29:58.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.081+0000 7f6f38c00640 1 -- 172.21.15.90:0/4123624625 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f6f2c00f040 con 0x7f6f3c102b30 2024-09-10T22:29:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.082+0000 7f6f38c00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f200b5450 0x7f6f200b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.082+0000 7f6f38c00640 1 -- 172.21.15.90:0/4123624625 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f6f2c0e67f0 con 0x7f6f3c102b30 2024-09-10T22:29:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.082+0000 7f6f32a00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f200b5450 0x7f6f200b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.083+0000 7f6f32a00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f200b5450 0x7f6f200b7910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f24005fd0 tx=0x7f6f24005850 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:58.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.086+0000 7f6f38c00640 1 -- 172.21.15.90:0/4123624625 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6f2c016030 con 0x7f6f3c102b30 2024-09-10T22:29:58.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.269+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6f3c1a80a0 con 0x7f6f3c102b30 2024-09-10T22:29:58.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.270+0000 7f6f38c00640 1 -- 172.21.15.90:0/4123624625 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v130) ==== 100+0+11 (secure 0 0 0) 0x7f6f2c0acf40 con 0x7f6f3c102b30 2024-09-10T22:29:58.272 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f200b5450 msgr2=0x7f6f200b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f200b5450 0x7f6f200b7910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6f24005fd0 tx=0x7f6f24005850 comp rx=0 tx=0).stop 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c102b30 msgr2=0x7f6f3c19b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c102b30 0x7f6f3c19b1f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00eab0 tx=0x7f6f2c00ef70 comp rx=0 tx=0).stop 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 shutdown_connections 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f6f200b5450 0x7f6f200b7910 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f6f3c108eb0 0x7f6f3c1a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f6f3c108530 0x7f6f3c19b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 --2- 172.21.15.90:0/4123624625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f3c102b30 0x7f6f3c19b1f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 >> 172.21.15.90:0/4123624625 conn(0x7f6f3c0fe550 msgr2=0x7f6f3c075380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:58.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 shutdown_connections 2024-09-10T22:29:58.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.272+0000 7f6f40a00640 1 -- 172.21.15.90:0/4123624625 wait complete. 2024-09-10T22:29:58.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:58 smithi090 ceph-mon[30350]: pgmap v1239: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-09-10T22:29:58.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:58 smithi090 ceph-mon[30350]: from='client.16668 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:29:58.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:58 smithi090 ceph-mon[30350]: from='client.16674 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:29:58.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:58 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2588186024' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:29:58.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:58 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3293380983' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:29:58.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:58 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/287747784' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T22:29:58.290 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T22:29:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:58 smithi154 ceph-mon[34495]: pgmap v1239: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-09-10T22:29:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:58 smithi154 ceph-mon[34495]: from='client.16668 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:29:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:58 smithi154 ceph-mon[34495]: from='client.16674 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:29:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:58 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2588186024' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:29:58.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:58 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3293380983' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:29:58.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:58 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/287747784' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T22:29:58.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.446+0000 7f27cf400640 1 -- 172.21.15.90:0/2239520658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c8102cd0 msgr2=0x7f27c81030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.446+0000 7f27cf400640 1 --2- 172.21.15.90:0/2239520658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c8102cd0 0x7f27c81030b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009900 tx=0x7f27bc02a410 comp rx=0 tx=0).stop 2024-09-10T22:29:58.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.446+0000 7f27cf400640 1 -- 172.21.15.90:0/2239520658 shutdown_connections 2024-09-10T22:29:58.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.446+0000 7f27cf400640 1 --2- 172.21.15.90:0/2239520658 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8109050 0x7f27c810f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.446+0000 7f27cf400640 1 --2- 172.21.15.90:0/2239520658 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27c81086d0 0x7f27c8108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.446+0000 7f27cf400640 1 --2- 172.21.15.90:0/2239520658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c8102cd0 0x7f27c81030b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.446+0000 7f27cf400640 1 -- 172.21.15.90:0/2239520658 >> 172.21.15.90:0/2239520658 conn(0x7f27c80fe710 msgr2=0x7f27c8100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:58.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.446+0000 7f27cf400640 1 -- 172.21.15.90:0/2239520658 shutdown_connections 2024-09-10T22:29:58.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.446+0000 7f27cf400640 1 -- 172.21.15.90:0/2239520658 wait complete. 2024-09-10T22:29:58.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.447+0000 7f27cf400640 1 Processor -- start 2024-09-10T22:29:58.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.447+0000 7f27cf400640 1 -- start start 2024-09-10T22:29:58.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.447+0000 7f27cf400640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27c8102cd0 0x7f27c8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.447+0000 7f27cf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c81086d0 0x7f27c81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.447+0000 7f27cf400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8109050 0x7f27c819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.447+0000 7f27cf400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8112b80 con 0x7f27c81086d0 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.447+0000 7f27cf400640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8112a00 con 0x7f27c8109050 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.447+0000 7f27cf400640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f27c8112d00 con 0x7f27c8102cd0 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c81086d0 0x7f27c81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c81086d0 0x7f27c81996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53838/0 (socket says 172.21.15.90:53838) 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27c7e00640 1 -- 172.21.15.90:0/608293970 learned_addr learned my addr 172.21.15.90:0/608293970 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27cca00640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27c8102cd0 0x7f27c8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27cd400640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8109050 0x7f27c819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27c7e00640 1 -- 172.21.15.90:0/608293970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27c8102cd0 msgr2=0x7f27c8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27c7e00640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27c8102cd0 0x7f27c8199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27c7e00640 1 -- 172.21.15.90:0/608293970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8109050 msgr2=0x7f27c819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27c7e00640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8109050 0x7f27c819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27c7e00640 1 -- 172.21.15.90:0/608293970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c819f220 con 0x7f27c81086d0 2024-09-10T22:29:58.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.448+0000 7f27c7e00640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c81086d0 0x7f27c81996c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f27b400d950 tx=0x7f27b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:58.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.449+0000 7f27c5600640 1 -- 172.21.15.90:0/608293970 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4017030 con 0x7f27c81086d0 2024-09-10T22:29:58.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.449+0000 7f27c5600640 1 -- 172.21.15.90:0/608293970 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f27b4004020 con 0x7f27c81086d0 2024-09-10T22:29:58.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.449+0000 7f27c5600640 1 -- 172.21.15.90:0/608293970 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b40055d0 con 0x7f27c81086d0 2024-09-10T22:29:58.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.449+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c819f510 con 0x7f27c81086d0 2024-09-10T22:29:58.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.449+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c81a5db0 con 0x7f27c81086d0 2024-09-10T22:29:58.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.450+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c81030b0 con 0x7f27c81086d0 2024-09-10T22:29:58.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.452+0000 7f27c5600640 1 -- 172.21.15.90:0/608293970 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f27b4009a30 con 0x7f27c81086d0 2024-09-10T22:29:58.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.454+0000 7f27c5600640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b00b54d0 0x7f27b00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.454+0000 7f27c5600640 1 -- 172.21.15.90:0/608293970 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f27b40e63b0 con 0x7f27c81086d0 2024-09-10T22:29:58.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.454+0000 7f27cca00640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b00b54d0 0x7f27b00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.455+0000 7f27cca00640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b00b54d0 0x7f27b00b7990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0057c0 tx=0x7f27bc0058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:58.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.455+0000 7f27c5600640 1 -- 172.21.15.90:0/608293970 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f27b40acb80 con 0x7f27c81086d0 2024-09-10T22:29:58.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:58 smithi171 ceph-mon[34138]: pgmap v1239: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.4 KiB/s wr, 12 op/s 2024-09-10T22:29:58.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:58 smithi171 ceph-mon[34138]: from='client.16668 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:29:58.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:58 smithi171 ceph-mon[34138]: from='client.16674 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T22:29:58.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:58 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2588186024' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:29:58.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:58 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3293380983' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T22:29:58.499 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:58 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/287747784' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T22:29:58.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.639+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f27c802e5c0 con 0x7f27c81086d0 2024-09-10T22:29:58.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.639+0000 7f27c5600640 1 -- 172.21.15.90:0/608293970 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v130) ==== 103+0+11 (secure 0 0 0) 0x7f27b40580f0 con 0x7f27c81086d0 2024-09-10T22:29:58.641 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-09-10T22:29:58.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b00b54d0 msgr2=0x7f27b00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b00b54d0 0x7f27b00b7990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0057c0 tx=0x7f27bc0058d0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c81086d0 msgr2=0x7f27c81996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c81086d0 0x7f27c81996c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f27b400d950 tx=0x7f27b400de10 comp rx=0 tx=0).stop 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 shutdown_connections 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f27b00b54d0 0x7f27b00b7990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f27c8109050 0x7f27c819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f27c81086d0 0x7f27c81996c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 --2- 172.21.15.90:0/608293970 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f27c8102cd0 0x7f27c8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 >> 172.21.15.90:0/608293970 conn(0x7f27c80fe710 msgr2=0x7f27c8100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 shutdown_connections 2024-09-10T22:29:58.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.642+0000 7f27cf400640 1 -- 172.21.15.90:0/608293970 wait complete. 2024-09-10T22:29:58.659 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T22:29:58.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.813+0000 7f9b30200640 1 -- 172.21.15.90:0/774723014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 msgr2=0x7f9b28111a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.813+0000 7f9b30200640 1 --2- 172.21.15.90:0/774723014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 0x7f9b28111a80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009ca0 tx=0x7f9b2002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.813+0000 7f9b30200640 1 -- 172.21.15.90:0/774723014 shutdown_connections 2024-09-10T22:29:58.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.813+0000 7f9b30200640 1 --2- 172.21.15.90:0/774723014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 0x7f9b28111a80 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.813+0000 7f9b30200640 1 --2- 172.21.15.90:0/774723014 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b2810a900 0x7f9b2810ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.813+0000 7f9b30200640 1 --2- 172.21.15.90:0/774723014 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b28104f00 0x7f9b281052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.813+0000 7f9b30200640 1 -- 172.21.15.90:0/774723014 >> 172.21.15.90:0/774723014 conn(0x7f9b28100920 msgr2=0x7f9b28102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:29:58.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.813+0000 7f9b30200640 1 -- 172.21.15.90:0/774723014 shutdown_connections 2024-09-10T22:29:58.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.813+0000 7f9b30200640 1 -- 172.21.15.90:0/774723014 wait complete. 2024-09-10T22:29:58.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b30200640 1 Processor -- start 2024-09-10T22:29:58.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b30200640 1 -- start start 2024-09-10T22:29:58.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b30200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b28104f00 0x7f9b2819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b30200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b2810a900 0x7f9b2819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b30200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 0x7f9b281a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b30200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9b28114dc0 con 0x7f9b2810b280 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b30200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f9b28114c40 con 0x7f9b28104f00 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b30200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f9b28114f40 con 0x7f9b2810a900 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b2e200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 0x7f9b281a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b2e200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 0x7f9b281a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53856/0 (socket says 172.21.15.90:53856) 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.814+0000 7f9b2e200640 1 -- 172.21.15.90:0/3924205596 learned_addr learned my addr 172.21.15.90:0/3924205596 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b2d800640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b28104f00 0x7f9b2819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b2ce00640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b2810a900 0x7f9b2819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b2e200640 1 -- 172.21.15.90:0/3924205596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b2810a900 msgr2=0x7f9b2819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b2e200640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b2810a900 0x7f9b2819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b2e200640 1 -- 172.21.15.90:0/3924205596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b28104f00 msgr2=0x7f9b2819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:29:58.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b2e200640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b28104f00 0x7f9b2819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:29:58.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b2e200640 1 -- 172.21.15.90:0/3924205596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b281a1360 con 0x7f9b2810b280 2024-09-10T22:29:58.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b2e200640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 0x7f9b281a0b90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9b20005e30 tx=0x7f9b200093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:58.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b26000640 1 -- 172.21.15.90:0/3924205596 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20032d10 con 0x7f9b2810b280 2024-09-10T22:29:58.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b26000640 1 -- 172.21.15.90:0/3924205596 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b20032eb0 con 0x7f9b2810b280 2024-09-10T22:29:58.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b26000640 1 -- 172.21.15.90:0/3924205596 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20030b20 con 0x7f9b2810b280 2024-09-10T22:29:58.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.815+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b281a15f0 con 0x7f9b2810b280 2024-09-10T22:29:58.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.816+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b281a7d10 con 0x7f9b2810b280 2024-09-10T22:29:58.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.817+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b281052e0 con 0x7f9b2810b280 2024-09-10T22:29:58.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.819+0000 7f9b26000640 1 -- 172.21.15.90:0/3924205596 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f9b20030ce0 con 0x7f9b2810b280 2024-09-10T22:29:58.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.820+0000 7f9b26000640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b100b54d0 0x7f9b100b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:29:58.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.820+0000 7f9b26000640 1 -- 172.21.15.90:0/3924205596 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9490+0+0 (secure 0 0 0) 0x7f9b20107550 con 0x7f9b2810b280 2024-09-10T22:29:58.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.820+0000 7f9b2d800640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b100b54d0 0x7f9b100b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:29:58.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.821+0000 7f9b2d800640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b100b54d0 0x7f9b100b7990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00bbe0 tx=0x7f9b1c00b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:29:58.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:58.824+0000 7f9b26000640 1 -- 172.21.15.90:0/3924205596 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9b200cdd00 con 0x7f9b2810b280 2024-09-10T22:29:59.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.058+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f9b2810cad0 con 0x7f9b2810b280 2024-09-10T22:29:59.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.068+0000 7f9b26000640 1 -- 172.21.15.90:0/3924205596 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+828473 (secure 0 0 0) 0x7f9b200cd600 con 0x7f9b2810b280 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 24, 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "active_gid": 14223, 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "active_name": "x", 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addrs": { 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6800", 2024-09-10T22:29:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3093492804 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6801", 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 3093492804 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addr": "172.21.15.90:6801/3093492804", 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "active_change": "2024-09-10T21:52:33.477512+0000", 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "active_mgr_features": 4540719139924082687, 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "available": true, 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: "standbys": [ 2024-09-10T22:29:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 24166, 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "z", 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540719139924082687, 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.083 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.084 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.085 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.086 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.087 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.088 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.089 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.090 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.091 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.092 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.093 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.094 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.095 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-09-10T22:29:59.096 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.097 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-09-10T22:29:59.098 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T22:29:59.099 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T22:29:59.100 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T22:29:59.101 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T22:29:59.102 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T22:29:59.103 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-09-10T22:29:59.104 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T22:29:59.105 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T22:29:59.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.107 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.108 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.109 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.110 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.111 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.112 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.113 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-09-10T22:29:59.114 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.115 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.116 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.117 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.118 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-09-10T22:29:59.119 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.120 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.121 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T22:29:59.122 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.123 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.124 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.125 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.126 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.127 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.128 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.129 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.130 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T22:29:59.131 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T22:29:59.132 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.133 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.134 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.135 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-09-10T22:29:59.136 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-09-10T22:29:59.137 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-09-10T22:29:59.138 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.139 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.140 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nginx": { 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nginx", 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nginx container image", 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.141 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.142 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.143 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_oauth2_proxy": { 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "oauth2-proxy container image", 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.144 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.145 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.146 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.147 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba_metrics": { 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba_metrics", 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T22:29:59.148 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-09-10T22:29:59.149 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T22:29:59.150 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T22:29:59.151 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.152 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.153 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.154 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.155 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.156 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.157 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.158 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.159 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.160 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.161 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.162 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.163 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.164 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.165 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.166 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.167 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T22:29:59.168 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-09-10T22:29:59.169 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.170 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.171 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.172 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-09-10T22:29:59.173 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.174 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.175 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.176 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.177 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.178 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.179 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.180 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.181 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.182 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.183 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.184 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.185 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.186 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.187 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.188 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.189 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.190 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.191 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.192 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.193 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.194 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-09-10T22:29:59.195 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.196 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.197 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.198 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.199 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.200 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.201 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.202 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.203 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.204 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.205 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.206 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.207 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.208 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.209 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.210 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.211 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.212 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.213 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.214 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.215 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.216 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.217 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.219 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.220 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.221 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.222 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.223 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.224 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.225 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.226 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.227 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.228 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.229 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.230 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T22:29:59.231 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.232 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.233 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.234 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.235 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.236 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.237 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.238 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T22:29:59.239 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T22:29:59.240 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.241 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.242 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.243 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.244 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.245 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-09-10T22:29:59.246 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.247 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.248 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.249 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.250 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.251 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.252 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.253 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.254 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.255 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.256 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.257 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.258 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-09-10T22:29:59.259 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.260 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-09-10T22:29:59.261 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.262 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.263 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.264 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.265 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.266 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.267 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.268 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.269 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-09-10T22:29:59.270 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.271 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.272 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.273 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.274 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.275 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.276 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.277 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.278 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.279 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.280 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.281 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.282 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.283 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.284 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-09-10T22:29:59.285 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.286 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.287 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.288 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.289 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.290 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.291 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.292 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.293 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.294 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.295 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.296 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.297 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.299 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.300 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.301 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.302 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-09-10T22:29:59.303 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.304 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.305 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.306 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.307 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.308 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.309 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.310 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.311 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.312 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.313 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.315 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.316 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.317 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-09-10T22:29:59.318 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.319 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T22:29:59.320 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.321 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.322 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.323 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.324 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.325 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.328 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.329 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.330 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.331 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.332 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.333 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.334 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.335 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.336 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.337 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.338 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-09-10T22:29:59.339 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.340 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.341 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.342 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.344 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.345 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.346 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.347 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.348 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.349 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.350 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.351 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.352 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.353 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-09-10T22:29:59.354 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.355 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.356 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.357 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.358 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.359 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.360 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.361 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.362 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.363 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.364 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.365 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.366 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.367 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.368 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.369 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.370 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.371 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.373 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.374 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.375 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.376 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.377 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.378 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.379 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.380 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.381 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.382 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.383 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.384 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.385 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.386 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.387 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.388 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.389 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-09-10T22:29:59.390 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.391 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.392 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.394 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.395 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.396 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.397 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.398 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.399 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.400 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.401 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.402 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.403 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.404 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.405 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.406 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.407 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.408 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.409 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.410 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-09-10T22:29:59.411 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-09-10T22:29:59.412 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-09-10T22:29:59.413 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-09-10T22:29:59.414 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.415 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.416 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.417 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.418 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.419 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.420 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.421 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.422 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.423 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.424 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-09-10T22:29:59.425 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T22:29:59.426 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.427 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-09-10T22:29:59.428 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.429 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.430 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.431 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.432 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.433 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.434 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.435 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.436 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "max_requests": { 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_requests", 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.437 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.438 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.439 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.440 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.441 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.442 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.444 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.445 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.446 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T22:29:59.447 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.448 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.449 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.450 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.451 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.452 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.453 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.454 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.455 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.456 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.458 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.459 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.460 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.461 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.462 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-09-10T22:29:59.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-09-10T22:29:59.464 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.465 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.466 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.467 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.468 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.469 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.470 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.471 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.472 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.473 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.475 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smb", 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "internal_store_backend": { 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "internal_store_backend", 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.476 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.477 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.478 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.479 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.480 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.481 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.482 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "update_orchestration": { 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "update_orchestration", 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T22:29:59.483 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-09-10T22:29:59.484 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-09-10T22:29:59.485 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.486 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.487 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.488 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.489 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.490 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.491 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.492 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.493 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.494 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.495 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.497 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.498 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.500 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.501 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.502 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.503 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.504 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.505 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.506 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.507 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.508 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.509 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.510 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.511 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.512 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.513 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.514 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.515 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.516 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.517 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.518 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T22:29:59.519 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.520 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.521 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.522 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.524 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.525 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.526 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.528 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.529 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.530 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.531 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.532 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.533 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.534 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.535 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.536 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.537 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.538 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.539 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.540 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.541 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.542 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-09-10T22:29:59.543 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.544 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.545 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.546 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.548 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.549 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.550 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.551 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.552 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.553 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.554 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.555 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.556 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.557 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:29:59.558 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.559 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.560 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.561 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.562 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.563 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.564 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.566 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.567 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.568 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 24196, 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "y", 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540719139924082687, 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.569 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.570 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.571 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.572 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.575 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.576 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.577 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.578 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.579 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.580 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.581 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-09-10T22:29:59.582 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-09-10T22:29:59.583 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.584 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.585 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.586 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-09-10T22:29:59.587 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.588 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.589 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.590 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-09-10T22:29:59.591 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-09-10T22:29:59.592 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-09-10T22:29:59.593 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.594 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.595 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.596 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.597 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.598 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.599 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.600 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-09-10T22:29:59.601 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.602 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:29:59.603 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.604 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.605 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.606 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-09-10T22:29:59.607 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.608 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.609 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.610 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T22:29:59.611 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T22:29:59.612 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.613 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.614 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.615 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.616 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.618 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.619 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.620 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-09-10T22:29:59.621 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.622 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.623 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-09-10T22:29:59.624 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.625 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.626 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.627 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.628 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nginx": { 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nginx", 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nginx container image", 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.629 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.630 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.631 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_oauth2_proxy": { 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "oauth2-proxy container image", 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.632 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.633 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.634 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.635 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba_metrics": { 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba_metrics", 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.636 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.637 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.638 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.639 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T22:29:59.640 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T22:29:59.641 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-09-10T22:29:59.642 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T22:29:59.643 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.644 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.645 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.646 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.647 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.648 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.649 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.650 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.651 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.653 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.654 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.656 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.657 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.658 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.659 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.660 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-09-10T22:29:59.661 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.662 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.663 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.664 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.665 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.666 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.667 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T22:29:59.668 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.670 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T22:29:59.671 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T22:29:59.672 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.673 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.674 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.676 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.677 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.678 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.679 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.680 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.681 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.682 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.683 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.684 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.685 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.686 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.687 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.688 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.689 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.690 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.691 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.692 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T22:29:59.693 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T22:29:59.694 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T22:29:59.695 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.696 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.697 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.698 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.699 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.700 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.701 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.702 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.703 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.704 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.705 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.706 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.707 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.708 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.709 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.710 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.711 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.712 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.713 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.714 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.715 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.716 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.717 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.718 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.719 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.720 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.721 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.722 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.723 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.724 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.725 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.726 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.727 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.728 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.729 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.730 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.731 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.732 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.733 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.734 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-09-10T22:29:59.735 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-09-10T22:29:59.736 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-09-10T22:29:59.737 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.738 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.739 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.740 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.741 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.742 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-09-10T22:29:59.743 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T22:29:59.744 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-09-10T22:29:59.745 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-09-10T22:29:59.746 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.747 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.748 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-09-10T22:29:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-09-10T22:29:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-09-10T22:29:59.750 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-09-10T22:29:59.752 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.753 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.754 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.755 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.756 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.757 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.758 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.759 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.760 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.761 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.762 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.763 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.764 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.765 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.766 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.767 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.768 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-09-10T22:29:59.769 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.770 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.771 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.772 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.773 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.774 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.776 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.777 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.778 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.779 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-09-10T22:29:59.780 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.781 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T22:29:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-09-10T22:29:59.783 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.784 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.785 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.786 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.787 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.788 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.789 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.790 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.791 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.792 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-09-10T22:29:59.793 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.794 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.795 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:29:59.796 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.797 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.798 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.799 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.800 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.801 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.802 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.803 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.804 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.805 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.806 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.807 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.808 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.809 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.810 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.811 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.812 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.813 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.814 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.815 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.816 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.817 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.818 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.819 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.820 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.821 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.822 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.823 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.824 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.825 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.826 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.827 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.828 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-09-10T22:29:59.829 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.830 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.831 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.832 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.833 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-09-10T22:29:59.834 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.835 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.836 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.837 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.838 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.839 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.840 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.841 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.842 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.843 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.844 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.845 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.846 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.847 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.848 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.849 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.850 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.851 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.852 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.853 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-09-10T22:29:59.854 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.855 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.856 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.857 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.858 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.859 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.861 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.862 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.863 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.864 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.865 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.866 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.867 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.868 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.869 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.870 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.871 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.872 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.873 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.874 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.875 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.876 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.877 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.878 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.879 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.880 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.881 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.882 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.883 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T22:29:59.884 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.885 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.886 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.887 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.888 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.889 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.890 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.891 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-09-10T22:29:59.892 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:29:59.893 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.894 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.895 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.896 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.897 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.898 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.899 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.900 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.901 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.904 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.905 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.906 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.907 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.908 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.909 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.910 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.911 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.912 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.913 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.914 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.915 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.916 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.917 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.918 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.919 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.920 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.921 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.922 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.923 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.924 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.925 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.926 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.927 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.928 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.929 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.930 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.931 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "max_requests": { 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_requests", 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.932 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.933 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.934 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.935 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-09-10T22:29:59.936 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.937 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.938 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.939 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.940 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.941 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.942 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.943 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.944 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.945 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.946 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.947 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.948 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.949 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.950 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.951 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.952 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.953 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.954 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.955 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.956 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.957 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.958 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.959 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.960 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-09-10T22:29:59.961 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.962 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-09-10T22:29:59.963 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-09-10T22:29:59.964 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-09-10T22:29:59.965 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.966 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.967 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.968 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.969 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smb", 2024-09-10T22:29:59.970 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "internal_store_backend": { 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "internal_store_backend", 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T22:29:59.971 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.972 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.973 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.974 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.975 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.976 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.977 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "update_orchestration": { 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "update_orchestration", 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:29:59.978 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.979 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.980 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.981 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.982 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.983 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.984 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.985 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.986 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.987 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.988 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.989 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.990 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.991 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.992 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:29:59.993 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.994 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.995 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:29:59.996 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:29:59.997 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:29:59.998 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:29:59.999 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.000 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.001 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.002 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.003 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.004 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.005 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.006 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.007 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.008 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.009 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.010 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.011 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.012 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.013 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.014 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.015 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-09-10T22:30:00.016 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-09-10T22:30:00.017 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.018 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.019 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.020 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.021 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.022 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.023 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.024 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.025 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.026 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.027 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.028 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.029 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.030 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.031 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.032 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.033 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.034 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.035 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.036 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.037 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.038 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-09-10T22:30:00.039 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.040 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.041 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.042 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.043 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.044 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.045 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.046 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.047 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.048 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.049 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.050 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.051 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:30:00.052 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.053 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.054 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.055 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.056 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.057 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.058 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.059 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.060 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.061 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.062 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "modules": [ 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard", 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "iostat", 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "nfs", 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "restful", 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: "snap_schedule" 2024-09-10T22:30:00.063 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.064 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.065 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.066 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.067 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.068 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.069 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.070 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-09-10T22:30:00.071 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.072 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.073 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.074 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.075 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.076 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.077 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-09-10T22:30:00.079 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T22:30:00.080 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T22:30:00.081 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T22:30:00.082 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.083 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.084 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.085 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.086 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T22:30:00.087 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.088 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.089 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.090 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.091 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.092 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.093 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.094 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.095 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.096 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.097 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.098 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:30:00.099 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.100 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T22:30:00.101 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T22:30:00.102 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T22:30:00.103 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T22:30:00.104 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T22:30:00.105 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T22:30:00.106 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.107 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.108 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.109 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.110 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.111 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.112 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.113 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.114 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.115 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.116 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.117 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.118 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.119 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.120 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nginx": { 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nginx", 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.121 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nginx container image", 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.122 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.123 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_oauth2_proxy": { 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.124 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "oauth2-proxy container image", 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.125 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T22:30:00.126 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T22:30:00.127 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba_metrics": { 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba_metrics", 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.128 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.129 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.130 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.131 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.132 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.133 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.134 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.135 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.136 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.137 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.138 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.139 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.140 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.141 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.142 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.143 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.144 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.145 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.146 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.147 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.148 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-09-10T22:30:00.149 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-09-10T22:30:00.150 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-09-10T22:30:00.151 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-09-10T22:30:00.152 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.153 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.154 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.155 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.156 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.157 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.158 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.159 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.161 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.162 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-09-10T22:30:00.163 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.164 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.165 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.166 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.167 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.168 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.169 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.170 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.171 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.172 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.173 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.174 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.175 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.176 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-09-10T22:30:00.177 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.178 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.179 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.180 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.181 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.182 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.183 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.184 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.185 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T22:30:00.186 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T22:30:00.187 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T22:30:00.188 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T22:30:00.189 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.190 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.191 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.192 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.193 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.194 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.195 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.196 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.197 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.198 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.199 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.200 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.201 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.202 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.203 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.204 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.205 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.206 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.207 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.208 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.209 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.210 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.211 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.212 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.213 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.214 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.215 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.216 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.217 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.218 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.219 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.220 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.221 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.222 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.223 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.224 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.225 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.226 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-09-10T22:30:00.227 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.228 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.229 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.230 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.231 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.232 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.233 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.234 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.235 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-09-10T22:30:00.236 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-09-10T22:30:00.237 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.238 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.239 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.240 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.241 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.242 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.243 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-09-10T22:30:00.244 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.245 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.246 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.247 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.248 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.249 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.250 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.251 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T22:30:00.252 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-09-10T22:30:00.253 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.254 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.255 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.256 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.257 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.258 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.259 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.260 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.261 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.262 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.263 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.264 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.265 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.266 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.267 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.268 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.269 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.270 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:30:00.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.272 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.273 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.274 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.275 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.276 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-09-10T22:30:00.277 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-09-10T22:30:00.278 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.279 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-09-10T22:30:00.280 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-09-10T22:30:00.281 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-09-10T22:30:00.282 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-09-10T22:30:00.283 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.284 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.285 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.286 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.287 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.288 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.289 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.290 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.291 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.292 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.293 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.295 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.297 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.299 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.300 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.301 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.302 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.303 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.304 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.305 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.306 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.307 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.308 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.309 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.310 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.311 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.312 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.313 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.314 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.315 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.316 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.317 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-09-10T22:30:00.318 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.319 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.320 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.321 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.322 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.323 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.324 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.325 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.326 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.327 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.328 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.329 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.330 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.331 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.332 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.333 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.334 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.335 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.336 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.337 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.338 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.339 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.340 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.341 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.342 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.343 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.344 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.345 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.346 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.347 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.348 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.349 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.350 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.351 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.352 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.353 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.354 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.355 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.356 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.357 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.358 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.359 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.360 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.361 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.362 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.363 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.364 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:30:00.365 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.366 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-09-10T22:30:00.367 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.368 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.369 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.370 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.371 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.372 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.373 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.374 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.375 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.376 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.378 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.379 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.380 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.381 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.382 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.383 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.384 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.385 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.386 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.387 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.388 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.389 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.390 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.391 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.392 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.393 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.394 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.395 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.396 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.397 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.398 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.399 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.400 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.401 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.402 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.403 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.404 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-09-10T22:30:00.405 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-09-10T22:30:00.406 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.407 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.408 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.409 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.410 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.411 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.412 INFO:teuthology.orchestra.run.smithi090.stdout: "max_requests": { 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_requests", 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.413 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.414 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.415 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.416 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.417 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.419 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.420 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.421 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.422 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.423 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.424 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.425 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.426 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.427 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.428 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.429 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.430 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.431 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.432 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.433 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.434 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.435 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.437 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.438 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-09-10T22:30:00.439 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-09-10T22:30:00.440 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-09-10T22:30:00.441 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-09-10T22:30:00.442 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-09-10T22:30:00.443 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-09-10T22:30:00.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-09-10T22:30:00.445 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.446 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.447 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.448 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.449 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.450 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smb", 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "internal_store_backend": { 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "internal_store_backend", 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.451 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.452 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.453 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.454 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.455 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.456 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "update_orchestration": { 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "update_orchestration", 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.458 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.459 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.460 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.461 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.462 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.463 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.464 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.466 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.467 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.468 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.469 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.471 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.472 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.473 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.474 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.475 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.476 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.477 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.478 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.479 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.480 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.481 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.482 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.483 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.484 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.486 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.487 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.488 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.489 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.490 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.491 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.492 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.493 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.494 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.495 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.497 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:30:00.498 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-09-10T22:30:00.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.500 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-09-10T22:30:00.501 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.502 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.503 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.505 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.506 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-09-10T22:30:00.507 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-09-10T22:30:00.508 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.509 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-09-10T22:30:00.510 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.511 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.512 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.514 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.515 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.517 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.518 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.519 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.520 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.521 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.522 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.524 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.525 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.526 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.527 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.528 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.529 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-09-10T22:30:00.531 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.532 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.533 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.534 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-09-10T22:30:00.535 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.536 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "sqlite3_killpoint": { 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sqlite3_killpoint", 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.537 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "dev", 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.538 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-09-10T22:30:00.539 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-09-10T22:30:00.540 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.541 INFO:teuthology.orchestra.run.smithi090.stdout: "services": { 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard": "https://172.21.15.90:8443/" 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "always_on_modules": { 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "octopus": [ 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-09-10T22:30:00.542 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "pacific": [ 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "quincy": [ 2024-09-10T22:30:00.543 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "reef": [ 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-09-10T22:30:00.544 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "squid": [ 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-09-10T22:30:00.545 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "force_disabled_modules": {}, 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "last_failure_osd_epoch": 5, 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "active_clients": [ 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.546 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 639385352 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "libcephsqlite", 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 4213121460 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.547 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 333946189 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.548 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2733819922 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2502884450 2024-09-10T22:30:00.549 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2933192978 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:00.550 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-09-10T22:30:00.551 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:30:00.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b100b54d0 msgr2=0x7f9b100b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:00.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b100b54d0 0x7f9b100b7990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00bbe0 tx=0x7f9b1c00b390 comp rx=0 tx=0).stop 2024-09-10T22:30:00.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 msgr2=0x7f9b281a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:00.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 0x7f9b281a0b90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9b20005e30 tx=0x7f9b200093d0 comp rx=0 tx=0).stop 2024-09-10T22:30:00.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 shutdown_connections 2024-09-10T22:30:00.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f9b100b54d0 0x7f9b100b7990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:00.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b2810b280 0x7f9b281a0b90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:00.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f9b2810a900 0x7f9b2819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:00.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 --2- 172.21.15.90:0/3924205596 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f9b28104f00 0x7f9b2819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:00.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 >> 172.21.15.90:0/3924205596 conn(0x7f9b28100920 msgr2=0x7f9b28102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:00.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 shutdown_connections 2024-09-10T22:30:00.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:29:59.076+0000 7f9b30200640 1 -- 172.21.15.90:0/3924205596 wait complete. 2024-09-10T22:30:00.560 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T22:30:00.560 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T22:30:00.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:59 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4123624625' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T22:30:00.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:29:59 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/608293970' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-10T22:30:00.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:00 smithi090 ceph-mon[30350]: pgmap v1240: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2024-09-10T22:30:00.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:00 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3924205596' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T22:30:00.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:00 smithi090 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T22:30:00.562 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:59 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4123624625' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T22:30:00.562 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:29:59 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/608293970' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-10T22:30:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:00 smithi171 ceph-mon[34138]: pgmap v1240: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2024-09-10T22:30:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:00 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3924205596' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T22:30:00.563 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:00 smithi171 ceph-mon[34138]: overall HEALTH_OK 2024-09-10T22:30:00.563 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:59 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4123624625' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T22:30:00.563 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:29:59 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/608293970' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-09-10T22:30:00.563 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:00 smithi154 ceph-mon[34495]: pgmap v1240: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 10 op/s 2024-09-10T22:30:00.563 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:00 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3924205596' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T22:30:00.563 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:00 smithi154 ceph-mon[34495]: overall HEALTH_OK 2024-09-10T22:30:00.938 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:01.884 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 3452, 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 3454, 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 67490138, 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 1097737, 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 8, 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 4, 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.563273492, 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.200814428, 2024-09-10T22:30:01.885 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.211531067, 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.112203265, 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3452, 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.196732436, 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000056990 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.886 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 704, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 1063, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 63096582, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 1030488, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 10, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.254147178, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.071625469, 2024-09-10T22:30:01.887 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.146825228, 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.024803863, 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1061, 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.086903086, 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000081906 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T22:30:01.888 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 807184 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 2060, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 1978, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 39583020, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 764144, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 6, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 4, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.358436747, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.140088019, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.163779343, 2024-09-10T22:30:01.889 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.051000828, 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 1978, 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.119441129, 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000060384 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1470672, 2024-09-10T22:30:01.890 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 301136 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 9147, 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.264625315, 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000028930 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:01.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 0, 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 0, 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-09-10T22:30:01.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 0, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 0, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 6808, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 0, 2024-09-10T22:30:01.893 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 6808, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 6808, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 1, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-09-10T22:30:01.894 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:01.895 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 0, 2024-09-10T22:30:01.896 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 0 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 0, 2024-09-10T22:30:01.897 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-09-10T22:30:01.898 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "uninline_started": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "uninline_write_failed": 0 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 34464, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-09-10T22:30:01.899 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 0, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 53, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 4, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 171324831, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 171318694, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 34464 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 1472, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 7647, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 6175, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 60, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 136, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 76, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 6808, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 8152, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 1344, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 0, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 0, 2024-09-10T22:30:01.901 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 0, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 61216, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 265506816 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 0, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 0, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 0, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 0, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 0, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.903 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-09-10T22:30:01.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.905 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.906 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:01.906 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.906 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.906 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.906 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.906 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-09-10T22:30:01.906 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.907 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.908 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:01.909 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.910 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.911 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-09-10T22:30:01.912 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.913 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 1, 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 2, 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 1, 2024-09-10T22:30:01.914 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 1, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 0, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 0, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 2915, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:01.915 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:01.916 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:01.917 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 240999, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 82, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:01.918 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 9648, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 196, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 8170544, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 35985, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-09-10T22:30:01.919 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 5610, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 21, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 3, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 5610, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 5610, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 6.159904952, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001098022 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.920 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 5610, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 5611 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 5610, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 5610, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 0, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 3321, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 0, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 2287, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 0, 2024-09-10T22:30:01.921 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 1, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-09-10T22:30:01.922 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-09-10T22:30:01.923 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 130, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 44, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 11, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 14, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 3, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-09-10T22:30:01.924 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 0, 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 2, 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 0 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 0, 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T22:30:01.925 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 0, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 0, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 0, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 6212, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 169577295, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 6212, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-10T22:30:01.926 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 6212, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 169577295, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-10T22:30:01.927 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 5610, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 167133488, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 5610, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 167133488, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:01.928 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 5610, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 5610, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 5610, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 5610, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.929 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-09-10T22:30:01.930 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x55eeadd25040": { 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:01.931 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: "trackedop": { 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:01.932 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:30:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:02 smithi090 ceph-mon[30350]: pgmap v1241: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 KiB/s wr, 16 op/s 2024-09-10T22:30:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:02 smithi154 ceph-mon[34495]: pgmap v1241: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 KiB/s wr, 16 op/s 2024-09-10T22:30:02.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:02 smithi171 ceph-mon[34138]: pgmap v1241: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 KiB/s wr, 16 op/s 2024-09-10T22:30:03.631 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-10T22:30:03.631 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-10T22:30:03.631 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-10T22:30:03.632 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-10T22:30:03.632 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-10T22:30:03.632 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-10T22:30:03.632 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T22:30:03.632 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T22:30:03.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T22:30:04.012 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:04 smithi090 ceph-mon[30350]: pgmap v1242: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-09-10T22:30:04.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:04 smithi154 ceph-mon[34495]: pgmap v1242: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-09-10T22:30:04.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:04 smithi171 ceph-mon[34138]: pgmap v1242: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.3 KiB/s wr, 12 op/s 2024-09-10T22:30:04.838 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:30:04.838 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:04.838 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 0, 2024-09-10T22:30:04.838 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 2, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 0, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 0, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 0, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.002928719, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.000330894, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:04.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 7, 2024-09-10T22:30:04.840 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 370, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 3424, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 306496, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 5, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.049220775, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.038289442, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.005724419, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 369, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.034119003, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000092463 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.841 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 306496 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 484, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 479, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 907152, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 296928, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 2, 2024-09-10T22:30:04.842 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 1, 2024-09-10T22:30:04.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.100558016, 2024-09-10T22:30:04.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.060121649, 2024-09-10T22:30:04.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.048810387, 2024-09-10T22:30:04.843 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 478, 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.034797811, 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000072798 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 907152, 2024-09-10T22:30:04.844 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 296928 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 0, 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:04.845 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:04.846 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:04.847 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 147920, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 27, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:04.848 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 968, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 2, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-10T22:30:04.849 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 487, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 838725, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 487, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 487, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 838725, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:04.850 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:30:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:05 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:30:05.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:05 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:30:05.497 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:05 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T22:30:05.768 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T22:30:05.768 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T22:30:05.768 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-10T22:30:05.768 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-10T22:30:05.768 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-10T22:30:05.768 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-10T22:30:05.769 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-10T22:30:05.769 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T22:30:05.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T22:30:06.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:06 smithi171 ceph-mon[34138]: pgmap v1243: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-10T22:30:06.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:06 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:06 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:06 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.136 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:06 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.140 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:06.144 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:06 smithi154 ceph-mon[34495]: pgmap v1243: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-10T22:30:06.145 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:06 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.145 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:06 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.145 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:06 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.145 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:06 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:06 smithi090 ceph-mon[30350]: pgmap v1243: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 53 KiB/s wr, 30 op/s 2024-09-10T22:30:06.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:06 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:06 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:06 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.336 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:06 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:06.858 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 2742, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 2662, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 39651835, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 982488, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 7, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 3, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.483236536, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.228435838, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.177121006, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.062474642, 2024-09-10T22:30:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 2661, 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.173477051, 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000065192 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 1468272, 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 301968 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 2672, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 3030, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 55218779, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 1565603, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 7, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 6, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.405545853, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.150730179, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.163165090, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.069893546, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 3029, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.167844798, 2024-09-10T22:30:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000055412 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 717440 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 799, 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 801, 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 71507359, 2024-09-10T22:30:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 254094, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 8, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 5, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.218235754, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.039515170, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.147797651, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.023801241, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 799, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.056405063, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000070594 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 1, 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-MDSRank": { 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 9375, 2024-09-10T22:30:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.256810513, 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000027393 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "queue_len": 0, 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "complete_latency": { 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "request": 0, 2024-09-10T22:30:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "reply": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "reply_latency": { 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_reply": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "forward": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_complete": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_commit": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_split": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_merge": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes": 8184, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_top": 0, 2024-09-10T22:30:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_bottom": 8184, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pin_tail": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_pinned": 8184, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_expired": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "inodes_with_caps": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "caps": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "subtrees": 1, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_hit": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_forward": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_discover": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_dir_fetch": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "traverse_lock": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "load_cent": 0, 2024-09-10T22:30:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "q": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "exported": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "exported_inodes": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "imported": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "imported_inodes": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rfiles": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rbytes": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "root_rsnaps": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_inode_file_caps": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_grant": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_caps_dirty": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_cap_release": 0, 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "process_request_cap_release": 0 2024-09-10T22:30:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_cache": { 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_update_receipt": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_try_discover": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_send_discover": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "dir_handle_discover": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_created": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_enqueued": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "strays_migrated": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_started": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "recovery_completed": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_path": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_quiesce_inode": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_exportdir": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_flush": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragmentdir": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "uninline_started": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "uninline_write_failed": 0 2024-09-10T22:30:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_log": { 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "evlrg": 0, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "evadd": 0, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "evex": 0, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "evtrm": 0, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "ev": 78194, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "evexg": 0, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "evexd": 0, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "segadd": 0, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "segex": 0, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "segtrm": 0, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "seg": 101, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "segmjr": 8, 2024-09-10T22:30:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "segexg": 0, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "segexd": 0, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "expos": 4194304, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "wrpos": 167523949, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "rdpos": 167523350, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "jlat": { 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "replayed": 78194 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_mem": { 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "ino": 806, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "ino+": 11880, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "ino-": 11074, 2024-09-10T22:30:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "dir": 34, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "dir+": 196, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "dir-": 162, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "dn": 8184, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "dn+": 12476, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "dn-": 4292, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "cap": 0, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "cap+": 0, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "cap-": 0, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "rss": 69336, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "heap": 274591744 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_server": { 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_client_request": 0, 2024-09-10T22:30:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "dispatch_server_request": 0, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_request": 0, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_client_session": 0, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "handle_peer_request": 0, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "req_create_latency": { 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getattr_latency": { 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "req_link_latency": { 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookup_latency": { 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupino_latency": { 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupname_latency": { 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "req_lssnap_latency": { 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mkdir_latency": { 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mknod_latency": { 2024-09-10T22:30:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "req_mksnap_latency": { 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "req_open_latency": { 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "req_readdir_latency": { 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rename_latency": { 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmdir_latency": { 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setattr_latency": { 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setlayout_latency": { 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "req_setxattr_latency": { 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "req_symlink_latency": { 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "req_unlink_latency": { 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_sessions": { 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "session_count": 1, 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "session_add": 2, 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "session_remove": 1, 2024-09-10T22:30:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_open": 1, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "sessions_stale": 0, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "total_load": 0, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "average_load": 0, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "avg_session_uptime": 2921, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 240255, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 81, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 9648, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 196, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 7550857, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 37380, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "objecter": { 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "op_active": 0, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "op_laggy": 0, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send": 5606, 2024-09-10T22:30:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "op_send_bytes": 21, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_resend": 3, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_reply": 5606, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_latency": { 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 5606, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 6.879894028, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.001227237 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_inflight": 0, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "oplen_avg": { 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 5606, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 5607 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op": 5606, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_r": 5606, 2024-09-10T22:30:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "op_w": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "op_rmw": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "op_pg": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_stat": 3315, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_create": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_read": 2289, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_write": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writefull": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_writesame": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_append": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_zero": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_truncate": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_delete": 0, 2024-09-10T22:30:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_mapext": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_getxattr": 1, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_setxattr": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_call": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_watch": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_notify": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "osdop_other": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_active": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_send": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_resend": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "linger_ping": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_active": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_send": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "poolop_resend": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_active": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_send": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "poolstat_resend": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_active": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_send": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "statfs_resend": 0, 2024-09-10T22:30:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "command_active": 0, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "command_send": 0, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "command_resend": 0, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "map_epoch": 130, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "map_full": 0, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "map_inc": 44, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_sessions": 11, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_open": 13, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_session_close": 2, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_laggy": 0, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_wr": 0, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_rd": 2, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_del": 0 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "oft": { 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_objs": 0, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_updates": 0, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "omap_total_removes": 0 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "purge_queue": { 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing": 0, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executing_high_water": 0, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed_ops": 0, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_executed": 0, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 6209, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 165785737, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 6209, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 6209, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 165785737, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 5606, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 163342139, 2024-09-10T22:30:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 5606, 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 163342139, 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 1024, 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-10T22:30:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 5606, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 5606, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 5606, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 5606, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-write_buf_throttle-0x55686c621040": { 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 3758096384, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "trackedop": { 2024-09-10T22:30:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "slow_ops_count": 0 2024-09-10T22:30:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:06.904 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:30:07.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:07 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-09-10T22:30:07.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:07 smithi171 ceph-mon[34138]: Unable to set osd_memory_target on smithi090 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-10T22:30:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:07 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-09-10T22:30:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:07 smithi090 ceph-mon[30350]: Unable to set osd_memory_target on smithi090 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-10T22:30:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:07 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi090 to 176.4M 2024-09-10T22:30:07.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:07 smithi154 ceph-mon[34495]: Unable to set osd_memory_target on smithi090 to 185013964: error parsing value: Value '185013964' is below minimum 939524096 2024-09-10T22:30:08.005 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-10T22:30:08.005 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-10T22:30:08.005 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-10T22:30:08.005 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-10T22:30:08.005 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-09-10T22:30:08.005 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-10T22:30:08.005 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T22:30:08.005 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T22:30:08.005 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T22:30:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:08 smithi090 ceph-mon[30350]: pgmap v1244: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-09-10T22:30:08.360 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:08 smithi154 ceph-mon[34495]: pgmap v1244: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-09-10T22:30:08.376 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:08.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:08 smithi171 ceph-mon[34138]: pgmap v1244: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 45 KiB/s wr, 31 op/s 2024-09-10T22:30:08.954 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 487, 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 482, 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 907968, 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 298848, 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 4, 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 0, 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.097406836, 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.055236168, 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.046587500, 2024-09-10T22:30:08.955 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 481, 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.037161464, 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000077258 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 907968, 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 298848 2024-09-10T22:30:08.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 7, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 372, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 3424, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 306848, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 5, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 2, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.050731643, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.040077555, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.005687934, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 371, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.034934168, 2024-09-10T22:30:08.957 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000094162 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 306848 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_messages": 0, 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages": 2, 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_bytes": 0, 2024-09-10T22:30:08.958 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_bytes": 0, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_created_connections": 2, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_active_connections": 1, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_total_time": 0.000563048, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_recv_time": 0.000424034, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:08.959 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "cct": { 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "total_workers": 0, 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "unhealthy_workers": 0 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "mempool": { 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:08.960 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:08.961 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:08.962 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_items": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_bytes": 147920, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_anon_items": 27, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_bytes": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_items": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:08.963 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_bytes": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_items": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "pgmap_items": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_bytes": 968, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "mds_co_items": 2, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_1_items": 0, 2024-09-10T22:30:08.964 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "unittest_2_items": 0 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "val": 0, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "max": 104857600, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "get_started": 0, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "get": 490, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "get_sum": 839115, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "get_or_fail_success": 490, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "take": 0, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "take_sum": 0, 2024-09-10T22:30:08.965 INFO:teuthology.orchestra.run.smithi090.stdout: "put": 490, 2024-09-10T22:30:08.966 INFO:teuthology.orchestra.run.smithi090.stdout: "put_sum": 839115, 2024-09-10T22:30:08.966 INFO:teuthology.orchestra.run.smithi090.stdout: "wait": { 2024-09-10T22:30:08.966 INFO:teuthology.orchestra.run.smithi090.stdout: "avgcount": 0, 2024-09-10T22:30:08.966 INFO:teuthology.orchestra.run.smithi090.stdout: "sum": 0.000000000, 2024-09-10T22:30:08.966 INFO:teuthology.orchestra.run.smithi090.stdout: "avgtime": 0.000000000 2024-09-10T22:30:08.966 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:08.966 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-09-10T22:30:08.966 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-09-10T22:30:09.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:09 smithi090 ceph-mon[30350]: pgmap v1245: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-10T22:30:09.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:09 smithi154 ceph-mon[34495]: pgmap v1245: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-10T22:30:09.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:09 smithi171 ceph-mon[34138]: pgmap v1245: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-09-10T22:30:10.071 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T22:30:10.072 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T22:30:10.072 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-10T22:30:10.072 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-10T22:30:10.072 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-10T22:30:10.072 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-10T22:30:10.072 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-10T22:30:10.072 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T22:30:10.072 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T22:30:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:10 smithi090 ceph-mon[30350]: osdmap e131: 12 total, 12 up, 12 in 2024-09-10T22:30:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:10 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:10 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:10 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:10 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:10 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:10 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:10 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi171 to 176.4M 2024-09-10T22:30:10.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:10 smithi090 ceph-mon[30350]: Unable to set osd_memory_target on smithi171 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:30:10.408 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:10 smithi154 ceph-mon[34495]: osdmap e131: 12 total, 12 up, 12 in 2024-09-10T22:30:10.408 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:10 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:10.409 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:10 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:10.409 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:10 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.409 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:10 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.409 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:10 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.409 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:10 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.409 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:10 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi171 to 176.4M 2024-09-10T22:30:10.409 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:10 smithi154 ceph-mon[34495]: Unable to set osd_memory_target on smithi171 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:30:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:10 smithi171 ceph-mon[34138]: osdmap e131: 12 total, 12 up, 12 in 2024-09-10T22:30:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:10 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:10 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:10 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:10 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:10 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:10 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:10 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi171 to 176.4M 2024-09-10T22:30:10.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:10 smithi171 ceph-mon[34138]: Unable to set osd_memory_target on smithi171 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:30:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:11 smithi090 ceph-mon[30350]: pgmap v1247: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-10T22:30:11.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:11 smithi154 ceph-mon[34495]: pgmap v1247: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-10T22:30:11.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:11 smithi171 ceph-mon[34138]: pgmap v1247: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-10T22:30:11.801 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:30:12.655 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-10T22:30:12.655 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:12.655 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 721222, 2024-09-10T22:30:12.655 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 752248, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 212604532, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 573605426, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 103, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 8, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 77.102587317, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 40.356912575, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 43.571852391, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 1.634812016, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 752245, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 39.430969202, 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000052417 2024-09-10T22:30:12.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1419209, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.321847450, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000226 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 56304, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 29856 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 154788, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 177574, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 44911507, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 204255408, 2024-09-10T22:30:12.657 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 10, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 6, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 17.904990812, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 8.780272057, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 10.517672921, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.493252125, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 177575, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 13.114430292, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000073852 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 299649, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.080999932, 2024-09-10T22:30:12.658 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000270 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1669840, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 320000 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 197848, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 247292, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 59168773, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 159608147, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 15, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 8, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 32.313518091, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 19.882087280, 2024-09-10T22:30:12.659 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 14.486662565, 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 1.214914687, 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 247288, 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 23.078549614, 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000093326 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 417618, 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.090556606, 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000216 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 851568, 2024-09-10T22:30:12.660 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 1001184 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 49467, 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 28.915631986, 2024-09-10T22:30:12.661 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000584543 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 6936, 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.584013054, 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000084200 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.662 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 563566, 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 526641, 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 526641, 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 824.799011865, 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001566150 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 36925, 2024-09-10T22:30:12.663 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 15, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 24, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 1, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 3, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 2493, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 406, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 1937, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 150, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 1913, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 3404, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 3, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 3, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 6, 2024-09-10T22:30:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 1084423, 2024-09-10T22:30:12.665 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 1058915, 2024-09-10T22:30:12.665 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 13428, 2024-09-10T22:30:12.665 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 338, 2024-09-10T22:30:12.665 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-09-10T22:30:12.665 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:12.665 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 108, 2024-09-10T22:30:12.665 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 118, 2024-09-10T22:30:12.665 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-09-10T22:30:12.665 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 86, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 2619, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 142, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 3675, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 4, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 234, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 6, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:12.666 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 284, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 30729, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 8250, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 34868, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 3012, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 498, 2024-09-10T22:30:12.667 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 9493 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 1396, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 576, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 373, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 568, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 8181, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 33, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 2472, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 2439, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:12.668 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 20, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 1988, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 92, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 4, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:12.669 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_started": 0, 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_write_failed": 0 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 1, 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 39995, 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 39995, 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-09-10T22:30:12.670 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 98, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 98, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 5, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 389329250, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 0, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 25433, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 104.214387566, 2024-09-10T22:30:12.671 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004097604 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 0 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 2104, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 7373, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 5269, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 128, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 595, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 467, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 2493, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 9856, 2024-09-10T22:30:12.672 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 7363, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 3, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 3361, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 3358, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 76636, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 294506496 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 622381, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 17666, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 563566, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 394, 2024-09-10T22:30:12.673 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 45414, 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2979, 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 119.966904364, 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.040270864 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 480668, 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 191.111847108, 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000397596 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:12.674 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 29978, 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 337.416454430, 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.011255469 2024-09-10T22:30:12.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.676 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-09-10T22:30:12.677 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.677 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.677 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.677 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:12.677 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.677 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 27, 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.007027209, 2024-09-10T22:30:12.678 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000260267 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 119, 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.505429340, 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004247305 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 27, 2024-09-10T22:30:12.679 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 13.199019104, 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.488852559 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1511, 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1.947476784, 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001288866 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 8267, 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 24.228267599, 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002930720 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-09-10T22:30:12.680 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 483, 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 93.916963545, 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.194445059 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 44, 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 26.461655682, 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.601401265 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 21, 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 5.516127312, 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.262672729 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2, 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.001225937, 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000612968 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 68, 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.028968411, 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000426006 2024-09-10T22:30:12.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 8, 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.023436096, 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002929512 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 2439, 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 10.468208944, 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004292008 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 2, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 4, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 2, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 2, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 21411, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 10705, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 1575, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-09-10T22:30:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 576103, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 176, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-09-10T22:30:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 9648, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 196, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 6991579, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 28524, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-09-10T22:30:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 33644, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 385839081, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 31, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 33644, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 33644, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 178.845396279, 2024-09-10T22:30:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005315818 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 33644, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 34628 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 33644, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 775, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 32869, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 0, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 11, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 760, 2024-09-10T22:30:12.692 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 26193, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 529, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 1, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 5487, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 0, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 22, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:12.693 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-09-10T22:30:12.694 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 131, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 52, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 12, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 12, 2024-09-10T22:30:12.695 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 1126, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 30, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 469 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 1, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T22:30:12.696 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 5, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 4878, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 2439, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 1073854, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 219905575, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 1073854, 2024-09-10T22:30:12.697 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1073854, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 219905575, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-09-10T22:30:12.698 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 33644, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 386109676, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 33644, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 386109676, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-09-10T22:30:12.699 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 33644, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 33644, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 33644, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 33644, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.700 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 2439, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 270835, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 2439, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 760, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 270835, 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:12.701 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x55b6b2b06dc0": { 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 136, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 39995, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 385134946, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 39995, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:12.702 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 25433, 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 385134810, 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: "trackedop": { 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 1832 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:12.703 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-10T22:30:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:12.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:12.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:12.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:12.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:12.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:12.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: Adjusting osd_memory_target on smithi154 to 176.4M 2024-09-10T22:30:12.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: Unable to set osd_memory_target on smithi154 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:30:12.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:30:12.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:30:12.919 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:12 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:12.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:12.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:12.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:12.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:12.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:12.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:12.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: Adjusting osd_memory_target on smithi154 to 176.4M 2024-09-10T22:30:12.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: Unable to set osd_memory_target on smithi154 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:30:12.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:30:12.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:30:12.999 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:12 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: Adjusting osd_memory_target on smithi154 to 176.4M 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: Unable to set osd_memory_target on smithi154 to 185039052: error parsing value: Value '185039052' is below minimum 939524096 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T22:30:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:12 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' 2024-09-10T22:30:13.667 INFO:tasks.check_counter:Daemon mds.b mds.exported=86 2024-09-10T22:30:13.667 INFO:tasks.check_counter:Daemon mds.b mds.imported=142 2024-09-10T22:30:13.668 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1396 2024-09-10T22:30:13.668 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=576 2024-09-10T22:30:13.668 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-10T22:30:13.668 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=27 2024-09-10T22:30:13.668 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=21 2024-09-10T22:30:13.668 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T22:30:13.668 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T22:30:13.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:13 smithi154 ceph-mon[34495]: pgmap v1248: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-10T22:30:13.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:13 smithi171 ceph-mon[34138]: pgmap v1248: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-10T22:30:14.064 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:30:14.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:13 smithi090 ceph-mon[30350]: pgmap v1248: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 31 op/s 2024-09-10T22:30:14.727 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 337403, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 365975, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 102175266, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 284393390, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 15, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 6, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 41.618500234, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 22.233262687, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 22.145114850, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 1.203646290, 2024-09-10T22:30:14.728 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 365973, 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 30.361779932, 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000082961 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 660089, 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.185017456, 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000280 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 1440 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.729 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 71397, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 72830, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 23199062, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 90907968, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 14, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 9, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 11.695417213, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 7.291396342, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 5.024991618, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.321391273, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 72829, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 7.953241806, 2024-09-10T22:30:14.730 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000109204 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 122012, 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.031486703, 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000258 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1490800, 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 313888 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 203819, 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 171593, 2024-09-10T22:30:14.731 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 62040883, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 86389480, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 13, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 6, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 25.016480707, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 14.212108150, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 13.620398213, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.269754038, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 171591, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 17.420049701, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000101520 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.732 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 355139, 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.080290697, 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000226 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 791376 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-09-10T22:30:14.733 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 18199, 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 21.073600967, 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001157953 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 11466, 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1.461365876, 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000127452 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:14.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 247739, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 245539, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 245539, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 583.147546602, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002374969 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 2200, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 18, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 47, 2024-09-10T22:30:14.735 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 4, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 1757, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 215, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 1394, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 148, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 1448, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 1853, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 6, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 481587, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 461118, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 2171, 2024-09-10T22:30:14.736 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 459, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 2, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 252, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 34, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 1287, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 34, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 1191, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 694, 2024-09-10T22:30:14.737 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 214554224, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 6, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 46, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 12718, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 14440, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:14.738 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 21794, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 6828, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 512, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 18269 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 376, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 6316, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 3270, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 6514, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 602, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 21, 2024-09-10T22:30:14.739 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 5044, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 5023, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 20, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 2712, 2024-09-10T22:30:14.740 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 35, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 4, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_started": 0, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_write_failed": 0 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 34464, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-09-10T22:30:14.741 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 34464, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 53, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 53, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 4, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 171324831, 2024-09-10T22:30:14.742 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 0, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 9225, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 50.656640606, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005491234 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 0 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 1448, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 7995, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 6547, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 80, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 3449, 2024-09-10T22:30:14.743 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 3369, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 1757, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 11585, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 9828, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 5301, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 5301, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 71220, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 279826432 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 270836, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 180, 2024-09-10T22:30:14.744 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 247739, 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 547, 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 1301, 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 4933, 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 277.087728674, 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.056170226 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 210439, 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 108.457451287, 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000515386 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:14.745 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 8241, 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 16.306276242, 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001978676 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.746 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.748 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.749 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 3456, 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 3.539660504, 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001024207 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 12191, 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 49.270980497, 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004041586 2024-09-10T22:30:14.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1075, 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 92.855916857, 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.086377597 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.751 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 21, 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.157269129, 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.007489006 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.752 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 4, 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.018375934, 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.004593983 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.753 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 156, 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.068357791, 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000438190 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 5023, 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 35.385529687, 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.007044700 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:14.755 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 1, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 31436, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 31436, 2024-09-10T22:30:14.756 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 1587, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:14.757 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:14.758 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 481284, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 158, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:14.759 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 9888, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 202, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-09-10T22:30:14.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 4786277, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 20954, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 22324, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 168579376, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 30, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 22324, 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-09-10T22:30:14.761 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 22324, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 156.904995745, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.007028534 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 22324, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 23403 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 22324, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 940, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 21384, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-09-10T22:30:14.762 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 0, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 10, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 922, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 10147, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 528, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 4, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 10097, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 0, 2024-09-10T22:30:14.763 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 20, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-09-10T22:30:14.764 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-09-10T22:30:14.765 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 131, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 45, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 12, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 15, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 3, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 1140, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 36, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 499 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 1, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:14.766 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 5, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 10046, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 5023, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:14.767 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 612615, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 132244215, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 612615, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 612615, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 132244215, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.768 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-09-10T22:30:14.769 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 22324, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 169135443, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 22324, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 169135443, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:14.770 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 22324, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 22324, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 22324, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 22324, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.771 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 5023, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 556355, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 5023, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 922, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 556355, 2024-09-10T22:30:14.772 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x55fac1ad2dc0": { 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 34464, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 167130527, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 34464, 2024-09-10T22:30:14.773 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 9225, 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 167130527, 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "trackedop": { 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 2491 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:14.774 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-10T22:30:15.701 INFO:tasks.check_counter:Daemon mds.e mds.exported=34 2024-09-10T22:30:15.701 INFO:tasks.check_counter:Daemon mds.e mds.imported=34 2024-09-10T22:30:15.701 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=376 2024-09-10T22:30:15.702 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=6316 2024-09-10T22:30:15.702 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-10T22:30:15.702 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-10T22:30:15.702 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T22:30:15.702 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T22:30:15.702 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T22:30:15.918 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:15 smithi154 ceph-mon[34495]: pgmap v1249: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2024-09-10T22:30:15.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:15 smithi171 ceph-mon[34138]: pgmap v1249: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2024-09-10T22:30:16.076 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:30:16.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:15 smithi090 ceph-mon[30350]: pgmap v1249: 129 pgs: 2 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 119 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2024-09-10T22:30:16.812 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-10T22:30:16.812 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 202049, 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 181314, 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 65213548, 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 107360954, 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 12, 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 28.656378270, 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 17.726708785, 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 13.764001272, 2024-09-10T22:30:16.813 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.339620836, 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 181312, 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 16.006882350, 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000088283 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 348927, 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.077680438, 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000222 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1655360, 2024-09-10T22:30:16.814 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 314320 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 295214, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 327554, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 88548866, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 313145329, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 20, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 8, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 34.233969787, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 17.900042955, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 19.323993286, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.385710371, 2024-09-10T22:30:16.815 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 327551, 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 21.887365534, 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000066821 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 579841, 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.154964572, 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000267 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 4352, 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 803520 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:16.816 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 150095, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 146906, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 45968329, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 131671958, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 10, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 20.001036150, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 10.868635309, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 9.674262114, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 1.474201701, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 146905, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 15.947948231, 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000108559 2024-09-10T22:30:16.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 259269, 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.059759619, 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000230 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.818 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 34743, 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 26.686220921, 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000768103 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 17409, 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 1.592427478, 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000091471 2024-09-10T22:30:16.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 252106, 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 249642, 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 249642, 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 673.195629999, 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002696644 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 2464, 2024-09-10T22:30:16.820 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 18, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 83, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 8, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 1758, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 237, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 1415, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 106, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 1417, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 1783, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 2, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 2, 2024-09-10T22:30:16.821 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 6, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 489312, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 453085, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 2178, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 412, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 0, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 258, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 54, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 1197, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 26, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 588, 2024-09-10T22:30:16.822 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 694, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 214554224, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 6, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:16.823 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 4, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 15742, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 21479, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 30606, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 10742, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 857, 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 25656 2024-09-10T22:30:16.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 2896, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 4744, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 2453, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 5024, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 3347, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 20, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 7090, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 7070, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-09-10T22:30:16.825 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 20, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 3185, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 66, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 8, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_started": 0, 2024-09-10T22:30:16.826 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_write_failed": 0 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 64101, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 64101, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 93, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-09-10T22:30:16.827 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 93, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 7, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 232973108, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 0, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 17234, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 64.994541176, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.003771297 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.828 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 0 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 1467, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 10005, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 8538, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 90, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 2717, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 2627, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 1758, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 14861, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 13103, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 2, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 7241, 2024-09-10T22:30:16.829 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 7239, 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 72584, 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 281927680 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 287608, 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 1040, 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 252106, 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 570, 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 7095, 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 6907, 2024-09-10T22:30:16.830 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 360.469386362, 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.052188994 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 195753, 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 104.760496977, 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000535166 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.831 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 11769, 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 10.755783651, 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000913908 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.832 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.833 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.834 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.835 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 5721, 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 5.985185173, 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001046178 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 20704, 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 46.721142564, 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.002256623 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.836 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1522, 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 65.818891028, 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.043245000 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 20, 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.128122628, 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.006406131 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.838 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:16.839 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 176, 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.130837130, 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000743392 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 7070, 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 78.425784486, 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.011092755 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:16.841 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 1, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 29754, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 29754, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 1587, 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:16.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:16.843 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:16.844 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:16.845 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 456513, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 143, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 9888, 2024-09-10T22:30:16.846 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 202, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 4930371, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 21263, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-09-10T22:30:16.847 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 36521, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 231178681, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 32, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 36521, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 36521, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 198.292913399, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005429558 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 36521, 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 38328 2024-09-10T22:30:16.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 36521, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 1546, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 34975, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 0, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 10, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 1528, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 18762, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 917, 2024-09-10T22:30:16.849 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 8, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 14206, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 0, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 20, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:16.850 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-09-10T22:30:16.851 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 131, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 45, 2024-09-10T22:30:16.852 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 12, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 12, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 0, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 1958, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 36, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 883 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 1, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.853 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 5, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 14140, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 7070, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:16.854 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 647354, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 141433023, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 647354, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 647354, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 141433023, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:16.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 36521, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 231959567, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 36521, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 231959567, 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:16.856 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:16.857 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 36521, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 36521, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 36521, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 36521, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 7070, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 781174, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 7070, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 1528, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 781174, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:16.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x555590c96f00": { 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 64101, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 228778804, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 64101, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 17234, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 228778804, 2024-09-10T22:30:16.860 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:16.861 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:16.861 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:16.861 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:16.861 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:16.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:16.861 INFO:teuthology.orchestra.run.smithi154.stdout: "trackedop": { 2024-09-10T22:30:16.861 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 2972 2024-09-10T22:30:16.861 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:16.861 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-10T22:30:17.811 INFO:tasks.check_counter:Daemon mds.h mds.exported=54 2024-09-10T22:30:17.811 INFO:tasks.check_counter:Daemon mds.h mds.imported=26 2024-09-10T22:30:17.811 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=2896 2024-09-10T22:30:17.811 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=4744 2024-09-10T22:30:17.811 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-10T22:30:17.811 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-10T22:30:17.812 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T22:30:17.812 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T22:30:17.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T22:30:18.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:17 smithi154 ceph-mon[34495]: pgmap v1250: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T22:30:18.187 INFO:teuthology.orchestra.run.smithi154.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T22:30:18.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:17 smithi171 ceph-mon[34138]: pgmap v1250: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T22:30:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:17 smithi090 ceph-mon[30350]: pgmap v1250: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T22:30:18.869 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-09-10T22:30:18.869 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:18.869 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 214728, 2024-09-10T22:30:18.869 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 187575, 2024-09-10T22:30:18.869 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 68053602, 2024-09-10T22:30:18.869 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 113221769, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 12, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 5, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 27.833040047, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 14.897549261, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 14.398474165, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 1.703533073, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 187573, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 18.127569985, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000096642 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 365973, 2024-09-10T22:30:18.870 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.087401836, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000238 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 239387, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 267553, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 77475863, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 198957774, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 16, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 8, 2024-09-10T22:30:18.871 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 31.121492344, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 17.026284776, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 16.461348634, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.567545197, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 267552, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 24.321794042, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000090904 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 467258, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.123810368, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000264 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.872 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 1486176, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 317536 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_messages": 70998, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages": 66104, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_bytes": 21787335, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_bytes": 97665551, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_created_connections": 15, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_active_connections": 8, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_total_time": 10.714304698, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_send_time": 6.263340012, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_recv_time": 4.659964683, 2024-09-10T22:30:18.873 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_running_fast_dispatch_time": 0.631925606, 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 66103, 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 6.821366515, 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000103192 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 105069, 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.025169394, 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000239 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: "msgr_send_encrypted_bytes": 747376 2024-09-10T22:30:18.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "cct": { 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "total_workers": 1, 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "unhealthy_workers": 0 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-MDSRank": { 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 33718, 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 28.333412047, 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000840305 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:18.875 INFO:teuthology.orchestra.run.smithi154.stdout: "queue_len": 0, 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "complete_latency": { 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 15486, 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 2.316737695, 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000149602 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "mds": { 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "request": 136943, 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "reply": 135872, 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "reply_latency": { 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 135872, 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 721.562473617, 2024-09-10T22:30:18.876 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005310604 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_reply": 0, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "forward": 1071, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_complete": 37, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_commit": 109, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_split": 11, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_merge": 0, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes": 754, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_top": 3, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_bottom": 2, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pin_tail": 749, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_pinned": 750, 2024-09-10T22:30:18.877 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_expired": 4904, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "inodes_with_caps": 0, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "caps": 0, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "subtrees": 6, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse": 269932, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_hit": 223227, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_forward": 1063, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_discover": 392, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_dir_fetch": 2, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "traverse_lock": 121, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "load_cent": 0, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "q": 0, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "exported": 126, 2024-09-10T22:30:18.878 INFO:teuthology.orchestra.run.smithi154.stdout: "exported_inodes": 4026, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "imported": 63, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "imported_inodes": 2098, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rfiles": 694, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rbytes": 214554224, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "root_rsnaps": 6, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:18.879 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_inode_file_caps": 871, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_revoke": 14021, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_grant": 22364, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps": 27981, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_caps_dirty": 10132, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_cap_release": 557, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "process_request_cap_release": 25785 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.880 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache": { 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update": 3993, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_update_receipt": 3911, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_try_discover": 2041, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_send_discover": 4057, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "dir_handle_discover": 4493, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays": 9, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_created": 7259, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_enqueued": 7250, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "strays_migrated": 0, 2024-09-10T22:30:18.881 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_started": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_completed": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_path": 20, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_quiesce_inode": 3575, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_exportdir": 218, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_flush": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragmentdir": 11, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_started": 0, 2024-09-10T22:30:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "uninline_write_failed": 0 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log": { 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "evlrg": 0, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "evadd": 78194, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "evex": 0, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "evtrm": 0, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "ev": 78194, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "evexg": 0, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "evexd": 0, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "segadd": 101, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "segex": 0, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "segtrm": 0, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "seg": 101, 2024-09-10T22:30:18.883 INFO:teuthology.orchestra.run.smithi154.stdout: "segmjr": 8, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "segexg": 0, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "segexd": 0, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "expos": 4194304, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "wrpos": 167523949, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "rdpos": 0, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "jlat": { 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 17260, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 96.551952995, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.005593971 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "replayed": 0 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.884 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mem": { 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "ino": 630, 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "ino+": 12116, 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "ino-": 11486, 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "dir": 47, 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "dir+": 2226, 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "dir-": 2179, 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "dn": 754, 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "dn+": 17285, 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "dn-": 16531, 2024-09-10T22:30:18.885 INFO:teuthology.orchestra.run.smithi154.stdout: "cap": 0, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "cap+": 7443, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "cap-": 7443, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "rss": 79428, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "heap": 289267712 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_server": { 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_client_request": 185085, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "dispatch_server_request": 1304, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_request": 136943, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_client_session": 466, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "handle_peer_request": 12162, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "req_create_latency": { 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 6964, 2024-09-10T22:30:18.886 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 255.985296456, 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.036758371 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getattr_latency": { 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 78647, 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 38.872183278, 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000494261 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "req_link_latency": { 2024-09-10T22:30:18.887 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookup_latency": { 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 11112, 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 13.955389915, 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001255884 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupino_latency": { 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupname_latency": { 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.889 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "req_lssnap_latency": { 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mkdir_latency": { 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.890 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mknod_latency": { 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "req_mksnap_latency": { 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.891 INFO:teuthology.orchestra.run.smithi154.stdout: "req_open_latency": { 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 5461, 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 5.006434949, 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000916761 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "req_readdir_latency": { 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 24782, 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 41.184764841, 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.001661882 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rename_latency": { 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 1440, 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 28.716134185, 2024-09-10T22:30:18.892 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.019941759 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmdir_latency": { 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 9, 2024-09-10T22:30:18.893 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.033355896, 2024-09-10T22:30:18.894 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.003706210 2024-09-10T22:30:18.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.894 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:18.894 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.894 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.894 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.895 INFO:teuthology.orchestra.run.smithi154.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:18.895 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.895 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.895 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.895 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setattr_latency": { 2024-09-10T22:30:18.896 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 21, 2024-09-10T22:30:18.896 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 111.139505498, 2024-09-10T22:30:18.896 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 5.292357404 2024-09-10T22:30:18.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.896 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:18.897 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.897 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.897 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.897 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:18.898 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 186, 2024-09-10T22:30:18.898 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.086981372, 2024-09-10T22:30:18.898 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000467641 2024-09-10T22:30:18.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.898 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setlayout_latency": { 2024-09-10T22:30:18.899 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.899 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.899 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.899 INFO:teuthology.orchestra.run.smithi154.stdout: "req_setxattr_latency": { 2024-09-10T22:30:18.899 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.900 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.900 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.900 INFO:teuthology.orchestra.run.smithi154.stdout: "req_symlink_latency": { 2024-09-10T22:30:18.900 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.901 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.901 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.901 INFO:teuthology.orchestra.run.smithi154.stdout: "req_unlink_latency": { 2024-09-10T22:30:18.901 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 7250, 2024-09-10T22:30:18.902 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 226.582427227, 2024-09-10T22:30:18.902 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.031252748 2024-09-10T22:30:18.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.902 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:18.902 INFO:teuthology.orchestra.run.smithi154.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:18.902 INFO:teuthology.orchestra.run.smithi154.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:18.903 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.903 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.903 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.903 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:18.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.904 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessions": { 2024-09-10T22:30:18.904 INFO:teuthology.orchestra.run.smithi154.stdout: "session_count": 1, 2024-09-10T22:30:18.905 INFO:teuthology.orchestra.run.smithi154.stdout: "session_add": 2, 2024-09-10T22:30:18.905 INFO:teuthology.orchestra.run.smithi154.stdout: "session_remove": 1, 2024-09-10T22:30:18.905 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_open": 1, 2024-09-10T22:30:18.905 INFO:teuthology.orchestra.run.smithi154.stdout: "sessions_stale": 0, 2024-09-10T22:30:18.905 INFO:teuthology.orchestra.run.smithi154.stdout: "total_load": 18961, 2024-09-10T22:30:18.905 INFO:teuthology.orchestra.run.smithi154.stdout: "average_load": 18961, 2024-09-10T22:30:18.905 INFO:teuthology.orchestra.run.smithi154.stdout: "avg_session_uptime": 1593, 2024-09-10T22:30:18.905 INFO:teuthology.orchestra.run.smithi154.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool": { 2024-09-10T22:30:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:18.906 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:18.907 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:18.907 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:18.907 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:18.907 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:18.907 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:18.907 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:18.907 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:18.907 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:18.907 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:18.908 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:18.908 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:18.908 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:18.908 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:18.908 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:18.908 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:18.908 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:18.908 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:18.909 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:18.909 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:18.909 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:18.909 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:18.909 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:18.909 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:18.909 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:18.909 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:18.909 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_items": 0, 2024-09-10T22:30:18.910 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:18.910 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:18.910 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:18.910 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:18.910 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_bytes": 370631, 2024-09-10T22:30:18.910 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_anon_items": 134, 2024-09-10T22:30:18.910 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:18.910 INFO:teuthology.orchestra.run.smithi154.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:18.910 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bytes": 0, 2024-09-10T22:30:18.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_items": 0, 2024-09-10T22:30:18.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:18.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:18.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:18.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:18.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_bytes": 9888, 2024-09-10T22:30:18.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_items": 202, 2024-09-10T22:30:18.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:18.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:18.912 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:18.912 INFO:teuthology.orchestra.run.smithi154.stdout: "pgmap_items": 0, 2024-09-10T22:30:18.912 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_bytes": 2208982, 2024-09-10T22:30:18.912 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_co_items": 9466, 2024-09-10T22:30:18.912 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:18.912 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_1_items": 0, 2024-09-10T22:30:18.912 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:18.912 INFO:teuthology.orchestra.run.smithi154.stdout: "unittest_2_items": 0 2024-09-10T22:30:18.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.913 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter": { 2024-09-10T22:30:18.913 INFO:teuthology.orchestra.run.smithi154.stdout: "op_active": 0, 2024-09-10T22:30:18.913 INFO:teuthology.orchestra.run.smithi154.stdout: "op_laggy": 0, 2024-09-10T22:30:18.913 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send": 35021, 2024-09-10T22:30:18.913 INFO:teuthology.orchestra.run.smithi154.stdout: "op_send_bytes": 166039694, 2024-09-10T22:30:18.913 INFO:teuthology.orchestra.run.smithi154.stdout: "op_resend": 27, 2024-09-10T22:30:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: "op_reply": 35021, 2024-09-10T22:30:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: "op_latency": { 2024-09-10T22:30:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 35021, 2024-09-10T22:30:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 249.760635028, 2024-09-10T22:30:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.007131739 2024-09-10T22:30:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: "op_inflight": 0, 2024-09-10T22:30:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: "oplen_avg": { 2024-09-10T22:30:18.914 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 35021, 2024-09-10T22:30:18.915 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 36259 2024-09-10T22:30:18.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.915 INFO:teuthology.orchestra.run.smithi154.stdout: "op": 35021, 2024-09-10T22:30:18.915 INFO:teuthology.orchestra.run.smithi154.stdout: "op_r": 1004, 2024-09-10T22:30:18.915 INFO:teuthology.orchestra.run.smithi154.stdout: "op_w": 34017, 2024-09-10T22:30:18.915 INFO:teuthology.orchestra.run.smithi154.stdout: "op_rmw": 0, 2024-09-10T22:30:18.915 INFO:teuthology.orchestra.run.smithi154.stdout: "op_pg": 0, 2024-09-10T22:30:18.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_stat": 0, 2024-09-10T22:30:18.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_create": 10, 2024-09-10T22:30:18.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_read": 967, 2024-09-10T22:30:18.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_write": 18227, 2024-09-10T22:30:18.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writefull": 548, 2024-09-10T22:30:18.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_writesame": 0, 2024-09-10T22:30:18.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_append": 0, 2024-09-10T22:30:18.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_zero": 0, 2024-09-10T22:30:18.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_truncate": 10, 2024-09-10T22:30:18.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_delete": 14551, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_mapext": 0, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_getxattr": 0, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_setxattr": 20, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_call": 0, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_watch": 0, 2024-09-10T22:30:18.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_notify": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "osdop_other": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_active": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_send": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_resend": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "linger_ping": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_active": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_send": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "poolop_resend": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_active": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_send": 0, 2024-09-10T22:30:18.918 INFO:teuthology.orchestra.run.smithi154.stdout: "poolstat_resend": 0, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_active": 0, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_send": 0, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "statfs_resend": 0, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "command_active": 0, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "command_send": 0, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "command_resend": 0, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "map_epoch": 131, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "map_full": 0, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "map_inc": 45, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_sessions": 12, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_open": 17, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_session_close": 5, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_laggy": 0, 2024-09-10T22:30:18.919 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_wr": 1293, 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_rd": 74, 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_del": 559 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "oft": { 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_objs": 1, 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_updates": 0, 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "omap_total_removes": 0 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "purge_queue": { 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:18.920 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T22:30:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing": 0, 2024-09-10T22:30:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executing_high_water": 5, 2024-09-10T22:30:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed_ops": 14500, 2024-09-10T22:30:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_executed": 7250, 2024-09-10T22:30:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 525109, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 120021440, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 525109, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 525109, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 120021440, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.922 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 104857600, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 35021, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 166845792, 2024-09-10T22:30:18.923 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 35021, 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 166845792, 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 1024, 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 0, 2024-09-10T22:30:18.924 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 0, 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 35021, 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 35021, 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 35021, 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 35021, 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:18.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 7250, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 806690, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 7250, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 967, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 806690, 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:18.926 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "throttle-write_buf_throttle-0x561e44052dc0": { 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "val": 0, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "max": 3758096384, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "get_started": 0, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "get": 78194, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "get_sum": 163329645, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "get_or_fail_success": 78194, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "take": 0, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "take_sum": 0, 2024-09-10T22:30:18.927 INFO:teuthology.orchestra.run.smithi154.stdout: "put": 17260, 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: "put_sum": 163329645, 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: "wait": { 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: "avgcount": 0, 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: "sum": 0.000000000, 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: "avgtime": 0.000000000 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: "trackedop": { 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: "slow_ops_count": 3218 2024-09-10T22:30:18.928 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-09-10T22:30:18.929 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-09-10T22:30:19.806 INFO:tasks.check_counter:Daemon mds.k mds.exported=126 2024-09-10T22:30:19.806 INFO:tasks.check_counter:Daemon mds.k mds.imported=63 2024-09-10T22:30:19.806 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=3993 2024-09-10T22:30:19.806 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=3911 2024-09-10T22:30:19.806 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-09-10T22:30:19.807 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-10T22:30:19.807 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T22:30:19.807 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T22:30:19.807 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T22:30:20.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:19 smithi154 ceph-mon[34495]: pgmap v1251: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T22:30:20.207 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:30:20.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:19 smithi171 ceph-mon[34138]: pgmap v1251: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T22:30:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:19 smithi090 ceph-mon[30350]: pgmap v1251: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 3875, 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 3877, 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 87299519, 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 1232257, 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.578090199, 2024-09-10T22:30:21.115 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.218283847, 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.207281444, 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.111501133, 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3875, 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.200697618, 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000051792 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.116 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1830, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1748, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 68899563, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 693698, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.344452642, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.116693010, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.195761717, 2024-09-10T22:30:21.117 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.034381258, 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1748, 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.098577881, 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000056394 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1475840, 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 307584 2024-09-10T22:30:21.118 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1020, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1386, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 75759644, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 1069948, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.251903159, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.070738878, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.138665052, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.026647881, 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:21.119 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1384, 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.068599096, 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000049565 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 746224 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.120 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 10282, 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.279653931, 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000027198 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:21.121 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.122 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 8829, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 10, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 8819, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 8814, 2024-09-10T22:30:21.123 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 1, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-09-10T22:30:21.124 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:21.125 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-09-10T22:30:21.126 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-09-10T22:30:21.127 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:21.128 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_started": 0, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_write_failed": 0 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 64101, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-09-10T22:30:21.129 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 93, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 7, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 232973108, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 232972509, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.130 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.131 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 64101 2024-09-10T22:30:21.131 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.131 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-09-10T22:30:21.131 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 1464, 2024-09-10T22:30:21.131 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 9650, 2024-09-10T22:30:21.131 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 8186, 2024-09-10T22:30:21.131 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 59, 2024-09-10T22:30:21.131 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 148, 2024-09-10T22:30:21.131 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 89, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 8829, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 10109, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 1280, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 63604, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 268226560 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-09-10T22:30:21.132 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.133 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.134 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.135 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.136 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.136 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:21.136 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.136 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.136 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.136 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.136 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-09-10T22:30:21.137 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.137 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.137 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.137 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.137 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.138 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.139 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.140 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.141 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-09-10T22:30:21.142 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.143 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-09-10T22:30:21.144 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 1, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 2, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 1, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 1, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 2968, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:21.145 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:21.146 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:21.147 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 242551, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 85, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:21.148 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 9888, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 202, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 9587518, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 44002, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-09-10T22:30:21.149 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 6107, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 21, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 3, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 6107, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6107, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 6.962855603, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001140143 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-09-10T22:30:21.150 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6107, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 6108 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 6107, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 6107, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 3411, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 2694, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-09-10T22:30:21.151 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 1, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-09-10T22:30:21.152 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-09-10T22:30:21.153 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 132, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 46, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 11, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 12, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 1, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-09-10T22:30:21.154 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-09-10T22:30:21.155 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 6721, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 231319868, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 6721, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 6721, 2024-09-10T22:30:21.156 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 231319868, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:21.157 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 6107, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 228790146, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 6107, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 228790146, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-09-10T22:30:21.158 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 6107, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 6107, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 6107, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 6107, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.159 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.160 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x56143d401040": { 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:21.161 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: "trackedop": { 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:21.162 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-10T22:30:22.102 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-10T22:30:22.103 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-10T22:30:22.103 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-10T22:30:22.103 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-10T22:30:22.103 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-10T22:30:22.103 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-10T22:30:22.103 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T22:30:22.103 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T22:30:22.103 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T22:30:22.156 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:21 smithi171 ceph-mon[34138]: pgmap v1252: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 184 B/s wr, 13 op/s 2024-09-10T22:30:22.156 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:21 smithi171 ceph-mon[34138]: osdmap e132: 12 total, 12 up, 12 in 2024-09-10T22:30:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:21 smithi154 ceph-mon[34495]: pgmap v1252: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 184 B/s wr, 13 op/s 2024-09-10T22:30:22.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:21 smithi154 ceph-mon[34495]: osdmap e132: 12 total, 12 up, 12 in 2024-09-10T22:30:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:21 smithi090 ceph-mon[30350]: pgmap v1252: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 184 B/s wr, 13 op/s 2024-09-10T22:30:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:21 smithi090 ceph-mon[30350]: osdmap e132: 12 total, 12 up, 12 in 2024-09-10T22:30:22.477 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 440545, 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 470675, 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 131364147, 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 325115882, 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 15, 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 42.056988512, 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 21.167177541, 2024-09-10T22:30:23.197 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 25.750955932, 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.432474868, 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 470674, 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 12.865484766, 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000027334 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 851950, 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.151569627, 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000177 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 3872, 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 1184 2024-09-10T22:30:23.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 87365, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 95838, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 27919667, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 105656000, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 11, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 7, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 8.663889340, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 4.371670947, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 4.533898461, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.391382530, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 95837, 2024-09-10T22:30:23.199 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 3.103009132, 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000032377 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 153554, 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.027147947, 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000176 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1502688, 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 347072 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 250340, 2024-09-10T22:30:23.200 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 234330, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 76180777, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 140126170, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 13, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 9, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 21.504547443, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 10.876136452, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 12.975775357, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.325666179, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 234328, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 6.981263566, 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000029792 2024-09-10T22:30:23.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 445678, 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.069910889, 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000156 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 748048 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-09-10T22:30:23.202 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 44789, 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 17.107488038, 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000381957 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 14849, 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.864119378, 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000058193 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:23.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 321239, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 306174, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 306174, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 473.136087223, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001545317 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 15065, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 1, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:23.204 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 20, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 1, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 2697, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 292, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 2324, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 81, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 2278, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 2161, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 2, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 2, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 6, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 635151, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 600059, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 2619, 2024-09-10T22:30:23.205 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 520, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 102, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 3000, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 3, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 56, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 1566, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 91, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 3143, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 694, 2024-09-10T22:30:23.206 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 214554224, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 6, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 8, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 17009, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 16268, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:23.207 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 23673, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 5399, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 724, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 21170 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 1077, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 4999, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 2671, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 4893, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 4433, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 30, 2024-09-10T22:30:23.208 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 5867, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 5837, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 20, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 2024, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 58, 2024-09-10T22:30:23.209 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 1, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_started": 0, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_write_failed": 0 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 51390, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 51390, 2024-09-10T22:30:23.210 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 82, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 82, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 6, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 218383879, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 0, 2024-09-10T22:30:23.211 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-09-10T22:30:23.212 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 22746, 2024-09-10T22:30:23.212 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 97.766374168, 2024-09-10T22:30:23.212 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.004298178 2024-09-10T22:30:23.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.212 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 0 2024-09-10T22:30:23.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.212 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-09-10T22:30:23.212 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 2257, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 9789, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 7532, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 100, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 2692, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 2592, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 2697, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 13859, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 11162, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 2, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 6155, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 6153, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 76800, 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 286121984 2024-09-10T22:30:23.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 353659, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 18103, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 321239, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 573, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 43165, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5840, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 190.186475127, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.032566177 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 274486, 2024-09-10T22:30:23.214 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 102.249987286, 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000372514 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-09-10T22:30:23.215 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 7263, 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 5.464939341, 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000752435 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.216 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-09-10T22:30:23.217 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.217 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.217 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.217 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.217 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:23.217 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.217 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.218 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 2742, 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 1.604327274, 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000585093 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 8496, 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 20.139226573, 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.002370436 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1186, 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 4.956753623, 2024-09-10T22:30:23.220 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.004179387 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-09-10T22:30:23.221 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 30, 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.212609639, 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.007086987 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.222 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6, 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 53.427015403, 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 8.904502567 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 288, 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.096538086, 2024-09-10T22:30:23.223 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000335201 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-09-10T22:30:23.224 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 5837, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 94.798214871, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.016240913 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.225 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 3, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 1, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 2, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 34328, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 17164, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 1588, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.226 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:23.227 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:23.228 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 552895, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 177, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:23.229 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 9648, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 196, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 7260502, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 29338, 2024-09-10T22:30:23.230 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 38940, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 215371425, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 33, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 38940, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 38940, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 163.597605430, 2024-09-10T22:30:23.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.004201273 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 38940, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 40487 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 38940, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 1324, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 37616, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 0, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 10, 2024-09-10T22:30:23.232 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 1323, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 24069, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 813, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 1, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 11737, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 0, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 20, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:23.233 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-09-10T22:30:23.234 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 132, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 46, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 12, 2024-09-10T22:30:23.235 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 12, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 1740, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 772 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 1, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:23.236 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 5, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 11674, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 5837, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 778246, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 165385509, 2024-09-10T22:30:23.237 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 778246, 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 778246, 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 165385509, 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:23.238 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 38940, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 216011322, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 38940, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 216011322, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.239 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 38940, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 38940, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 38940, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 38940, 2024-09-10T22:30:23.240 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 5837, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 639913, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 5837, 2024-09-10T22:30:23.241 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 1323, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 639913, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x559f79368dc0": { 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 51390, 2024-09-10T22:30:23.242 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 214189575, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 51390, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 22746, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 214189575, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "trackedop": { 2024-09-10T22:30:23.243 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 1874 2024-09-10T22:30:23.244 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:23.244 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-10T22:30:23.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:23 smithi171 ceph-mon[34138]: pgmap v1254: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T22:30:24.130 INFO:tasks.check_counter:Daemon mds.f mds.exported=56 2024-09-10T22:30:24.130 INFO:tasks.check_counter:Daemon mds.f mds.imported=91 2024-09-10T22:30:24.130 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=1077 2024-09-10T22:30:24.130 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=4999 2024-09-10T22:30:24.130 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-10T22:30:24.130 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-10T22:30:24.131 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T22:30:24.131 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T22:30:24.131 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T22:30:24.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:23 smithi154 ceph-mon[34495]: pgmap v1254: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T22:30:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:23 smithi090 ceph-mon[30350]: pgmap v1254: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T22:30:24.503 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:30:25.182 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-10T22:30:25.182 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:25.182 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 3988, 2024-09-10T22:30:25.182 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 3906, 2024-09-10T22:30:25.182 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 111644628, 2024-09-10T22:30:25.182 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 1379404, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 7, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 3, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.671432540, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.247024991, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.294991356, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.101019500, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3906, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.216271514, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000055369 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:25.183 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1475728, 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 307104 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1470, 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1835, 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 130396466, 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 1268620, 2024-09-10T22:30:25.184 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 9, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 6, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.356786631, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.094156800, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.210347617, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.035636976, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1833, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.094500327, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000051555 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.185 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 802048 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1705, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1707, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 146492306, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 542197, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 6, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.344686470, 2024-09-10T22:30:25.186 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.099730895, 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.186561083, 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.042654388, 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1705, 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.082380053, 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000048316 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.187 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 10992, 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.295539874, 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000026886 2024-09-10T22:30:25.188 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-09-10T22:30:25.189 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 4980, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 440, 2024-09-10T22:30:25.190 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 4540, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 4352, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 2, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:25.191 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 4, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 234, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 6, 2024-09-10T22:30:25.192 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:25.193 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:25.194 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-09-10T22:30:25.195 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-09-10T22:30:25.196 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:25.196 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:25.196 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_started": 0, 2024-09-10T22:30:25.196 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:25.196 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_write_failed": 0 2024-09-10T22:30:25.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.196 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-09-10T22:30:25.196 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 39995, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 98, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 5, 2024-09-10T22:30:25.197 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 389329250, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 389329114, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 39995 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 2148, 2024-09-10T22:30:25.198 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 7024, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 4876, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 111, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 240, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 129, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 4980, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 7709, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 2729, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 63124, 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 273043456 2024-09-10T22:30:25.199 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-09-10T22:30:25.200 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.201 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.202 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:25.203 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.204 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.205 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.206 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.208 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.209 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-09-10T22:30:25.210 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 4, 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 2, 2024-09-10T22:30:25.212 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 2, 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 2970, 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:25.213 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:25.214 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:25.215 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 241023, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 82, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:25.216 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 9648, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 196, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 8581027, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 30921, 2024-09-10T22:30:25.217 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 6544, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 26, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 4, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 6544, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6544, 2024-09-10T22:30:25.218 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 7.472844890, 2024-09-10T22:30:25.219 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001141938 2024-09-10T22:30:25.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.219 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-09-10T22:30:25.219 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-09-10T22:30:25.219 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6544, 2024-09-10T22:30:25.219 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 6545 2024-09-10T22:30:25.219 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.219 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 6544, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 6544, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 3553, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 2988, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-09-10T22:30:25.220 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 2, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-09-10T22:30:25.221 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-09-10T22:30:25.222 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 132, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 46, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 11, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 11, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 0, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-09-10T22:30:25.223 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-09-10T22:30:25.224 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 7159, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 387855085, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 7159, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 7159, 2024-09-10T22:30:25.225 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 387855085, 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-10T22:30:25.226 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 6544, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 385252266, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 6544, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 385252266, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:25.227 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 6544, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 6544, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 6544, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 6544, 2024-09-10T22:30:25.228 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:25.229 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x55c4c09e7040": { 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-10T22:30:25.230 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:25.231 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:25.232 INFO:teuthology.orchestra.run.smithi171.stdout: "trackedop": { 2024-09-10T22:30:25.232 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-09-10T22:30:25.232 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:25.232 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-10T22:30:26.126 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-10T22:30:26.127 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-10T22:30:26.127 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-10T22:30:26.127 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-09-10T22:30:26.127 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-10T22:30:26.127 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-10T22:30:26.127 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T22:30:26.127 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T22:30:26.127 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T22:30:26.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:25 smithi154 ceph-mon[34495]: pgmap v1255: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-10T22:30:26.176 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:25 smithi171 ceph-mon[34138]: pgmap v1255: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-10T22:30:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:25 smithi090 ceph-mon[30350]: pgmap v1255: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-10T22:30:26.499 INFO:teuthology.orchestra.run.smithi171.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T22:30:27.156 INFO:teuthology.orchestra.run.smithi171.stdout:{ 2024-09-10T22:30:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T22:30:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1534, 2024-09-10T22:30:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 1534, 2024-09-10T22:30:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 54919435, 2024-09-10T22:30:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 487052, 2024-09-10T22:30:27.156 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 9, 2024-09-10T22:30:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-09-10T22:30:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.249869157, 2024-09-10T22:30:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.083588322, 2024-09-10T22:30:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.108987071, 2024-09-10T22:30:27.157 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.040276847, 2024-09-10T22:30:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1533, 2024-09-10T22:30:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.092009816, 2024-09-10T22:30:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000060019 2024-09-10T22:30:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.158 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 1628, 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 2000, 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 80106545, 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 1321683, 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 8, 2024-09-10T22:30:27.159 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 5, 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.329385603, 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.104195375, 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.162307450, 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.042931267, 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 1998, 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.098970599, 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000049534 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:27.160 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 804928 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_messages": 3647, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages": 3567, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_bytes": 82357735, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_bytes": 1274520, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_created_connections": 5, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_active_connections": 4, 2024-09-10T22:30:27.161 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_total_time": 0.595895404, 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_send_time": 0.233357348, 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_recv_time": 0.249425307, 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_running_fast_dispatch_time": 0.092104089, 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 3566, 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.199598751, 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000055972 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_handle_ack_lat": { 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.162 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_recv_encrypted_bytes": 1474800, 2024-09-10T22:30:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "msgr_send_encrypted_bytes": 311616 2024-09-10T22:30:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.163 INFO:teuthology.orchestra.run.smithi171.stdout: "cct": { 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "total_workers": 1, 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "unhealthy_workers": 0 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-MDSRank": { 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 10471, 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.284697417, 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000027189 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:27.164 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "finisher-PurgeQueue": { 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "queue_len": 0, 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "complete_latency": { 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "mds": { 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "request": 0, 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "reply": 0, 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "reply_latency": { 2024-09-10T22:30:27.165 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_reply": 0, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "forward": 0, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_complete": 0, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_fetch_keys": 0, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_commit": 0, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_split": 0, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_merge": 0, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes": 8523, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_top": 289, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_bottom": 8234, 2024-09-10T22:30:27.166 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pin_tail": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_pinned": 8111, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_expired": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "inodes_with_caps": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "caps": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "subtrees": 1, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_hit": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_forward": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_discover": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_dir_fetch": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_remote_ino": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "traverse_lock": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "load_cent": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "q": 0, 2024-09-10T22:30:27.167 INFO:teuthology.orchestra.run.smithi171.stdout: "exported": 0, 2024-09-10T22:30:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "exported_inodes": 0, 2024-09-10T22:30:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "imported": 0, 2024-09-10T22:30:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "imported_inodes": 0, 2024-09-10T22:30:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_dir_fetch": 0, 2024-09-10T22:30:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_backtrace_fetch": 0, 2024-09-10T22:30:27.168 INFO:teuthology.orchestra.run.smithi171.stdout: "openino_peer_discover": 0, 2024-09-10T22:30:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rfiles": 0, 2024-09-10T22:30:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rbytes": 0, 2024-09-10T22:30:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "root_rsnaps": 0, 2024-09-10T22:30:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T22:30:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_set_tag": 0, 2024-09-10T22:30:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T22:30:27.169 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_inotable_repaired": 0, 2024-09-10T22:30:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_inodes": 0, 2024-09-10T22:30:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T22:30:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T22:30:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "scrub_file_inodes": 0, 2024-09-10T22:30:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_inode_file_caps": 0, 2024-09-10T22:30:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T22:30:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_grant": 0, 2024-09-10T22:30:27.170 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T22:30:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T22:30:27.172 INFO:teuthology.orchestra.run.smithi171.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T22:30:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps": 0, 2024-09-10T22:30:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_caps_dirty": 0, 2024-09-10T22:30:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_cap_release": 0, 2024-09-10T22:30:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "process_request_cap_release": 0 2024-09-10T22:30:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.173 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_cache": { 2024-09-10T22:30:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update": 0, 2024-09-10T22:30:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_update_receipt": 0, 2024-09-10T22:30:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_try_discover": 0, 2024-09-10T22:30:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_send_discover": 0, 2024-09-10T22:30:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "dir_handle_discover": 0, 2024-09-10T22:30:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays": 0, 2024-09-10T22:30:27.174 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_delayed": 0, 2024-09-10T22:30:27.175 INFO:teuthology.orchestra.run.smithi171.stdout: "num_strays_enqueuing": 0, 2024-09-10T22:30:27.175 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_created": 0, 2024-09-10T22:30:27.175 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_enqueued": 0, 2024-09-10T22:30:27.175 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_reintegrated": 0, 2024-09-10T22:30:27.175 INFO:teuthology.orchestra.run.smithi171.stdout: "strays_migrated": 0, 2024-09-10T22:30:27.175 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_processing": 0, 2024-09-10T22:30:27.175 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_enqueued": 0, 2024-09-10T22:30:27.176 INFO:teuthology.orchestra.run.smithi171.stdout: "num_recovering_prioritized": 0, 2024-09-10T22:30:27.176 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_started": 0, 2024-09-10T22:30:27.176 INFO:teuthology.orchestra.run.smithi171.stdout: "recovery_completed": 0, 2024-09-10T22:30:27.176 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_path": 0, 2024-09-10T22:30:27.176 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_quiesce_inode": 0, 2024-09-10T22:30:27.176 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T22:30:27.177 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_exportdir": 0, 2024-09-10T22:30:27.177 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_flush": 0, 2024-09-10T22:30:27.177 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragmentdir": 0, 2024-09-10T22:30:27.177 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_fragstats": 0, 2024-09-10T22:30:27.177 INFO:teuthology.orchestra.run.smithi171.stdout: "ireq_inodestats": 0, 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_started": 0, 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_succeeded": 0, 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: "uninline_write_failed": 0 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_log": { 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: "evlrg": 0, 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: "evadd": 0, 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: "evex": 0, 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: "evtrm": 0, 2024-09-10T22:30:27.178 INFO:teuthology.orchestra.run.smithi171.stdout: "ev": 51390, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "evexg": 0, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "evexd": 0, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "segadd": 0, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "segex": 0, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "segtrm": 0, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "seg": 82, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "segmjr": 6, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "segexg": 0, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "segexd": 0, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "expos": 4194304, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "wrpos": 218383879, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "rdpos": 218368547, 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "jlat": { 2024-09-10T22:30:27.179 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "replayed": 51390 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_mem": { 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "ino": 2242, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "ino+": 9358, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "ino-": 7116, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "dir": 83, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "dir+": 120, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "dir-": 37, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "dn": 8523, 2024-09-10T22:30:27.180 INFO:teuthology.orchestra.run.smithi171.stdout: "dn+": 10120, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "dn-": 1597, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "cap": 0, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "cap+": 0, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "cap-": 0, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "rss": 65532, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "heap": 267366400 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_server": { 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_client_request": 0, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "dispatch_server_request": 0, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_request": 0, 2024-09-10T22:30:27.181 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_client_session": 0, 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "handle_peer_request": 0, 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "req_create_latency": { 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getattr_latency": { 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getfilelock_latency": { 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.182 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "req_link_latency": { 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookup_latency": { 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.183 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookuphash_latency": { 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupino_latency": { 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupname_latency": { 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.184 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupparent_latency": { 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lookupsnap_latency": { 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.185 INFO:teuthology.orchestra.run.smithi171.stdout: "req_lssnap_latency": { 2024-09-10T22:30:27.187 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.187 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.187 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.187 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mkdir_latency": { 2024-09-10T22:30:27.187 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.187 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.187 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.187 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.188 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mknod_latency": { 2024-09-10T22:30:27.188 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.188 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.188 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.188 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.188 INFO:teuthology.orchestra.run.smithi171.stdout: "req_mksnap_latency": { 2024-09-10T22:30:27.189 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.189 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.189 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.189 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.189 INFO:teuthology.orchestra.run.smithi171.stdout: "req_open_latency": { 2024-09-10T22:30:27.189 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.189 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "req_readdir_latency": { 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rename_latency": { 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "req_renamesnap_latency": { 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.190 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "req_snapdiff_latency": { 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmdir_latency": { 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmsnap_latency": { 2024-09-10T22:30:27.191 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "req_rmxattr_latency": { 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setattr_latency": { 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.192 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setdirlayout_latency": { 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setfilelock_latency": { 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.193 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setlayout_latency": { 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "req_setxattr_latency": { 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "req_symlink_latency": { 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.194 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "req_unlink_latency": { 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_revoke_eviction": 0, 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "cap_acquisition_throttle": 0, 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "req_getvxattr_latency": { 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:27.195 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_sessions": { 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "session_count": 2, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "session_add": 3, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "session_remove": 1, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_open": 2, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "sessions_stale": 0, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "total_load": 0, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "average_load": 0, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "avg_session_uptime": 2974, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "mdthresh_evicted": 0 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "mempool": { 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_bytes": 0, 2024-09-10T22:30:27.196 INFO:teuthology.orchestra.run.smithi171.stdout: "bloom_filter_items": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_alloc_items": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_data_items": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_other_items": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T22:30:27.197 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_bytes": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_extent_items": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_bytes": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_blob_items": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_fsck_items": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_bytes": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_txc_items": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_bytes": 0, 2024-09-10T22:30:27.198 INFO:teuthology.orchestra.run.smithi171.stdout: "bluestore_writing_items": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_bytes": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_items": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_reader_items": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "bluefs_file_writer_items": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_bytes": 233720, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_anon_items": 81, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_bytes": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "buffer_meta_items": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_bytes": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_items": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_bytes": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_mapbl_items": 0, 2024-09-10T22:30:27.199 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_bytes": 0, 2024-09-10T22:30:27.200 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_pglog_items": 0, 2024-09-10T22:30:27.200 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_bytes": 9648, 2024-09-10T22:30:27.200 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_items": 196, 2024-09-10T22:30:27.200 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T22:30:27.200 INFO:teuthology.orchestra.run.smithi171.stdout: "osdmap_mapping_items": 0, 2024-09-10T22:30:27.200 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_bytes": 0, 2024-09-10T22:30:27.200 INFO:teuthology.orchestra.run.smithi171.stdout: "pgmap_items": 0, 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_bytes": 11004609, 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "mds_co_items": 45506, 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_bytes": 0, 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_1_items": 0, 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_bytes": 0, 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "unittest_2_items": 0 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "objecter": { 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "op_active": 0, 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "op_laggy": 0, 2024-09-10T22:30:27.201 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send": 6183, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "op_send_bytes": 21, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "op_resend": 3, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "op_reply": 6183, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "op_latency": { 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6183, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 6.812858788, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.001101869 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "op_inflight": 0, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "oplen_avg": { 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 6183, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 6184 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.202 INFO:teuthology.orchestra.run.smithi171.stdout: "op": 6183, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "op_r": 6183, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "op_w": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "op_rmw": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "op_pg": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_stat": 3402, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_create": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_read": 2779, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_write": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writefull": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_writesame": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_append": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_zero": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_truncate": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_delete": 0, 2024-09-10T22:30:27.203 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_mapext": 0, 2024-09-10T22:30:27.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_sparse_read": 0, 2024-09-10T22:30:27.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_clonerange": 0, 2024-09-10T22:30:27.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_getxattr": 1, 2024-09-10T22:30:27.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_setxattr": 0, 2024-09-10T22:30:27.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_cmpxattr": 0, 2024-09-10T22:30:27.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_rmxattr": 0, 2024-09-10T22:30:27.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_resetxattrs": 0, 2024-09-10T22:30:27.204 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_call": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_watch": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_notify": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_pgls_filter": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "osdop_other": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_active": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_send": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_resend": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "linger_ping": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_active": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_send": 0, 2024-09-10T22:30:27.205 INFO:teuthology.orchestra.run.smithi171.stdout: "poolop_resend": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_active": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_send": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "poolstat_resend": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_active": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_send": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "statfs_resend": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "command_active": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "command_send": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "command_resend": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "map_epoch": 132, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "map_full": 0, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "map_inc": 46, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_sessions": 10, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_open": 11, 2024-09-10T22:30:27.206 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_session_close": 1, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "osd_laggy": 0, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_wr": 0, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_rd": 2, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_del": 0 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "oft": { 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_objs": 0, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_kv_pairs": 0, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_updates": 0, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "omap_total_removes": 0 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "purge_queue": { 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops": 0, 2024-09-10T22:30:27.207 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing": 0, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executing_high_water": 0, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed_ops": 0, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_executed": 0, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "pq_item_in_journal": 0 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 6805, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 216736890, 2024-09-10T22:30:27.208 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 6805, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 6805, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 216736890, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_bytes": { 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:27.209 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 104857600, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 6183, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 214193441, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 6183, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 214193441, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.210 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-objecter_ops": { 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 1024, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 6183, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 6183, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 6183, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 6183, 2024-09-10T22:30:27.211 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle": { 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:27.212 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "throttle-write_buf_throttle-0x55c2ef95d040": { 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "val": 0, 2024-09-10T22:30:27.213 INFO:teuthology.orchestra.run.smithi171.stdout: "max": 3758096384, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "get_started": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "get": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "get_sum": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_fail": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "get_or_fail_success": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "take": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "take_sum": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "put": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "put_sum": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "wait": { 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "avgcount": 0, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "sum": 0.000000000, 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: "avgtime": 0.000000000 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:27.214 INFO:teuthology.orchestra.run.smithi171.stdout: }, 2024-09-10T22:30:27.215 INFO:teuthology.orchestra.run.smithi171.stdout: "trackedop": { 2024-09-10T22:30:27.215 INFO:teuthology.orchestra.run.smithi171.stdout: "slow_ops_count": 0 2024-09-10T22:30:27.215 INFO:teuthology.orchestra.run.smithi171.stdout: } 2024-09-10T22:30:27.215 INFO:teuthology.orchestra.run.smithi171.stdout:} 2024-09-10T22:30:28.026 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-10T22:30:28.026 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-10T22:30:28.026 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-09-10T22:30:28.026 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-09-10T22:30:28.026 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-10T22:30:28.026 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-10T22:30:28.026 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-10T22:30:28.026 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T22:30:28.037 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T22:30:28.037 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T22:30:28.037 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:30:28.089 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T22:30:28.090 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:30:28.168 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T22:30:28.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:27 smithi154 ceph-mon[34495]: pgmap v1256: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-10T22:30:28.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:27 smithi171 ceph-mon[34138]: pgmap v1256: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-10T22:30:28.261 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:27 smithi090 ceph-mon[30350]: pgmap v1256: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-10T22:30:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:29.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.225+0000 7fda1b200640 1 -- 172.21.15.90:0/1131233483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c104df0 msgr2=0x7fda1c1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:29.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.225+0000 7fda1b200640 1 --2- 172.21.15.90:0/1131233483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c104df0 0x7fda1c1051d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009900 tx=0x7fda0c02a410 comp rx=0 tx=0).stop 2024-09-10T22:30:29.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.225+0000 7fda1b200640 1 -- 172.21.15.90:0/1131233483 shutdown_connections 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.225+0000 7fda1b200640 1 --2- 172.21.15.90:0/1131233483 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda1c10b180 0x7fda1c111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.225+0000 7fda1b200640 1 --2- 172.21.15.90:0/1131233483 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda1c10a800 0x7fda1c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.225+0000 7fda1b200640 1 --2- 172.21.15.90:0/1131233483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c104df0 0x7fda1c1051d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.225+0000 7fda1b200640 1 -- 172.21.15.90:0/1131233483 >> 172.21.15.90:0/1131233483 conn(0x7fda1c100840 msgr2=0x7fda1c102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.225+0000 7fda1b200640 1 -- 172.21.15.90:0/1131233483 shutdown_connections 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.225+0000 7fda1b200640 1 -- 172.21.15.90:0/1131233483 wait complete. 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.226+0000 7fda1b200640 1 Processor -- start 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.226+0000 7fda1b200640 1 -- start start 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.226+0000 7fda1b200640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda1c104df0 0x7fda1c19b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.226+0000 7fda1b200640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda1c10a800 0x7fda1c19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.226+0000 7fda1b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c10b180 0x7fda1c1a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.226+0000 7fda1b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c114d00 con 0x7fda1c10b180 2024-09-10T22:30:29.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.226+0000 7fda1b200640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c114b80 con 0x7fda1c104df0 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.226+0000 7fda1b200640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fda1c114e80 con 0x7fda1c10a800 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda1a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c10b180 0x7fda1c1a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda1a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c10b180 0x7fda1c1a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37680/0 (socket says 172.21.15.90:37680) 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda1a800640 1 -- 172.21.15.90:0/2289099063 learned_addr learned my addr 172.21.15.90:0/2289099063 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda19e00640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda1c104df0 0x7fda1c19b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda19400640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda1c10a800 0x7fda1c19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda1a800640 1 -- 172.21.15.90:0/2289099063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda1c10a800 msgr2=0x7fda1c19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda1a800640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda1c10a800 0x7fda1c19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda1a800640 1 -- 172.21.15.90:0/2289099063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda1c104df0 msgr2=0x7fda1c19b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda1a800640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda1c104df0 0x7fda1c19b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.227+0000 7fda1a800640 1 -- 172.21.15.90:0/2289099063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1c1a11c0 con 0x7fda1c10b180 2024-09-10T22:30:29.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.228+0000 7fda1a800640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c10b180 0x7fda1c1a09f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fda1400eaa0 tx=0x7fda1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.228+0000 7fda12a00640 1 -- 172.21.15.90:0/2289099063 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1400cdb0 con 0x7fda1c10b180 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.228+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1c1a14b0 con 0x7fda1c10b180 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.228+0000 7fda12a00640 1 -- 172.21.15.90:0/2289099063 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fda14005020 con 0x7fda1c10b180 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.228+0000 7fda12a00640 1 -- 172.21.15.90:0/2289099063 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140116d0 con 0x7fda1c10b180 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.228+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1c1a7b70 con 0x7fda1c10b180 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.229+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda1c1051d0 con 0x7fda1c10b180 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.231+0000 7fda12a00640 1 -- 172.21.15.90:0/2289099063 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fda1400f040 con 0x7fda1c10b180 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.233+0000 7fda12a00640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda000b5500 0x7fda000b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.234+0000 7fda12a00640 1 -- 172.21.15.90:0/2289099063 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9490+0+0 (secure 0 0 0) 0x7fda140e6250 con 0x7fda1c10b180 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.234+0000 7fda19e00640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda000b5500 0x7fda000b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.234+0000 7fda19e00640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda000b5500 0x7fda000b79c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fda0c006010 tx=0x7fda0c005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:29.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.237+0000 7fda12a00640 1 -- 172.21.15.90:0/2289099063 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fda14016030 con 0x7fda1c10b180 2024-09-10T22:30:29.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.434+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fda1c10c9d0 con 0x7fda1c10b180 2024-09-10T22:30:29.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.434+0000 7fda12a00640 1 -- 172.21.15.90:0/2289099063 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v132) ==== 81+0+886 (secure 0 0 0) 0x7fda140ac350 con 0x7fda1c10b180 2024-09-10T22:30:29.436 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2204308252 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:29.436 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1572343768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:29.436 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1613780567 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1385951279 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/323032163 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3233554768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1613780567 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3858975894 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2255342982 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3643795744 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2773946294 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2694078227 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/323032163 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3233554768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:29.437 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3503416190 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:29.438 INFO:teuthology.orchestra.run.smithi090.stderr:listed 15 entries 2024-09-10T22:30:29.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.436+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda000b5500 msgr2=0x7fda000b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:29.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.436+0000 7fda1b200640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda000b5500 0x7fda000b79c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fda0c006010 tx=0x7fda0c005e80 comp rx=0 tx=0).stop 2024-09-10T22:30:29.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.436+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c10b180 msgr2=0x7fda1c1a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:29.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.436+0000 7fda1b200640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c10b180 0x7fda1c1a09f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fda1400eaa0 tx=0x7fda1400ef60 comp rx=0 tx=0).stop 2024-09-10T22:30:29.438 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.436+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 shutdown_connections 2024-09-10T22:30:29.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.436+0000 7fda1b200640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fda000b5500 0x7fda000b79c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:29.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.436+0000 7fda1b200640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda1c10b180 0x7fda1c1a09f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:29.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.436+0000 7fda1b200640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fda1c10a800 0x7fda1c19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:29.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.436+0000 7fda1b200640 1 --2- 172.21.15.90:0/2289099063 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fda1c104df0 0x7fda1c19b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:29.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.437+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 >> 172.21.15.90:0/2289099063 conn(0x7fda1c100840 msgr2=0x7fda1c102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:29.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.437+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 shutdown_connections 2024-09-10T22:30:29.439 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:29.437+0000 7fda1b200640 1 -- 172.21.15.90:0/2289099063 wait complete. 2024-09-10T22:30:30.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:29 smithi171 ceph-mon[34138]: pgmap v1257: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-10T22:30:30.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:29 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2289099063' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:30:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:29 smithi090 ceph-mon[30350]: pgmap v1257: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-10T22:30:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:29 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2289099063' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:30:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:29 smithi154 ceph-mon[34495]: pgmap v1257: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2024-09-10T22:30:30.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:29 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2289099063' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:30:30.521 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-09-10T22:30:30.521 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T22:30:30.559 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T22:30:30.968 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:31.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.626+0000 7f17f2c00640 1 -- 172.21.15.90:0/258125552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec113990 msgr2=0x7f17ec115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:31.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.626+0000 7f17f2c00640 1 --2- 172.21.15.90:0/258125552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec113990 0x7f17ec115d80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009ca0 tx=0x7f17e402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.626+0000 7f17f2c00640 1 -- 172.21.15.90:0/258125552 shutdown_connections 2024-09-10T22:30:31.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.626+0000 7f17f2c00640 1 --2- 172.21.15.90:0/258125552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec113990 0x7f17ec115d80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.626+0000 7f17f2c00640 1 --2- 172.21.15.90:0/258125552 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17ec077ff0 0x7f17ec113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.626+0000 7f17f2c00640 1 --2- 172.21.15.90:0/258125552 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ec0776d0 0x7f17ec077ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.626+0000 7f17f2c00640 1 -- 172.21.15.90:0/258125552 >> 172.21.15.90:0/258125552 conn(0x7f17ec100a50 msgr2=0x7f17ec102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:31.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.626+0000 7f17f2c00640 1 -- 172.21.15.90:0/258125552 shutdown_connections 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.626+0000 7f17f2c00640 1 -- 172.21.15.90:0/258125552 wait complete. 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.627+0000 7f17f2c00640 1 Processor -- start 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.627+0000 7f17f2c00640 1 -- start start 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.627+0000 7f17f2c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec0776d0 0x7f17ec19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.627+0000 7f17f2c00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17ec077ff0 0x7f17ec19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.627+0000 7f17f2c00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ec113990 0x7f17ec1a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.627+0000 7f17f2c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec119270 con 0x7f17ec0776d0 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.627+0000 7f17f2c00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1190f0 con 0x7f17ec077ff0 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.627+0000 7f17f2c00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f17ec1193f0 con 0x7f17ec113990 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17ebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec0776d0 0x7f17ec19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17ebe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec0776d0 0x7f17ec19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37692/0 (socket says 172.21.15.90:37692) 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17ebe00640 1 -- 172.21.15.90:0/804275923 learned_addr learned my addr 172.21.15.90:0/804275923 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17f0c00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ec113990 0x7f17ec1a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:31.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17eb400640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17ec077ff0 0x7f17ec19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17ebe00640 1 -- 172.21.15.90:0/804275923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ec113990 msgr2=0x7f17ec1a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17ebe00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ec113990 0x7f17ec1a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17ebe00640 1 -- 172.21.15.90:0/804275923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17ec077ff0 msgr2=0x7f17ec19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17ebe00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17ec077ff0 0x7f17ec19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17ebe00640 1 -- 172.21.15.90:0/804275923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17ec1a5820 con 0x7f17ec0776d0 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.628+0000 7f17eb400640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17ec077ff0 0x7f17ec19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.629+0000 7f17ebe00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec0776d0 0x7f17ec19f770 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00d770 tx=0x7f17dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.629+0000 7f17e8c00640 1 -- 172.21.15.90:0/804275923 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc017070 con 0x7f17ec0776d0 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.629+0000 7f17e8c00640 1 -- 172.21.15.90:0/804275923 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f17dc0054b0 con 0x7f17ec0776d0 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.629+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17ec1a5b10 con 0x7f17ec0776d0 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.629+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17ec1ac1d0 con 0x7f17ec0776d0 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.629+0000 7f17e8c00640 1 -- 172.21.15.90:0/804275923 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17dc004c60 con 0x7f17ec0776d0 2024-09-10T22:30:31.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.630+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17ec077ab0 con 0x7f17ec0776d0 2024-09-10T22:30:31.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.632+0000 7f17e8c00640 1 -- 172.21.15.90:0/804275923 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f17dc004e00 con 0x7f17ec0776d0 2024-09-10T22:30:31.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.634+0000 7f17e8c00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17d00b5450 0x7f17d00b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:31.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.634+0000 7f17e8c00640 1 -- 172.21.15.90:0/804275923 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9490+0+0 (secure 0 0 0) 0x7f17dc0e62a0 con 0x7f17ec0776d0 2024-09-10T22:30:31.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.634+0000 7f17eb400640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17d00b5450 0x7f17d00b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:31.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.635+0000 7f17eb400640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17d00b5450 0x7f17d00b7910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f17ec115fb0 tx=0x7f17d800b5f0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:31.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.639+0000 7f17e8c00640 1 -- 172.21.15.90:0/804275923 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f17dc0ac9f0 con 0x7f17ec0776d0 2024-09-10T22:30:31.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.845+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f17ec1ac540 con 0x7f17ec0776d0 2024-09-10T22:30:31.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.846+0000 7f17e8c00640 1 -- 172.21.15.90:0/804275923 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v132) ==== 81+0+886 (secure 0 0 0) 0x7f17dc0ac2f0 con 0x7f17ec0776d0 2024-09-10T22:30:31.848 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2204308252 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:31.848 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1572343768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:31.848 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1613780567 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:31.848 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1385951279 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:31.848 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/323032163 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:31.848 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3233554768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:31.848 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1613780567 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:31.848 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3858975894 2024-09-11T21:50:49.799954+0000 2024-09-10T22:30:31.849 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2255342982 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:31.849 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3643795744 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:31.849 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2773946294 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:31.849 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2694078227 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:31.849 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/323032163 2024-09-11T21:51:17.626867+0000 2024-09-10T22:30:31.849 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3233554768 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:31.849 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3503416190 2024-09-11T21:52:33.477408+0000 2024-09-10T22:30:31.849 INFO:teuthology.orchestra.run.smithi090.stderr:listed 15 entries 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.848+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17d00b5450 msgr2=0x7f17d00b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.848+0000 7f17f2c00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17d00b5450 0x7f17d00b7910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f17ec115fb0 tx=0x7f17d800b5f0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.848+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec0776d0 msgr2=0x7f17ec19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.848+0000 7f17f2c00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec0776d0 0x7f17ec19f770 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f17dc00d770 tx=0x7f17dc00dc30 comp rx=0 tx=0).stop 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.849+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 shutdown_connections 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.849+0000 7f17f2c00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f17d00b5450 0x7f17d00b7910 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.849+0000 7f17f2c00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f17ec113990 0x7f17ec1a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.849+0000 7f17f2c00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f17ec077ff0 0x7f17ec19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.849+0000 7f17f2c00640 1 --2- 172.21.15.90:0/804275923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17ec0776d0 0x7f17ec19f770 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.849+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 >> 172.21.15.90:0/804275923 conn(0x7f17ec100a50 msgr2=0x7f17ec102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.849+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 shutdown_connections 2024-09-10T22:30:31.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:31.849+0000 7f17f2c00640 1 -- 172.21.15.90:0/804275923 wait complete. 2024-09-10T22:30:31.997 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:31 smithi171 ceph-mon[34138]: pgmap v1258: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-10T22:30:31.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:31 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/804275923' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:30:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:31 smithi090 ceph-mon[30350]: pgmap v1258: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-10T22:30:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:31 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/804275923' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:30:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:31 smithi154 ceph-mon[34495]: pgmap v1258: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-10T22:30:32.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:31 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/804275923' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T22:30:33.042 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi090.front.sepia.ceph.com... 2024-09-10T22:30:33.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:30:33.043 DEBUG:teuthology.orchestra.run.smithi090:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T22:30:33.109 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T22:30:33.110 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi090.stderr:ceph-fuse[89329]: fuse finished with error 0 and tester_r 0 2024-09-10T22:30:34.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:33 smithi171 ceph-mon[34138]: pgmap v1259: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T22:30:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:33 smithi090 ceph-mon[30350]: pgmap v1259: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T22:30:34.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:33 smithi154 ceph-mon[34495]: pgmap v1259: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T22:30:36.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:35 smithi171 ceph-mon[34138]: pgmap v1260: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T22:30:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:35 smithi090 ceph-mon[30350]: pgmap v1260: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T22:30:36.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:35 smithi154 ceph-mon[34495]: pgmap v1260: 129 pgs: 129 active+clean; 1.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 11 op/s 2024-09-10T22:30:38.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:37 smithi171 ceph-mon[34138]: pgmap v1261: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 13 op/s 2024-09-10T22:30:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:37 smithi090 ceph-mon[30350]: pgmap v1261: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 13 op/s 2024-09-10T22:30:38.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:37 smithi154 ceph-mon[34495]: pgmap v1261: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 13 op/s 2024-09-10T22:30:39.211 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-09-10T22:30:39.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:30:39.212 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T22:30:39.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:30:39.286 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-09-10T22:30:39.367 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T22:30:39.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:30:39.368 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T22:30:39.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T22:30:39.429 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-09-10T22:30:39.802 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T22:30:39.812 INFO:tasks.cephadm:Teardown begin 2024-09-10T22:30:39.813 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T22:30:39.813 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:30:40.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:39 smithi090 ceph-mon[30350]: pgmap v1262: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T22:30:40.192 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:40.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:39 smithi171 ceph-mon[34138]: pgmap v1262: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T22:30:40.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:39 smithi154 ceph-mon[34495]: pgmap v1262: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T22:30:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.839+0000 7f2f8ca00640 1 -- 172.21.15.90:0/2930677871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f8810d120 msgr2=0x7f2f8810f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.839+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/2930677871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f8810d120 0x7f2f8810f580 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009ca0 tx=0x7f2f8002a8c0 comp rx=0 tx=0).stop 2024-09-10T22:30:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.840+0000 7f2f8ca00640 1 -- 172.21.15.90:0/2930677871 shutdown_connections 2024-09-10T22:30:40.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.840+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/2930677871 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f8810d120 0x7f2f8810f580 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.840+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/2930677871 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f881045e0 0x7f2f8810cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.840+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/2930677871 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f88103cc0 0x7f2f881040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.840+0000 7f2f8ca00640 1 -- 172.21.15.90:0/2930677871 >> 172.21.15.90:0/2930677871 conn(0x7f2f880fd700 msgr2=0x7f2f880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.840+0000 7f2f8ca00640 1 -- 172.21.15.90:0/2930677871 shutdown_connections 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.840+0000 7f2f8ca00640 1 -- 172.21.15.90:0/2930677871 wait complete. 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.841+0000 7f2f8ca00640 1 Processor -- start 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.841+0000 7f2f8ca00640 1 -- start start 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.841+0000 7f2f8ca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f88103cc0 0x7f2f88101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.841+0000 7f2f8ca00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f881045e0 0x7f2f88101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.841+0000 7f2f8ca00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f8810d120 0x7f2f881023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:40.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.841+0000 7f2f8ca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88112cf0 con 0x7f2f88103cc0 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f8ca00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88112b70 con 0x7f2f881045e0 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f8ca00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2f88112e70 con 0x7f2f8810d120 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f85e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f88103cc0 0x7f2f88101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f85e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f88103cc0 0x7f2f88101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40756/0 (socket says 172.21.15.90:40756) 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f85e00640 1 -- 172.21.15.90:0/1455074304 learned_addr learned my addr 172.21.15.90:0/1455074304 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f7d400640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f881045e0 0x7f2f88101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f86800640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f8810d120 0x7f2f881023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f85e00640 1 -- 172.21.15.90:0/1455074304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f8810d120 msgr2=0x7f2f881023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f85e00640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f8810d120 0x7f2f881023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f85e00640 1 -- 172.21.15.90:0/1455074304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f881045e0 msgr2=0x7f2f88101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f85e00640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f881045e0 0x7f2f88101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:40.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f85e00640 1 -- 172.21.15.90:0/1455074304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f881a7620 con 0x7f2f88103cc0 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.842+0000 7f2f86800640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f8810d120 0x7f2f881023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.843+0000 7f2f85e00640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f88103cc0 0x7f2f88101970 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800d950 tx=0x7f2f7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.843+0000 7f2f7f400640 1 -- 172.21.15.90:0/1455074304 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f78017070 con 0x7f2f88103cc0 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.843+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f881a7910 con 0x7f2f88103cc0 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.843+0000 7f2f7f400640 1 -- 172.21.15.90:0/1455074304 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f78009c30 con 0x7f2f88103cc0 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.843+0000 7f2f7f400640 1 -- 172.21.15.90:0/1455074304 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f7800f620 con 0x7f2f88103cc0 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.843+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f881a7ea0 con 0x7f2f88103cc0 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.844+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f881040a0 con 0x7f2f88103cc0 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.846+0000 7f2f7f400640 1 -- 172.21.15.90:0/1455074304 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2f78005030 con 0x7f2f88103cc0 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.847+0000 7f2f7f400640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f6c0b54d0 0x7f2f6c0b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:40.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.848+0000 7f2f7f400640 1 -- 172.21.15.90:0/1455074304 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9490+0+0 (secure 0 0 0) 0x7f2f780e5fb0 con 0x7f2f88103cc0 2024-09-10T22:30:40.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.848+0000 7f2f7d400640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f6c0b54d0 0x7f2f6c0b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:40.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.848+0000 7f2f7d400640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f6c0b54d0 0x7f2f6c0b7990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2f88103460 tx=0x7f2f70009060 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:40.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:40.849+0000 7f2f7f400640 1 -- 172.21.15.90:0/1455074304 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2f780ac780 con 0x7f2f88103cc0 2024-09-10T22:30:41.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.069+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f88102c70 con 0x7f2f88103cc0 2024-09-10T22:30:41.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.070+0000 7f2f7f400640 1 -- 172.21.15.90:0/1455074304 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+12015 (secure 0 0 0) 0x7f2f88102c70 con 0x7f2f88103cc0 2024-09-10T22:30:41.072 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:30:41.072 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":103,"btime":"2024-09-10T22:29:17:913064+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":103,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:29:17.776682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:30:41.072 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 103 2024-09-10T22:30:41.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.072+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f6c0b54d0 msgr2=0x7f2f6c0b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:41.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.072+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f6c0b54d0 0x7f2f6c0b7990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2f88103460 tx=0x7f2f70009060 comp rx=0 tx=0).stop 2024-09-10T22:30:41.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.072+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f88103cc0 msgr2=0x7f2f88101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:41.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.072+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f88103cc0 0x7f2f88101970 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800d950 tx=0x7f2f7800de10 comp rx=0 tx=0).stop 2024-09-10T22:30:41.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.072+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 shutdown_connections 2024-09-10T22:30:41.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.072+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2f6c0b54d0 0x7f2f6c0b7990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:41.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.072+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2f8810d120 0x7f2f881023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:41.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.073+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2f881045e0 0x7f2f88101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:41.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.073+0000 7f2f8ca00640 1 --2- 172.21.15.90:0/1455074304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f88103cc0 0x7f2f88101970 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:41.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.073+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 >> 172.21.15.90:0/1455074304 conn(0x7f2f880fd700 msgr2=0x7f2f880fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:41.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.073+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 shutdown_connections 2024-09-10T22:30:41.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:41.073+0000 7f2f8ca00640 1 -- 172.21.15.90:0/1455074304 wait complete. 2024-09-10T22:30:42.232 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:41 smithi090 ceph-mon[30350]: pgmap v1263: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-10T22:30:42.232 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:41 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1455074304' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:30:42.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:41 smithi171 ceph-mon[34138]: pgmap v1263: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-10T22:30:42.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:41 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1455074304' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:30:42.277 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T22:30:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:41 smithi154 ceph-mon[34495]: pgmap v1263: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 15 op/s 2024-09-10T22:30:42.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:41 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1455074304' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:30:42.654 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:43.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.285+0000 7f2e76600640 1 -- 172.21.15.90:0/1641172919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e7010b330 msgr2=0x7f2e70111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:43.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.285+0000 7f2e76600640 1 --2- 172.21.15.90:0/1641172919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e7010b330 0x7f2e70111b30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009ca0 tx=0x7f2e6802a8c0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.285+0000 7f2e76600640 1 -- 172.21.15.90:0/1641172919 shutdown_connections 2024-09-10T22:30:43.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.285+0000 7f2e76600640 1 --2- 172.21.15.90:0/1641172919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e7010b330 0x7f2e70111b30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.285+0000 7f2e76600640 1 --2- 172.21.15.90:0/1641172919 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e7010a9b0 0x7f2e7010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.285+0000 7f2e76600640 1 --2- 172.21.15.90:0/1641172919 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e70104fa0 0x7f2e70105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.285+0000 7f2e76600640 1 -- 172.21.15.90:0/1641172919 >> 172.21.15.90:0/1641172919 conn(0x7f2e70100a50 msgr2=0x7f2e70102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:43.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.285+0000 7f2e76600640 1 -- 172.21.15.90:0/1641172919 shutdown_connections 2024-09-10T22:30:43.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.285+0000 7f2e76600640 1 -- 172.21.15.90:0/1641172919 wait complete. 2024-09-10T22:30:43.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.287+0000 7f2e76600640 1 Processor -- start 2024-09-10T22:30:43.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.287+0000 7f2e76600640 1 -- start start 2024-09-10T22:30:43.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.287+0000 7f2e76600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e70104fa0 0x7f2e7019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:43.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.287+0000 7f2e76600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e7010a9b0 0x7f2e7019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:43.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.287+0000 7f2e76600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e7010b330 0x7f2e701a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:43.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.287+0000 7f2e76600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70114e60 con 0x7f2e70104fa0 2024-09-10T22:30:43.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.287+0000 7f2e76600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70114ce0 con 0x7f2e7010a9b0 2024-09-10T22:30:43.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e76600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f2e70114fe0 con 0x7f2e7010b330 2024-09-10T22:30:43.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e70104fa0 0x7f2e7019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e70104fa0 0x7f2e7019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40782/0 (socket says 172.21.15.90:40782) 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6f400640 1 -- 172.21.15.90:0/4234713648 learned_addr learned my addr 172.21.15.90:0/4234713648 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6ea00640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e7010a9b0 0x7f2e7019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6fe00640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e7010b330 0x7f2e701a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6f400640 1 -- 172.21.15.90:0/4234713648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e7010b330 msgr2=0x7f2e701a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6f400640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e7010b330 0x7f2e701a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6f400640 1 -- 172.21.15.90:0/4234713648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e7010a9b0 msgr2=0x7f2e7019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6f400640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e7010a9b0 0x7f2e7019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6f400640 1 -- 172.21.15.90:0/4234713648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e701a1410 con 0x7f2e70104fa0 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.288+0000 7f2e6ea00640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e7010a9b0 0x7f2e7019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.289+0000 7f2e6f400640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e70104fa0 0x7f2e7019b360 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000e9f0 tx=0x7f2e6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.289+0000 7f2e67e00640 1 -- 172.21.15.90:0/4234713648 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000cd80 con 0x7f2e70104fa0 2024-09-10T22:30:43.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.289+0000 7f2e67e00640 1 -- 172.21.15.90:0/4234713648 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2e600108c0 con 0x7f2e70104fa0 2024-09-10T22:30:43.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.289+0000 7f2e67e00640 1 -- 172.21.15.90:0/4234713648 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e600187c0 con 0x7f2e70104fa0 2024-09-10T22:30:43.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.289+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e701a1700 con 0x7f2e70104fa0 2024-09-10T22:30:43.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.289+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e701a7dc0 con 0x7f2e70104fa0 2024-09-10T22:30:43.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.290+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e70105380 con 0x7f2e70104fa0 2024-09-10T22:30:43.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.293+0000 7f2e67e00640 1 -- 172.21.15.90:0/4234713648 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f2e60010430 con 0x7f2e70104fa0 2024-09-10T22:30:43.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.294+0000 7f2e67e00640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2e540b54d0 0x7f2e540b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:43.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.294+0000 7f2e67e00640 1 -- 172.21.15.90:0/4234713648 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9490+0+0 (secure 0 0 0) 0x7f2e600e6d40 con 0x7f2e70104fa0 2024-09-10T22:30:43.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.295+0000 7f2e6ea00640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2e540b54d0 0x7f2e540b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:43.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.295+0000 7f2e6ea00640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2e540b54d0 0x7f2e540b7990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e701071c0 tx=0x7f2e5c005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:43.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.296+0000 7f2e67e00640 1 -- 172.21.15.90:0/4234713648 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2e600ad490 con 0x7f2e70104fa0 2024-09-10T22:30:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.529+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e7002e5c0 con 0x7f2e70104fa0 2024-09-10T22:30:43.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.530+0000 7f2e67e00640 1 -- 172.21.15.90:0/4234713648 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) ==== 95+0+12015 (secure 0 0 0) 0x7f2e600acd90 con 0x7f2e70104fa0 2024-09-10T22:30:43.532 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:30:43.532 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":103,"btime":"2024-09-10T22:29:17:913064+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/1514545418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":1514545418},{"type":"v1","addr":"172.21.15.90:6835","nonce":1514545418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/4068804869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":4068804869},{"type":"v1","addr":"172.21.15.90:6837","nonce":4068804869}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":103,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T22:00:35.595646+0000","modified":"2024-09-10T22:29:17.776682+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":34703,"mds_1":34670,"mds_2":34646,"mds_3":34640,"mds_4":34607},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14841":{"gid":14841,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6839/3596805819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":3596805819},{"type":"v1","addr":"172.21.15.90:6839","nonce":3596805819}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14853":{"gid":14853,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.90:6841/3169114114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3169114114},{"type":"v1","addr":"172.21.15.90:6841","nonce":3169114114}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24598":{"gid":24598,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6833/3906453887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6832","nonce":3906453887},{"type":"v1","addr":"172.21.15.171:6833","nonce":3906453887}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24655":{"gid":24655,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6837/2889350064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6836","nonce":2889350064},{"type":"v1","addr":"172.21.15.171:6837","nonce":2889350064}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24676":{"gid":24676,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.171:6839/3372214298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6838","nonce":3372214298},{"type":"v1","addr":"172.21.15.171:6839","nonce":3372214298}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34607":{"gid":34607,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":29,"addr":"172.21.15.154:6833/2781539957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6832","nonce":2781539957},{"type":"v1","addr":"172.21.15.154:6833","nonce":2781539957}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34640":{"gid":34640,"name":"e","rank":3,"incarnation":24,"state":"up:active","state_seq":25,"addr":"172.21.15.154:6835/622088003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6834","nonce":622088003},{"type":"v1","addr":"172.21.15.154:6835","nonce":622088003}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34646":{"gid":34646,"name":"f","rank":2,"incarnation":22,"state":"up:active","state_seq":24,"addr":"172.21.15.171:6835/2969925217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6834","nonce":2969925217},{"type":"v1","addr":"172.21.15.171:6835","nonce":2969925217}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34670":{"gid":34670,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":22,"addr":"172.21.15.154:6837/3907253955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6836","nonce":3907253955},{"type":"v1","addr":"172.21.15.154:6837","nonce":3907253955}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34703":{"gid":34703,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.154:6839/703390433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6838","nonce":703390433},{"type":"v1","addr":"172.21.15.154:6839","nonce":703390433}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":34703,"qdb_cluster":[34607,34646,34640,34670,34703]},"id":1}]} 2024-09-10T22:30:43.533 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 103 2024-09-10T22:30:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.532+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2e540b54d0 msgr2=0x7f2e540b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.532+0000 7f2e76600640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2e540b54d0 0x7f2e540b7990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2e701071c0 tx=0x7f2e5c005f90 comp rx=0 tx=0).stop 2024-09-10T22:30:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e70104fa0 msgr2=0x7f2e7019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e70104fa0 0x7f2e7019b360 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000e9f0 tx=0x7f2e6000eeb0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 shutdown_connections 2024-09-10T22:30:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f2e540b54d0 0x7f2e540b7990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f2e7010b330 0x7f2e701a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f2e7010a9b0 0x7f2e7019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 --2- 172.21.15.90:0/4234713648 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2e70104fa0 0x7f2e7019b360 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 >> 172.21.15.90:0/4234713648 conn(0x7f2e70100a50 msgr2=0x7f2e70102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 shutdown_connections 2024-09-10T22:30:43.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:43.533+0000 7f2e76600640 1 -- 172.21.15.90:0/4234713648 wait complete. 2024-09-10T22:30:44.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:43 smithi090 ceph-mon[30350]: pgmap v1264: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T22:30:44.037 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:43 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/4234713648' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:30:44.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:43 smithi171 ceph-mon[34138]: pgmap v1264: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T22:30:44.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:43 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/4234713648' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:30:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:43 smithi154 ceph-mon[34495]: pgmap v1264: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T22:30:44.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:43 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/4234713648' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T22:30:44.512 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T22:30:44.890 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:45.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.590+0000 7fa5d3600640 1 -- 172.21.15.90:0/3797722359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc10f3e0 msgr2=0x7fa5cc1117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:45.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.590+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3797722359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc10f3e0 0x7fa5cc1117d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4009ca0 tx=0x7fa5c402a8c0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.590+0000 7fa5d3600640 1 -- 172.21.15.90:0/3797722359 shutdown_connections 2024-09-10T22:30:45.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.590+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3797722359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc10f3e0 0x7fa5cc1117d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.590+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3797722359 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5cc075f40 0x7fa5cc10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.590+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3797722359 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5cc075540 0x7fa5cc075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.590+0000 7fa5d3600640 1 -- 172.21.15.90:0/3797722359 >> 172.21.15.90:0/3797722359 conn(0x7fa5cc0fe830 msgr2=0x7fa5cc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:45.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.590+0000 7fa5d3600640 1 -- 172.21.15.90:0/3797722359 shutdown_connections 2024-09-10T22:30:45.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.590+0000 7fa5d3600640 1 -- 172.21.15.90:0/3797722359 wait complete. 2024-09-10T22:30:45.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.591+0000 7fa5d3600640 1 Processor -- start 2024-09-10T22:30:45.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.591+0000 7fa5d3600640 1 -- start start 2024-09-10T22:30:45.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d3600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc075540 0x7fa5cc19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:45.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d3600640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5cc075f40 0x7fa5cc19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:45.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d3600640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5cc10f3e0 0x7fa5cc1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:45.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d3600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc114e80 con 0x7fa5cc075540 2024-09-10T22:30:45.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d3600640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc114d00 con 0x7fa5cc10f3e0 2024-09-10T22:30:45.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d3600640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc115000 con 0x7fa5cc075f40 2024-09-10T22:30:45.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d0c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc075540 0x7fa5cc19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:45.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d0c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc075540 0x7fa5cc19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40798/0 (socket says 172.21.15.90:40798) 2024-09-10T22:30:45.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d0c00640 1 -- 172.21.15.90:0/3033665721 learned_addr learned my addr 172.21.15.90:0/3033665721 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:45.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5cbe00640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5cc075f40 0x7fa5cc19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:45.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d0c00640 1 -- 172.21.15.90:0/3033665721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5cc075f40 msgr2=0x7fa5cc19b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:45.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d0c00640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5cc075f40 0x7fa5cc19b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d0c00640 1 -- 172.21.15.90:0/3033665721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5cc10f3e0 msgr2=0x7fa5cc1a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:45.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d0c00640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5cc10f3e0 0x7fa5cc1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.592+0000 7fa5d0c00640 1 -- 172.21.15.90:0/3033665721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc1a14a0 con 0x7fa5cc075540 2024-09-10T22:30:45.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.593+0000 7fa5d0c00640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc075540 0x7fa5cc19b3f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e9f0 tx=0x7fa5bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:45.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.593+0000 7fa5c9600640 1 -- 172.21.15.90:0/3033665721 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00cd80 con 0x7fa5cc075540 2024-09-10T22:30:45.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.593+0000 7fa5c9600640 1 -- 172.21.15.90:0/3033665721 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa5bc0108c0 con 0x7fa5cc075540 2024-09-10T22:30:45.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.593+0000 7fa5c9600640 1 -- 172.21.15.90:0/3033665721 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0187c0 con 0x7fa5cc075540 2024-09-10T22:30:45.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.593+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc1a1790 con 0x7fa5cc075540 2024-09-10T22:30:45.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.593+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc1a7e50 con 0x7fa5cc075540 2024-09-10T22:30:45.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.594+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc075920 con 0x7fa5cc075540 2024-09-10T22:30:45.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.596+0000 7fa5c9600640 1 -- 172.21.15.90:0/3033665721 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fa5bc010430 con 0x7fa5cc075540 2024-09-10T22:30:45.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.598+0000 7fa5c9600640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa5b00b54d0 0x7fa5b00b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:45.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.598+0000 7fa5c9600640 1 -- 172.21.15.90:0/3033665721 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9490+0+0 (secure 0 0 0) 0x7fa5bc0e67a0 con 0x7fa5cc075540 2024-09-10T22:30:45.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.598+0000 7fa5cbe00640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa5b00b54d0 0x7fa5b00b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:45.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.599+0000 7fa5cbe00640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa5b00b54d0 0x7fa5b00b7990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc111a70 tx=0x7fa5b8005f90 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:45.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.602+0000 7fa5c9600640 1 -- 172.21.15.90:0/3033665721 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa5bc0acef0 con 0x7fa5cc075540 2024-09-10T22:30:45.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.787+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5cc02e5c0 con 0x7fa5cc075540 2024-09-10T22:30:45.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.788+0000 7fa5c9600640 1 -- 172.21.15.90:0/3033665721 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+25415 (secure 0 0 0) 0x7fa5bc0ac7f0 con 0x7fa5cc075540 2024-09-10T22:30:45.790 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-09-10T22:30:45.790 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":132,"fsid":"815bde3e-6fbe-11ef-bcea-c7b262605968","created":"2024-09-10T21:50:11.271441+0000","modified":"2024-09-10T22:30:20.898518+0000","last_up_change":"2024-09-10T21:59:29.223379+0000","last_in_change":"2024-09-10T21:59:03.888523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:54:47.794295+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T22:00:28.928263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T22:00:32.528935+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":49,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T22:00:41.695408+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":49,"snap_epoch":131,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"e7cc8866-7bb1-4da4-ba54-9492e1ccb1aa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6803","nonce":405981593}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6805","nonce":405981593}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6809","nonce":405981593}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":405981593},{"type":"v1","addr":"172.21.15.90:6807","nonce":405981593}]},"public_addr":"172.21.15.90:6803/405981593","cluster_addr":"172.21.15.90:6805/405981593","heartbeat_back_addr":"172.21.15.90:6809/405981593","heartbeat_front_addr":"172.21.15.90:6807/405981593","state":["exists","up"]},{"osd":1,"uuid":"417251ca-442c-4c88-881d-c28668067397","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6801","nonce":2687850784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6803","nonce":2687850784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6807","nonce":2687850784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":2687850784},{"type":"v1","addr":"172.21.15.154:6805","nonce":2687850784}]},"public_addr":"172.21.15.154:6801/2687850784","cluster_addr":"172.21.15.154:6803/2687850784","heartbeat_back_addr":"172.21.15.154:6807/2687850784","heartbeat_front_addr":"172.21.15.154:6805/2687850784","state":["exists","up"]},{"osd":2,"uuid":"529eeb1a-fe6a-4077-b8a5-97246ab90ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6800","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6801","nonce":862625037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6802","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6803","nonce":862625037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6806","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6807","nonce":862625037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6804","nonce":862625037},{"type":"v1","addr":"172.21.15.171:6805","nonce":862625037}]},"public_addr":"172.21.15.171:6801/862625037","cluster_addr":"172.21.15.171:6803/862625037","heartbeat_back_addr":"172.21.15.171:6807/862625037","heartbeat_front_addr":"172.21.15.171:6805/862625037","state":["exists","up"]},{"osd":3,"uuid":"5d6629c1-3c41-462a-84ce-21a906d40bb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6811","nonce":995208383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6813","nonce":995208383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6817","nonce":995208383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":995208383},{"type":"v1","addr":"172.21.15.90:6815","nonce":995208383}]},"public_addr":"172.21.15.90:6811/995208383","cluster_addr":"172.21.15.90:6813/995208383","heartbeat_back_addr":"172.21.15.90:6817/995208383","heartbeat_front_addr":"172.21.15.90:6815/995208383","state":["exists","up"]},{"osd":4,"uuid":"99cff575-9f60-486c-b99a-48d5c3cf531c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6809","nonce":1188268169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6811","nonce":1188268169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6815","nonce":1188268169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":1188268169},{"type":"v1","addr":"172.21.15.154:6813","nonce":1188268169}]},"public_addr":"172.21.15.154:6809/1188268169","cluster_addr":"172.21.15.154:6811/1188268169","heartbeat_back_addr":"172.21.15.154:6815/1188268169","heartbeat_front_addr":"172.21.15.154:6813/1188268169","state":["exists","up"]},{"osd":5,"uuid":"f7931a53-09a3-408d-91c4-b85dd2fec784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6808","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6809","nonce":2109988699}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6810","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6811","nonce":2109988699}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6814","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6815","nonce":2109988699}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6812","nonce":2109988699},{"type":"v1","addr":"172.21.15.171:6813","nonce":2109988699}]},"public_addr":"172.21.15.171:6809/2109988699","cluster_addr":"172.21.15.171:6811/2109988699","heartbeat_back_addr":"172.21.15.171:6815/2109988699","heartbeat_front_addr":"172.21.15.171:6813/2109988699","state":["exists","up"]},{"osd":6,"uuid":"2bdf9899-a84f-42fd-a62f-6c19c39f8ba4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6819","nonce":3973504003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6821","nonce":3973504003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6825","nonce":3973504003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":3973504003},{"type":"v1","addr":"172.21.15.90:6823","nonce":3973504003}]},"public_addr":"172.21.15.90:6819/3973504003","cluster_addr":"172.21.15.90:6821/3973504003","heartbeat_back_addr":"172.21.15.90:6825/3973504003","heartbeat_front_addr":"172.21.15.90:6823/3973504003","state":["exists","up"]},{"osd":7,"uuid":"627062fc-f4ab-4e64-a68a-33beb4664f3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6817","nonce":1617520219}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6818","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6819","nonce":1617520219}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6822","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6823","nonce":1617520219}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6820","nonce":1617520219},{"type":"v1","addr":"172.21.15.154:6821","nonce":1617520219}]},"public_addr":"172.21.15.154:6817/1617520219","cluster_addr":"172.21.15.154:6819/1617520219","heartbeat_back_addr":"172.21.15.154:6823/1617520219","heartbeat_front_addr":"172.21.15.154:6821/1617520219","state":["exists","up"]},{"osd":8,"uuid":"6ec3145c-deaf-467c-800c-257e339ebde5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6816","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6817","nonce":4137047501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6818","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6819","nonce":4137047501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6822","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6823","nonce":4137047501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6820","nonce":4137047501},{"type":"v1","addr":"172.21.15.171:6821","nonce":4137047501}]},"public_addr":"172.21.15.171:6817/4137047501","cluster_addr":"172.21.15.171:6819/4137047501","heartbeat_back_addr":"172.21.15.171:6823/4137047501","heartbeat_front_addr":"172.21.15.171:6821/4137047501","state":["exists","up"]},{"osd":9,"uuid":"7e58c2f4-87df-4021-98ff-fd65c0a4fbcf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6827","nonce":1146146212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6829","nonce":1146146212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6833","nonce":1146146212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1146146212},{"type":"v1","addr":"172.21.15.90:6831","nonce":1146146212}]},"public_addr":"172.21.15.90:6827/1146146212","cluster_addr":"172.21.15.90:6829/1146146212","heartbeat_back_addr":"172.21.15.90:6833/1146146212","heartbeat_front_addr":"172.21.15.90:6831/1146146212","state":["exists","up"]},{"osd":10,"uuid":"e57aa10e-348d-4de3-be31-927241f583e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6824","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6825","nonce":1803626214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6826","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6827","nonce":1803626214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6830","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6831","nonce":1803626214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6828","nonce":1803626214},{"type":"v1","addr":"172.21.15.154:6829","nonce":1803626214}]},"public_addr":"172.21.15.154:6825/1803626214","cluster_addr":"172.21.15.154:6827/1803626214","heartbeat_back_addr":"172.21.15.154:6831/1803626214","heartbeat_front_addr":"172.21.15.154:6829/1803626214","state":["exists","up"]},{"osd":11,"uuid":"599b64e5-d2f1-42c7-9532-67156d42d99a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6824","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6825","nonce":1226533587}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6826","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6827","nonce":1226533587}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6830","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6831","nonce":1226533587}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.171:6828","nonce":1226533587},{"type":"v1","addr":"172.21.15.171:6829","nonce":1226533587}]},"public_addr":"172.21.15.171:6825/1226533587","cluster_addr":"172.21.15.171:6827/1226533587","heartbeat_back_addr":"172.21.15.171:6831/1226533587","heartbeat_front_addr":"172.21.15.171:6829/1226533587","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:53:46.249758+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:16.782094+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:54:46.004547+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:18.687226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:55:49.935434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:20.496784+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:56:52.492605+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:23.332169+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:57:54.579750+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:25.398185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:58:56.925986+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:59:27.582032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":2,"to":1},{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/2204308252":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1572343768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6800/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/1385951279":"2024-09-11T21:50:49.799954+0000","172.21.15.90:6800/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:6801/1613780567":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/3858975894":"2024-09-11T21:50:49.799954+0000","172.21.15.90:0/2255342982":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/3643795744":"2024-09-11T21:51:17.626867+0000","172.21.15.90:0/2773946294":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/2694078227":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6801/323032163":"2024-09-11T21:51:17.626867+0000","172.21.15.90:6800/3233554768":"2024-09-11T21:52:33.477408+0000","172.21.15.90:0/3503416190":"2024-09-11T21:52:33.477408+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":36,"length":1},{"begin":49,"length":1}]},{"pool":4,"snaps":[{"begin":36,"length":1},{"begin":49,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T22:30:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa5b00b54d0 msgr2=0x7fa5b00b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa5b00b54d0 0x7fa5b00b7990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc111a70 tx=0x7fa5b8005f90 comp rx=0 tx=0).stop 2024-09-10T22:30:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc075540 msgr2=0x7fa5cc19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc075540 0x7fa5cc19b3f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e9f0 tx=0x7fa5bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 shutdown_connections 2024-09-10T22:30:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fa5b00b54d0 0x7fa5b00b7990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fa5cc10f3e0 0x7fa5cc1a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fa5cc075f40 0x7fa5cc19b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 --2- 172.21.15.90:0/3033665721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5cc075540 0x7fa5cc19b3f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 >> 172.21.15.90:0/3033665721 conn(0x7fa5cc0fe830 msgr2=0x7fa5cc0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 shutdown_connections 2024-09-10T22:30:45.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:45.791+0000 7fa5d3600640 1 -- 172.21.15.90:0/3033665721 wait complete. 2024-09-10T22:30:46.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:45 smithi171 ceph-mon[34138]: pgmap v1265: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T22:30:46.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:45 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/3033665721' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:30:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:45 smithi090 ceph-mon[30350]: pgmap v1265: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T22:30:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:45 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/3033665721' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:30:46.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:45 smithi154 ceph-mon[34495]: pgmap v1265: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T22:30:46.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:45 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/3033665721' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T22:30:46.807 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T22:30:47.176 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:47.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.818+0000 7f462fe00640 1 -- 172.21.15.90:0/3392165325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4630108610 msgr2=0x7f4630108a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:47.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.818+0000 7f462fe00640 1 --2- 172.21.15.90:0/3392165325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4630108610 0x7f4630108a50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f461c009900 tx=0x7f461c02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:30:47.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.818+0000 7f462fe00640 1 -- 172.21.15.90:0/3392165325 shutdown_connections 2024-09-10T22:30:47.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.818+0000 7f462fe00640 1 --2- 172.21.15.90:0/3392165325 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4630108f90 0x7f463010f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:47.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.818+0000 7f462fe00640 1 --2- 172.21.15.90:0/3392165325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4630108610 0x7f4630108a50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:47.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.818+0000 7f462fe00640 1 --2- 172.21.15.90:0/3392165325 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4630102c00 0x7f4630102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:47.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.818+0000 7f462fe00640 1 -- 172.21.15.90:0/3392165325 >> 172.21.15.90:0/3392165325 conn(0x7f46300fe6d0 msgr2=0x7f4630100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:47.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.818+0000 7f462fe00640 1 -- 172.21.15.90:0/3392165325 shutdown_connections 2024-09-10T22:30:47.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.818+0000 7f462fe00640 1 -- 172.21.15.90:0/3392165325 wait complete. 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.819+0000 7f462fe00640 1 Processor -- start 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.819+0000 7f462fe00640 1 -- start start 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462fe00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4630102c00 0x7f463019b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462fe00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4630108610 0x7f463019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4630108f90 0x7f46301a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4630112a90 con 0x7f4630108f90 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462fe00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7f4630112910 con 0x7f4630108610 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462fe00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7f4630112c10 con 0x7f4630102c00 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4630102c00 0x7f463019b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4630108f90 0x7f46301a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462ea00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4630102c00 0x7f463019b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.171:3300/0 says I am v2:172.21.15.90:40002/0 (socket says 172.21.15.90:40002) 2024-09-10T22:30:47.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462ea00640 1 -- 172.21.15.90:0/99229519 learned_addr learned my addr 172.21.15.90:0/99229519 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:47.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462e000640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4630108610 0x7f463019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462ea00640 1 -- 172.21.15.90:0/99229519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4630108610 msgr2=0x7f463019b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462ea00640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4630108610 0x7f463019b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462ea00640 1 -- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4630108f90 msgr2=0x7f46301a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462ea00640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4630108f90 0x7f46301a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462ea00640 1 -- 172.21.15.90:0/99229519 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46301a12d0 con 0x7f4630102c00 2024-09-10T22:30:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.820+0000 7f462f400640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4630108f90 0x7f46301a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:30:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.821+0000 7f462ea00640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4630102c00 0x7f463019b220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f462000ea60 tx=0x7f462000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.821+0000 7f4627400640 1 -- 172.21.15.90:0/99229519 <== mon.1 v2:172.21.15.171:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462000cde0 con 0x7f4630102c00 2024-09-10T22:30:47.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.821+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46301a15c0 con 0x7f4630102c00 2024-09-10T22:30:47.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.821+0000 7f4627400640 1 -- 172.21.15.90:0/99229519 <== mon.1 v2:172.21.15.171:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f46200035f0 con 0x7f4630102c00 2024-09-10T22:30:47.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.821+0000 7f4627400640 1 -- 172.21.15.90:0/99229519 <== mon.1 v2:172.21.15.171:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620010690 con 0x7f4630102c00 2024-09-10T22:30:47.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.821+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46301a7e50 con 0x7f4630102c00 2024-09-10T22:30:47.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.822+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4630103eb0 con 0x7f4630102c00 2024-09-10T22:30:47.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.825+0000 7f4627400640 1 -- 172.21.15.90:0/99229519 <== mon.1 v2:172.21.15.171:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7f4620004000 con 0x7f4630102c00 2024-09-10T22:30:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.826+0000 7f4627400640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f46140b5500 0x7f46140b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.826+0000 7f4627400640 1 -- 172.21.15.90:0/99229519 <== mon.1 v2:172.21.15.171:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9490+0+0 (secure 0 0 0) 0x7f46200e6950 con 0x7f4630102c00 2024-09-10T22:30:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.826+0000 7f462e000640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f46140b5500 0x7f46140b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:47.827 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.827+0000 7f462e000640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f46140b5500 0x7f46140b79c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c005a20 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:47.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:47.828+0000 7f4627400640 1 -- 172.21.15.90:0/99229519 <== mon.1 v2:172.21.15.171:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4620056220 con 0x7f4630102c00 2024-09-10T22:30:48.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.051+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f463010f1e0 con 0x7f4630102c00 2024-09-10T22:30:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.065+0000 7f4627400640 1 -- 172.21.15.90:0/99229519 <== mon.1 v2:172.21.15.171:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) ==== 161+0+0 (secure 0 0 0) 0x7f46200ad0a0 con 0x7f4630102c00 2024-09-10T22:30:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f46140b5500 msgr2=0x7f46140b79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f46140b5500 0x7f46140b79c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c005a20 comp rx=0 tx=0).stop 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4630102c00 msgr2=0x7f463019b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4630102c00 0x7f463019b220 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f462000ea60 tx=0x7f462000ef20 comp rx=0 tx=0).stop 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 shutdown_connections 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7f46140b5500 0x7f46140b79c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4630108f90 0x7f46301a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7f4630108610 0x7f463019b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 --2- 172.21.15.90:0/99229519 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7f4630102c00 0x7f463019b220 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 >> 172.21.15.90:0/99229519 conn(0x7f46300fe6d0 msgr2=0x7f4630105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 shutdown_connections 2024-09-10T22:30:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:48.068+0000 7f462fe00640 1 -- 172.21.15.90:0/99229519 wait complete. 2024-09-10T22:30:48.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:47 smithi090 ceph-mon[30350]: pgmap v1266: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T22:30:48.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a[30346]: 2024-09-10T22:30:48.053+0000 7f513aa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:30:48.250 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:47 smithi171 ceph-mon[34138]: pgmap v1266: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T22:30:48.250 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i[65488]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.251 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i[65488]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.251 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mds[65492]: -- 172.21.15.171:0/4273019919 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9b8dd5860 con 0x55d9b8db3400 2024-09-10T22:30:48.251 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i[65488]: starting mds.i at 2024-09-10T22:30:48.251 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i[65488]: 2024-09-10T22:30:48.139+0000 7f24db000640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-10T22:30:48.251 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c[65189]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.251 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c[65189]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.252 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mds[65193]: -- 172.21.15.171:0/2857003129 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56298ee9b860 con 0x56298ee79000 2024-09-10T22:30:48.252 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c[65189]: starting mds.c at 2024-09-10T22:30:48.252 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l[64579]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.252 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l[64579]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.252 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mds[64583]: -- 172.21.15.171:0/146184775 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55faa3c07c20 con 0x55faa3be5000 2024-09-10T22:30:48.252 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l[64579]: starting mds.l at 2024-09-10T22:30:48.252 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-f[64885]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.252 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-f[64885]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.253 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mds[64894]: -- 172.21.15.171:0/3061077347 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56101f101860 con 0x56101f0df400 2024-09-10T22:30:48.253 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-f[64885]: starting mds.f at 2024-09-10T22:30:48.357 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g[75428]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.357 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g[75428]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.357 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-mds[75436]: -- 172.21.15.90:0/2192150306 <== mon.2 v2:172.21.15.154:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5610c29c3860 con 0x5610c29a0800 2024-09-10T22:30:48.357 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g[75428]: starting mds.g at 2024-09-10T22:30:48.357 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-a[75842]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.357 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-a[75842]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.357 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-mds[75847]: -- 172.21.15.90:0/168949419 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d59cd25860 con 0x55d59cd02800 2024-09-10T22:30:48.357 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-a[75842]: starting mds.a at 2024-09-10T22:30:48.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:47 smithi154 ceph-mon[34495]: pgmap v1266: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T22:30:48.419 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h[65292]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.419 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h[65292]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.419 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mds[65296]: -- 172.21.15.154:0/3930528390 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560896c89860 con 0x560896c67400 2024-09-10T22:30:48.419 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h[65292]: starting mds.h at 2024-09-10T22:30:48.419 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b[65608]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.419 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b[65608]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.419 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mds[65612]: -- 172.21.15.154:0/3613544270 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593f050b860 con 0x5593f04e9400 2024-09-10T22:30:48.419 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b[65608]: starting mds.b at 2024-09-10T22:30:48.420 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e[64990]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.420 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e[64990]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.420 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mds[64994]: -- 172.21.15.154:0/207057481 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5556404ebc20 con 0x5556404c8800 2024-09-10T22:30:48.420 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e[64990]: starting mds.e at 2024-09-10T22:30:48.420 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k[64688]: ignoring --setuser ceph since I am not root 2024-09-10T22:30:48.420 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k[64688]: ignoring --setgroup ceph since I am not root 2024-09-10T22:30:48.420 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mds[64692]: -- 172.21.15.154:0/4113149540 <== mon.1 v2:172.21.15.171:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636dc047860 con 0x5636dc025400 2024-09-10T22:30:48.420 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k[64688]: starting mds.k at 2024-09-10T22:30:49.034 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T22:30:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/99229519' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T22:30:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mon[34138]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T22:30:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mon[34138]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T22:30:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mon[34138]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:30:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mon[34138]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T22:30:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mon[34138]: osdmap e133: 12 total, 12 up, 12 in 2024-09-10T22:30:49.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:48 smithi171 ceph-mon[34138]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T22:30:49.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/99229519' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T22:30:49.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T22:30:49.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-mon[30350]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T22:30:49.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-mon[30350]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:30:49.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T22:30:49.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-mon[30350]: osdmap e133: 12 total, 12 up, 12 in 2024-09-10T22:30:49.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:48 smithi090 ceph-mon[30350]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T22:30:49.408 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/99229519' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T22:30:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mon[34495]: from='client.? ' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T22:30:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mon[34495]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T22:30:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mon[34495]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T22:30:49.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mon[34495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T22:30:49.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mon[34495]: osdmap e133: 12 total, 12 up, 12 in 2024-09-10T22:30:49.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:48 smithi154 ceph-mon[34495]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: pgmap v1268: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.171:6833/2952733796,v1:172.21.15.171:6835/2952733796] up:boot 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.171:6832/476287336,v1:172.21.15.171:6834/476287336] up:boot 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.171:6836/1079481616,v1:172.21.15.171:6838/1079481616] up:boot 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.171:6837/1620523962,v1:172.21.15.171:6839/1620523962] up:boot 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6833/569304473,v1:172.21.15.154:6837/569304473] up:boot 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6832/731838476,v1:172.21.15.154:6836/731838476] up:boot 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6834/3910627399,v1:172.21.15.154:6838/3910627399] up:boot 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.154:6835/3539618602,v1:172.21.15.154:6839/3539618602] up:boot 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.90:6838/4104577207,v1:172.21.15.90:6841/4104577207] up:boot 2024-09-10T22:30:50.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: mds.? [v2:172.21.15.90:6839/2143735684,v1:172.21.15.90:6840/2143735684] up:boot 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:30:50.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:49 smithi090 ceph-mon[30350]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:30:50.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.060+0000 7fc029a00640 1 -- 172.21.15.90:0/1867593210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 msgr2=0x7fc02410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.060+0000 7fc029a00640 1 --2- 172.21.15.90:0/1867593210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 0x7fc02410f950 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009ca0 tx=0x7fc01c02a8c0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.060+0000 7fc029a00640 1 -- 172.21.15.90:0/1867593210 shutdown_connections 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.060+0000 7fc029a00640 1 --2- 172.21.15.90:0/1867593210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 0x7fc02410f950 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.060+0000 7fc029a00640 1 --2- 172.21.15.90:0/1867593210 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0241087d0 0x7fc024108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.060+0000 7fc029a00640 1 --2- 172.21.15.90:0/1867593210 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc024102dc0 0x7fc0241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.060+0000 7fc029a00640 1 -- 172.21.15.90:0/1867593210 >> 172.21.15.90:0/1867593210 conn(0x7fc0240fe830 msgr2=0x7fc024100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.060+0000 7fc029a00640 1 -- 172.21.15.90:0/1867593210 shutdown_connections 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.060+0000 7fc029a00640 1 -- 172.21.15.90:0/1867593210 wait complete. 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.061+0000 7fc029a00640 1 Processor -- start 2024-09-10T22:30:50.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.062+0000 7fc029a00640 1 -- start start 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.062+0000 7fc029a00640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc024102dc0 0x7fc02419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.062+0000 7fc029a00640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0241087d0 0x7fc02419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.062+0000 7fc029a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 0x7fc0241a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.062+0000 7fc029a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc024112d10 con 0x7fc024109150 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.062+0000 7fc029a00640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fc024112b90 con 0x7fc0241087d0 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.062+0000 7fc029a00640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fc024112e90 con 0x7fc024102dc0 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc023400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 0x7fc0241a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc023400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 0x7fc0241a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52992/0 (socket says 172.21.15.90:52992) 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc023400640 1 -- 172.21.15.90:0/1417544748 learned_addr learned my addr 172.21.15.90:0/1417544748 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:50.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc022000640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0241087d0 0x7fc02419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc022a00640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc024102dc0 0x7fc02419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc023400640 1 -- 172.21.15.90:0/1417544748 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc024102dc0 msgr2=0x7fc02419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc023400640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc024102dc0 0x7fc02419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc023400640 1 -- 172.21.15.90:0/1417544748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0241087d0 msgr2=0x7fc02419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc023400640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0241087d0 0x7fc02419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc023400640 1 -- 172.21.15.90:0/1417544748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0241a1460 con 0x7fc024109150 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.063+0000 7fc023400640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 0x7fc0241a0c90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc01c005cc0 tx=0x7fc01c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.064+0000 7fc01b400640 1 -- 172.21.15.90:0/1417544748 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c032e00 con 0x7fc024109150 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.064+0000 7fc01b400640 1 -- 172.21.15.90:0/1417544748 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc01c0044d0 con 0x7fc024109150 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.064+0000 7fc01b400640 1 -- 172.21.15.90:0/1417544748 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c030b70 con 0x7fc024109150 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.064+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0241a16f0 con 0x7fc024109150 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.064+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0241a7e10 con 0x7fc024109150 2024-09-10T22:30:50.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.065+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0241031a0 con 0x7fc024109150 2024-09-10T22:30:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.067+0000 7fc01b400640 1 -- 172.21.15.90:0/1417544748 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fc01c004680 con 0x7fc024109150 2024-09-10T22:30:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.068+0000 7fc01b400640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc0080b5450 0x7fc0080b7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.069+0000 7fc01b400640 1 -- 172.21.15.90:0/1417544748 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9832+0+0 (secure 0 0 0) 0x7fc01c1078c0 con 0x7fc024109150 2024-09-10T22:30:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.069+0000 7fc022a00640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc0080b5450 0x7fc0080b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.069+0000 7fc022a00640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc0080b5450 0x7fc0080b7910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc014005fd0 tx=0x7fc01400b390 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.072+0000 7fc01b400640 1 -- 172.21.15.90:0/1417544748 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc01c0cdec0 con 0x7fc024109150 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: pgmap v1268: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.171:6833/2952733796,v1:172.21.15.171:6835/2952733796] up:boot 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.171:6832/476287336,v1:172.21.15.171:6834/476287336] up:boot 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.171:6836/1079481616,v1:172.21.15.171:6838/1079481616] up:boot 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.171:6837/1620523962,v1:172.21.15.171:6839/1620523962] up:boot 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6833/569304473,v1:172.21.15.154:6837/569304473] up:boot 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6832/731838476,v1:172.21.15.154:6836/731838476] up:boot 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6834/3910627399,v1:172.21.15.154:6838/3910627399] up:boot 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.154:6835/3539618602,v1:172.21.15.154:6839/3539618602] up:boot 2024-09-10T22:30:50.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.90:6838/4104577207,v1:172.21.15.90:6841/4104577207] up:boot 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: mds.? [v2:172.21.15.90:6839/2143735684,v1:172.21.15.90:6840/2143735684] up:boot 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:30:50.249 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:49 smithi171 ceph-mon[34138]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:30:50.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.295+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc02402e5c0 con 0x7fc024109150 2024-09-10T22:30:50.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: pgmap v1268: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 102 B/s wr, 10 op/s 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.171:6833/2952733796,v1:172.21.15.171:6835/2952733796] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.171:6832/476287336,v1:172.21.15.171:6834/476287336] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.171:6836/1079481616,v1:172.21.15.171:6838/1079481616] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.171:6837/1620523962,v1:172.21.15.171:6839/1620523962] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6833/569304473,v1:172.21.15.154:6837/569304473] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6832/731838476,v1:172.21.15.154:6836/731838476] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6834/3910627399,v1:172.21.15.154:6838/3910627399] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.154:6835/3539618602,v1:172.21.15.154:6839/3539618602] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.90:6838/4104577207,v1:172.21.15.90:6841/4104577207] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: mds.? [v2:172.21.15.90:6839/2143735684,v1:172.21.15.90:6840/2143735684] up:boot 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T22:30:50.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T22:30:50.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T22:30:50.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T22:30:50.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T22:30:50.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T22:30:50.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T22:30:50.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T22:30:50.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T22:30:50.420 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:49 smithi154 ceph-mon[34495]: from='mgr.14223 172.21.15.90:0/1939519089' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T22:30:50.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.965+0000 7fc01b400640 1 -- 172.21.15.90:0/1417544748 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v106) ==== 104+0+0 (secure 0 0 0) 0x7fc01c0cd7c0 con 0x7fc024109150 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc0080b5450 msgr2=0x7fc0080b7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc0080b5450 0x7fc0080b7910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc014005fd0 tx=0x7fc01400b390 comp rx=0 tx=0).stop 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 msgr2=0x7fc0241a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 0x7fc0241a0c90 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc01c005cc0 tx=0x7fc01c032a10 comp rx=0 tx=0).stop 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 shutdown_connections 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fc0080b5450 0x7fc0080b7910 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc024109150 0x7fc0241a0c90 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fc0241087d0 0x7fc02419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 --2- 172.21.15.90:0/1417544748 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fc024102dc0 0x7fc02419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 >> 172.21.15.90:0/1417544748 conn(0x7fc0240fe830 msgr2=0x7fc024104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 shutdown_connections 2024-09-10T22:30:50.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:50.968+0000 7fc029a00640 1 -- 172.21.15.90:0/1417544748 wait complete. 2024-09-10T22:30:51.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:50 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1417544748' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:30:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:50 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1417544748' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:30:51.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:50 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1417544748' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T22:30:51.847 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T22:30:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:51 smithi090 ceph-mon[30350]: pgmap v1269: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:30:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:51 smithi090 ceph-mon[30350]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T22:30:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:51 smithi090 ceph-mon[30350]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T22:30:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:51 smithi090 ceph-mon[30350]: Cluster is now healthy 2024-09-10T22:30:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:51 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/1417544748' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:30:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:51 smithi090 ceph-mon[30350]: fsmap 12 up:standby 2024-09-10T22:30:52.218 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:51 smithi171 ceph-mon[34138]: pgmap v1269: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:30:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:51 smithi171 ceph-mon[34138]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T22:30:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:51 smithi171 ceph-mon[34138]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T22:30:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:51 smithi171 ceph-mon[34138]: Cluster is now healthy 2024-09-10T22:30:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:51 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/1417544748' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:30:52.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:51 smithi171 ceph-mon[34138]: fsmap 12 up:standby 2024-09-10T22:30:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:51 smithi154 ceph-mon[34495]: pgmap v1269: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:30:52.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:51 smithi154 ceph-mon[34495]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T22:30:52.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:51 smithi154 ceph-mon[34495]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T22:30:52.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:51 smithi154 ceph-mon[34495]: Cluster is now healthy 2024-09-10T22:30:52.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:51 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/1417544748' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T22:30:52.419 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:51 smithi154 ceph-mon[34495]: fsmap 12 up:standby 2024-09-10T22:30:52.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.765+0000 7fb211800640 1 -- 172.21.15.90:0/2500143251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c102c00 msgr2=0x7fb20c102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:52.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.765+0000 7fb211800640 1 --2- 172.21.15.90:0/2500143251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c102c00 0x7fb20c102fe0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009900 tx=0x7fb1fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.765+0000 7fb211800640 1 -- 172.21.15.90:0/2500143251 shutdown_connections 2024-09-10T22:30:52.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.765+0000 7fb211800640 1 --2- 172.21.15.90:0/2500143251 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c108f90 0x7fb20c10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.765+0000 7fb211800640 1 --2- 172.21.15.90:0/2500143251 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb20c108610 0x7fb20c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.765+0000 7fb211800640 1 --2- 172.21.15.90:0/2500143251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c102c00 0x7fb20c102fe0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.765+0000 7fb211800640 1 -- 172.21.15.90:0/2500143251 >> 172.21.15.90:0/2500143251 conn(0x7fb20c0fe6d0 msgr2=0x7fb20c100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:52.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.765+0000 7fb211800640 1 -- 172.21.15.90:0/2500143251 shutdown_connections 2024-09-10T22:30:52.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.765+0000 7fb211800640 1 -- 172.21.15.90:0/2500143251 wait complete. 2024-09-10T22:30:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.766+0000 7fb211800640 1 Processor -- start 2024-09-10T22:30:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.766+0000 7fb211800640 1 -- start start 2024-09-10T22:30:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb211800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb20c102c00 0x7fb20c19b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb211800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c108610 0x7fb20c19b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb211800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c108f90 0x7fb20c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb211800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c112b50 con 0x7fb20c108610 2024-09-10T22:30:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb211800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1129d0 con 0x7fb20c108f90 2024-09-10T22:30:52.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb211800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c112cd0 con 0x7fb20c102c00 2024-09-10T22:30:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb203e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c108610 0x7fb20c19b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb203e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c108610 0x7fb20c19b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53006/0 (socket says 172.21.15.90:53006) 2024-09-10T22:30:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb203e00640 1 -- 172.21.15.90:0/2468617521 learned_addr learned my addr 172.21.15.90:0/2468617521 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb20be00640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb20c102c00 0x7fb20c19b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb210e00640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c108f90 0x7fb20c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.767+0000 7fb203e00640 1 -- 172.21.15.90:0/2468617521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb20c102c00 msgr2=0x7fb20c19b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb203e00640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb20c102c00 0x7fb20c19b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb203e00640 1 -- 172.21.15.90:0/2468617521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c108f90 msgr2=0x7fb20c1a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:52.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb203e00640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c108f90 0x7fb20c1a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb203e00640 1 -- 172.21.15.90:0/2468617521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c1a1190 con 0x7fb20c108610 2024-09-10T22:30:52.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb210e00640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c108f90 0x7fb20c1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:30:52.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb203e00640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c108610 0x7fb20c19b620 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400d770 tx=0x7fb1f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:52.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb209600640 1 -- 172.21.15.90:0/2468617521 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4017070 con 0x7fb20c108610 2024-09-10T22:30:52.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb209600640 1 -- 172.21.15.90:0/2468617521 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb1f40054b0 con 0x7fb20c108610 2024-09-10T22:30:52.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb209600640 1 -- 172.21.15.90:0/2468617521 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4004c60 con 0x7fb20c108610 2024-09-10T22:30:52.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c1a1480 con 0x7fb20c108610 2024-09-10T22:30:52.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.768+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c1a7b40 con 0x7fb20c108610 2024-09-10T22:30:52.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.769+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c102fe0 con 0x7fb20c108610 2024-09-10T22:30:52.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.771+0000 7fb209600640 1 -- 172.21.15.90:0/2468617521 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb1f4004e00 con 0x7fb20c108610 2024-09-10T22:30:52.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.774+0000 7fb209600640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1f00b5580 0x7fb1f00b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:52.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.774+0000 7fb209600640 1 -- 172.21.15.90:0/2468617521 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9832+0+0 (secure 0 0 0) 0x7fb1f40e5f70 con 0x7fb20c108610 2024-09-10T22:30:52.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.774+0000 7fb20be00640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1f00b5580 0x7fb1f00b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:52.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.775+0000 7fb20be00640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1f00b5580 0x7fb1f00b7a40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006010 tx=0x7fb1fc005e80 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:52.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.777+0000 7fb209600640 1 -- 172.21.15.90:0/2468617521 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb1f40ac570 con 0x7fb20c108610 2024-09-10T22:30:52.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.959+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb20c02e5c0 con 0x7fb20c108610 2024-09-10T22:30:52.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.967+0000 7fb209600640 1 -- 172.21.15.90:0/2468617521 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v134) ==== 181+0+0 (secure 0 0 0) 0x7fb1f40abe70 con 0x7fb20c108610 2024-09-10T22:30:52.969 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' removed 2024-09-10T22:30:52.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1f00b5580 msgr2=0x7fb1f00b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:52.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1f00b5580 0x7fb1f00b7a40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006010 tx=0x7fb1fc005e80 comp rx=0 tx=0).stop 2024-09-10T22:30:52.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c108610 msgr2=0x7fb20c19b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:52.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c108610 0x7fb20c19b620 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400d770 tx=0x7fb1f400dc30 comp rx=0 tx=0).stop 2024-09-10T22:30:52.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 shutdown_connections 2024-09-10T22:30:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb1f00b5580 0x7fb1f00b7a40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb20c108f90 0x7fb20c1a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb20c108610 0x7fb20c19b620 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 --2- 172.21.15.90:0/2468617521 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb20c102c00 0x7fb20c19b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 >> 172.21.15.90:0/2468617521 conn(0x7fb20c0fe6d0 msgr2=0x7fb20c075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 shutdown_connections 2024-09-10T22:30:52.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:52.969+0000 7fb211800640 1 -- 172.21.15.90:0/2468617521 wait complete. 2024-09-10T22:30:53.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:52 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2468617521' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T22:30:53.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:52 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2468617521' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T22:30:53.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:52 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2468617521' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T22:30:53.914 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T22:30:54.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:53 smithi171 ceph-mon[34138]: pgmap v1270: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:30:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:53 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2468617521' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T22:30:54.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:53 smithi171 ceph-mon[34138]: osdmap e134: 12 total, 12 up, 12 in 2024-09-10T22:30:54.287 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:53 smithi090 ceph-mon[30350]: pgmap v1270: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:30:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:53 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2468617521' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T22:30:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:53 smithi090 ceph-mon[30350]: osdmap e134: 12 total, 12 up, 12 in 2024-09-10T22:30:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:53 smithi154 ceph-mon[34495]: pgmap v1270: 129 pgs: 129 active+clean; 1.4 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T22:30:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:53 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2468617521' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T22:30:54.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:53 smithi154 ceph-mon[34495]: osdmap e134: 12 total, 12 up, 12 in 2024-09-10T22:30:54.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.896+0000 7fcb1a800640 1 -- 172.21.15.90:0/1675963328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb14104df0 msgr2=0x7fcb141051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:54.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.896+0000 7fcb1a800640 1 --2- 172.21.15.90:0/1675963328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb14104df0 0x7fcb141051d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009900 tx=0x7fcb0802a410 comp rx=0 tx=0).stop 2024-09-10T22:30:54.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.896+0000 7fcb1a800640 1 -- 172.21.15.90:0/1675963328 shutdown_connections 2024-09-10T22:30:54.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.896+0000 7fcb1a800640 1 --2- 172.21.15.90:0/1675963328 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb1410b180 0x7fcb14111980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:54.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.896+0000 7fcb1a800640 1 --2- 172.21.15.90:0/1675963328 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb1410a800 0x7fcb1410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:54.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.896+0000 7fcb1a800640 1 --2- 172.21.15.90:0/1675963328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb14104df0 0x7fcb141051d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:54.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.896+0000 7fcb1a800640 1 -- 172.21.15.90:0/1675963328 >> 172.21.15.90:0/1675963328 conn(0x7fcb14100840 msgr2=0x7fcb14102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:54.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.896+0000 7fcb1a800640 1 -- 172.21.15.90:0/1675963328 shutdown_connections 2024-09-10T22:30:54.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.896+0000 7fcb1a800640 1 -- 172.21.15.90:0/1675963328 wait complete. 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.897+0000 7fcb1a800640 1 Processor -- start 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.897+0000 7fcb1a800640 1 -- start start 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb1a800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb14104df0 0x7fcb1419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb1a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1410a800 0x7fcb1419b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb1a800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb1410b180 0x7fcb141a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb1a800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14114cf0 con 0x7fcb1410a800 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb1a800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14114b70 con 0x7fcb1410b180 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb1a800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fcb14114e70 con 0x7fcb14104df0 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb18a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1410a800 0x7fcb1419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:54.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb18a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1410a800 0x7fcb1419b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53034/0 (socket says 172.21.15.90:53034) 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb18a00640 1 -- 172.21.15.90:0/582304527 learned_addr learned my addr 172.21.15.90:0/582304527 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb19400640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb14104df0 0x7fcb1419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb19e00640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb1410b180 0x7fcb141a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb18a00640 1 -- 172.21.15.90:0/582304527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb14104df0 msgr2=0x7fcb1419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb18a00640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb14104df0 0x7fcb1419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb18a00640 1 -- 172.21.15.90:0/582304527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb1410b180 msgr2=0x7fcb141a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb18a00640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb1410b180 0x7fcb141a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.898+0000 7fcb18a00640 1 -- 172.21.15.90:0/582304527 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb141a1270 con 0x7fcb1410a800 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.899+0000 7fcb18a00640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1410a800 0x7fcb1419b700 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000d950 tx=0x7fcb0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.899+0000 7fcb12000640 1 -- 172.21.15.90:0/582304527 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00017030 con 0x7fcb1410a800 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.899+0000 7fcb12000640 1 -- 172.21.15.90:0/582304527 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb00004020 con 0x7fcb1410a800 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.899+0000 7fcb12000640 1 -- 172.21.15.90:0/582304527 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb000055d0 con 0x7fcb1410a800 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.899+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb141a1560 con 0x7fcb1410a800 2024-09-10T22:30:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.899+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb141a7df0 con 0x7fcb1410a800 2024-09-10T22:30:54.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.900+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb141051d0 con 0x7fcb1410a800 2024-09-10T22:30:54.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.903+0000 7fcb12000640 1 -- 172.21.15.90:0/582304527 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fcb00009a30 con 0x7fcb1410a800 2024-09-10T22:30:54.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.904+0000 7fcb12000640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcafc0b5580 0x7fcafc0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:54.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.904+0000 7fcb12000640 1 -- 172.21.15.90:0/582304527 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 9299+0+0 (secure 0 0 0) 0x7fcb000e5c70 con 0x7fcb1410a800 2024-09-10T22:30:54.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.905+0000 7fcb19400640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcafc0b5580 0x7fcafc0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:54.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.905+0000 7fcb19400640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcafc0b5580 0x7fcafc0b7a40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcb080057c0 tx=0x7fcb080058d0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:54.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:54.908+0000 7fcb12000640 1 -- 172.21.15.90:0/582304527 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcb000ac480 con 0x7fcb1410a800 2024-09-10T22:30:55.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.094+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcb1410cfc0 con 0x7fcb1410a800 2024-09-10T22:30:55.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.972+0000 7fcb12000640 1 -- 172.21.15.90:0/582304527 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v135) ==== 169+0+0 (secure 0 0 0) 0x7fcb000abd80 con 0x7fcb1410a800 2024-09-10T22:30:55.975 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' removed 2024-09-10T22:30:55.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcafc0b5580 msgr2=0x7fcafc0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:55.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcafc0b5580 0x7fcafc0b7a40 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcb080057c0 tx=0x7fcb080058d0 comp rx=0 tx=0).stop 2024-09-10T22:30:55.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1410a800 msgr2=0x7fcb1419b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:55.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1410a800 0x7fcb1419b700 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000d950 tx=0x7fcb0000de10 comp rx=0 tx=0).stop 2024-09-10T22:30:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 shutdown_connections 2024-09-10T22:30:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fcafc0b5580 0x7fcafc0b7a40 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcb1410b180 0x7fcb141a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb1410a800 0x7fcb1419b700 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 --2- 172.21.15.90:0/582304527 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fcb14104df0 0x7fcb1419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.977+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 >> 172.21.15.90:0/582304527 conn(0x7fcb14100840 msgr2=0x7fcb14101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.978+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 shutdown_connections 2024-09-10T22:30:55.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:55.978+0000 7fcb1a800640 1 -- 172.21.15.90:0/582304527 wait complete. 2024-09-10T22:30:56.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:55 smithi171 ceph-mon[34138]: pgmap v1272: 97 pgs: 97 active+clean; 305 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:30:56.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:55 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/582304527' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T22:30:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:55 smithi090 ceph-mon[30350]: pgmap v1272: 97 pgs: 97 active+clean; 305 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:30:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:55 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/582304527' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T22:30:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:55 smithi154 ceph-mon[34495]: pgmap v1272: 97 pgs: 97 active+clean; 305 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:30:56.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:55 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/582304527' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T22:30:56.930 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T22:30:57.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:56 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/582304527' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T22:30:57.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:56 smithi171 ceph-mon[34138]: osdmap e135: 12 total, 12 up, 12 in 2024-09-10T22:30:57.300 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T22:30:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:56 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/582304527' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T22:30:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:56 smithi090 ceph-mon[30350]: osdmap e135: 12 total, 12 up, 12 in 2024-09-10T22:30:57.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:56 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/582304527' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T22:30:57.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:56 smithi154 ceph-mon[34495]: osdmap e135: 12 total, 12 up, 12 in 2024-09-10T22:30:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.022+0000 7fb675800640 1 -- 172.21.15.90:0/3402590928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010b330 msgr2=0x7fb670111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.022+0000 7fb675800640 1 --2- 172.21.15.90:0/3402590928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010b330 0x7fb670111b30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb668009900 tx=0x7fb66802a410 comp rx=0 tx=0).stop 2024-09-10T22:30:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.023+0000 7fb675800640 1 -- 172.21.15.90:0/3402590928 shutdown_connections 2024-09-10T22:30:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.023+0000 7fb675800640 1 --2- 172.21.15.90:0/3402590928 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010b330 0x7fb670111b30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.023+0000 7fb675800640 1 --2- 172.21.15.90:0/3402590928 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb67010a9b0 0x7fb67010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:58.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.023+0000 7fb675800640 1 --2- 172.21.15.90:0/3402590928 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb670104fa0 0x7fb670105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.023+0000 7fb675800640 1 -- 172.21.15.90:0/3402590928 >> 172.21.15.90:0/3402590928 conn(0x7fb670100a50 msgr2=0x7fb670102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.023+0000 7fb675800640 1 -- 172.21.15.90:0/3402590928 shutdown_connections 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.023+0000 7fb675800640 1 -- 172.21.15.90:0/3402590928 wait complete. 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.024+0000 7fb675800640 1 Processor -- start 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.024+0000 7fb675800640 1 -- start start 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb675800640 1 --2- >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb670104fa0 0x7fb67019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb675800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010a9b0 0x7fb67019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb675800640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb67010b330 0x7fb6701a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb675800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb670114e10 con 0x7fb67010a9b0 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb675800640 1 -- --> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] -- mon_getmap magic: 0 -- 0x7fb670114c90 con 0x7fb67010b330 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb675800640 1 -- --> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] -- mon_getmap magic: 0 -- 0x7fb670114f90 con 0x7fb670104fa0 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010a9b0 0x7fb67019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:58.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66e000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010a9b0 0x7fb67019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50742/0 (socket says 172.21.15.90:50742) 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66f400640 1 --2- >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb67010b330 0x7fb6701a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66e000640 1 -- 172.21.15.90:0/2539444818 learned_addr learned my addr 172.21.15.90:0/2539444818 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66ea00640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb670104fa0 0x7fb67019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66e000640 1 -- 172.21.15.90:0/2539444818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb670104fa0 msgr2=0x7fb67019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66e000640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb670104fa0 0x7fb67019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66e000640 1 -- 172.21.15.90:0/2539444818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb67010b330 msgr2=0x7fb6701a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66e000640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb67010b330 0x7fb6701a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66e000640 1 -- 172.21.15.90:0/2539444818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6701a1430 con 0x7fb67010a9b0 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66ea00640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb670104fa0 0x7fb67019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.025+0000 7fb66f400640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb67010b330 0x7fb6701a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.026+0000 7fb66e000640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010a9b0 0x7fb67019b8c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00ea60 tx=0x7fb65c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.026+0000 7fb667400640 1 -- 172.21.15.90:0/2539444818 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c00cde0 con 0x7fb67010a9b0 2024-09-10T22:30:58.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.026+0000 7fb667400640 1 -- 172.21.15.90:0/2539444818 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb65c0035f0 con 0x7fb67010a9b0 2024-09-10T22:30:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.026+0000 7fb667400640 1 -- 172.21.15.90:0/2539444818 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c010690 con 0x7fb67010a9b0 2024-09-10T22:30:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.026+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6701a1720 con 0x7fb67010a9b0 2024-09-10T22:30:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.026+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6701a7fb0 con 0x7fb67010a9b0 2024-09-10T22:30:58.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.027+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb670105380 con 0x7fb67010a9b0 2024-09-10T22:30:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.030+0000 7fb667400640 1 -- 172.21.15.90:0/2539444818 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 153381+0+0 (secure 0 0 0) 0x7fb65c004000 con 0x7fb67010a9b0 2024-09-10T22:30:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.031+0000 7fb667400640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6540b54d0 0x7fb6540b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T22:30:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.032+0000 7fb667400640 1 -- 172.21.15.90:0/2539444818 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 8711+0+0 (secure 0 0 0) 0x7fb65c0e63f0 con 0x7fb67010a9b0 2024-09-10T22:30:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.032+0000 7fb66ea00640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6540b54d0 0x7fb6540b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T22:30:58.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.032+0000 7fb66ea00640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6540b54d0 0x7fb6540b7990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb660005fd0 tx=0x7fb6600093e0 comp rx=0 tx=0).ready entity=mgr.14223 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T22:30:58.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.035+0000 7fb667400640 1 -- 172.21.15.90:0/2539444818 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb65c0ace50 con 0x7fb67010a9b0 2024-09-10T22:30:58.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:57 smithi154 ceph-mon[34495]: pgmap v1274: 33 pgs: 33 active+clean; 305 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:30:58.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:58.219+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb67010d170 con 0x7fb67010a9b0 2024-09-10T22:30:58.247 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:57 smithi171 ceph-mon[34138]: pgmap v1274: 33 pgs: 33 active+clean; 305 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:30:58.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:57 smithi090 ceph-mon[30350]: pgmap v1274: 33 pgs: 33 active+clean; 305 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:30:59.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.000+0000 7fb667400640 1 -- 172.21.15.90:0/2539444818 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v136) ==== 178+0+0 (secure 0 0 0) 0x7fb65c0ac750 con 0x7fb67010a9b0 2024-09-10T22:30:59.002 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data_ec' removed 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6540b54d0 msgr2=0x7fb6540b7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6540b54d0 0x7fb6540b7990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb660005fd0 tx=0x7fb6600093e0 comp rx=0 tx=0).stop 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010a9b0 msgr2=0x7fb67019b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010a9b0 0x7fb67019b8c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb65c00ea60 tx=0x7fb65c00ef20 comp rx=0 tx=0).stop 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 shutdown_connections 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:6800/3093492804,v1:172.21.15.90:6801/3093492804] conn(0x7fb6540b54d0 0x7fb6540b7990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fb67010b330 0x7fb6701a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb67010a9b0 0x7fb67019b8c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 --2- 172.21.15.90:0/2539444818 >> [v2:172.21.15.171:3300/0,v1:172.21.15.171:6789/0] conn(0x7fb670104fa0 0x7fb67019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T22:30:59.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.003+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 >> 172.21.15.90:0/2539444818 conn(0x7fb670100a50 msgr2=0x7fb670102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T22:30:59.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.004+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 shutdown_connections 2024-09-10T22:30:59.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-09-10T22:30:59.004+0000 7fb675800640 1 -- 172.21.15.90:0/2539444818 wait complete. 2024-09-10T22:30:59.248 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:30:58 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2539444818' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T22:30:59.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:30:58 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2539444818' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T22:30:59.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:30:58 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2539444818' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T22:30:59.878 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:30:59.935 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:31:00.016 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:31:00.082 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T22:31:00.082 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T22:31:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-mon[30350]: pgmap v1275: 33 pgs: 33 active+clean; 305 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:31:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-mon[30350]: from='client.? 172.21.15.90:0/2539444818' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T22:31:00.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-mon[30350]: osdmap e136: 12 total, 12 up, 12 in 2024-09-10T22:31:00.130 DEBUG:teuthology.orchestra.run.smithi154:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T22:31:00.166 DEBUG:teuthology.orchestra.run.smithi171:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T22:31:00.203 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T22:31:00.203 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T22:31:00.203 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.a 2024-09-10T22:31:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:00 smithi154 ceph-mon[34495]: pgmap v1275: 33 pgs: 33 active+clean; 305 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:31:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:00 smithi154 ceph-mon[34495]: from='client.? 172.21.15.90:0/2539444818' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T22:31:00.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:00 smithi154 ceph-mon[34495]: osdmap e136: 12 total, 12 up, 12 in 2024-09-10T22:31:00.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:00 smithi171 ceph-mon[34138]: pgmap v1275: 33 pgs: 33 active+clean; 305 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T22:31:00.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:00 smithi171 ceph-mon[34138]: from='client.? 172.21.15.90:0/2539444818' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T22:31:00.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:00 smithi171 ceph-mon[34138]: osdmap e136: 12 total, 12 up, 12 in 2024-09-10T22:31:00.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:00 smithi090 systemd[1]: Stopping Ceph mon.a for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:31:00.466+0000 7fb785000640 -1 client.14235: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:31:00.468+0000 7fb785000640 -1 client.14235: SimpleRADOSStriper: lock: snap_db_v0.db: lock failed: (2) No such file or directory 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: Exception in thread Thread-32: 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: Traceback (most recent call last): 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/threading.py", line 980, in _bootstrap_inner 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self.run() 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/threading.py", line 1306, in run 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self.function(*self.args, **self.kwargs) 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: File "/usr/share/ceph/mgr/snap_schedule/fs/schedule_client.py", line 355, in create_scheduled_snapshot 2024-09-10T22:31:00.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: self.prune_snapshots(sched) 2024-09-10T22:31:00.608 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: UnboundLocalError: local variable 'sched' referenced before assignment 2024-09-10T22:31:01.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a[30346]: 2024-09-10T22:31:00.648+0000 7f5141800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:01.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a[30346]: 2024-09-10T22:31:00.648+0000 7f5141800640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T22:31:01.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:00 smithi090 podman[115293]: 2024-09-10 22:31:00.75181524 +0000 UTC m=+0.245498256 container died ebe243d48873048e3afe33be4664c0d09e07d38bba69980f2dc888198be36372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:31:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:01 smithi090 podman[115293]: 2024-09-10 22:31:01.285991924 +0000 UTC m=+0.779674939 container cleanup ebe243d48873048e3afe33be4664c0d09e07d38bba69980f2dc888198be36372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T22:31:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:01 smithi090 bash[115293]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a 2024-09-10T22:31:01.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Sep 10 22:31:01 smithi090 podman[115304]: 2024-09-10 22:31:01.62452364 +0000 UTC m=+0.847932875 container remove ebe243d48873048e3afe33be4664c0d09e07d38bba69980f2dc888198be36372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:31:01.937 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T22:31:02.037 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:02.038 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T22:31:02.038 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T22:31:02.039 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.b 2024-09-10T22:31:02.418 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:02 smithi154 systemd[1]: Stopping Ceph mon.b for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:02 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-b[34491]: 2024-09-10T22:31:02.671+0000 7fb15da00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:03.168 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:02 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-b[34491]: 2024-09-10T22:31:02.671+0000 7fb15da00640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T22:31:03.169 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:02 smithi154 podman[74879]: 2024-09-10 22:31:02.918172703 +0000 UTC m=+0.526749106 container died 670bf7b33c22dbffd812e48187972e765bd1a7d0c6c8b75c8ca0b0d07d6acaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-b, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T22:31:03.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:03 smithi154 podman[74879]: 2024-09-10 22:31:03.511261721 +0000 UTC m=+1.119838129 container cleanup 670bf7b33c22dbffd812e48187972e765bd1a7d0c6c8b75c8ca0b0d07d6acaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T22:31:03.668 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:03 smithi154 bash[74879]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-b 2024-09-10T22:31:04.039 INFO:journalctl@ceph.mon.b.smithi154.stdout:Sep 10 22:31:03 smithi154 podman[74890]: 2024-09-10 22:31:03.788139858 +0000 UTC m=+0.857667052 container remove 670bf7b33c22dbffd812e48187972e765bd1a7d0c6c8b75c8ca0b0d07d6acaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T22:31:04.062 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T22:31:04.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:04.180 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T22:31:04.180 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T22:31:04.180 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.c 2024-09-10T22:31:04.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:04 smithi171 systemd[1]: Stopping Ceph mon.c for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:04 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-c[34134]: 2024-09-10T22:31:04.718+0000 7fc9d2c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:04 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-c[34134]: 2024-09-10T22:31:04.718+0000 7fc9d2c00640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T22:31:04.998 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:04 smithi171 podman[77979]: 2024-09-10 22:31:04.791062383 +0000 UTC m=+0.316740106 container died 6336244ad0d5abdcdd9238a440382ab7a77b11c385549d9cabc4bee403362aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-10T22:31:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:05 smithi171 podman[77979]: 2024-09-10 22:31:05.275635628 +0000 UTC m=+0.801313349 container cleanup 6336244ad0d5abdcdd9238a440382ab7a77b11c385549d9cabc4bee403362aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:31:05.498 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:05 smithi171 bash[77979]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-c 2024-09-10T22:31:05.750 INFO:journalctl@ceph.mon.c.smithi171.stdout:Sep 10 22:31:05 smithi171 podman[77991]: 2024-09-10 22:31:05.577881644 +0000 UTC m=+0.776930410 container remove 6336244ad0d5abdcdd9238a440382ab7a77b11c385549d9cabc4bee403362aa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mon-c, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T22:31:05.773 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T22:31:05.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:05.868 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T22:31:05.868 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T22:31:05.868 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.x 2024-09-10T22:31:06.307 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:05 smithi090 systemd[1]: Stopping Ceph mgr.x for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:06.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:06 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:31:06.304+0000 7fb84d000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:06.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:06 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T22:31:06.304+0000 7fb84d000640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T22:31:06.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:06 smithi090 podman[115421]: 2024-09-10 22:31:06.387913636 +0000 UTC m=+0.219535846 container died bd728928651d2b5e5ca7c48d61af2025b0321c6c7514dea9a0e4b4905ad8a622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:31:07.106 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:06 smithi090 podman[115421]: 2024-09-10 22:31:06.920056976 +0000 UTC m=+0.751679185 container cleanup bd728928651d2b5e5ca7c48d61af2025b0321c6c7514dea9a0e4b4905ad8a622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T22:31:07.107 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:06 smithi090 bash[115421]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x 2024-09-10T22:31:07.450 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Sep 10 22:31:07 smithi090 podman[115433]: 2024-09-10 22:31:07.189437437 +0000 UTC m=+0.790128192 container remove bd728928651d2b5e5ca7c48d61af2025b0321c6c7514dea9a0e4b4905ad8a622 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T22:31:07.460 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T22:31:07.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:07.537 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T22:31:07.537 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T22:31:07.537 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.y 2024-09-10T22:31:07.918 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 22:31:07 smithi154 systemd[1]: Stopping Ceph mgr.y for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:08.403 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 22:31:08 smithi154 podman[74995]: 2024-09-10 22:31:08.041285373 +0000 UTC m=+0.209370107 container died bd31d1cd99cb0a81edc9761ab4c9053d13297e196197bdcb50cdfcd31622d83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T22:31:09.062 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 22:31:08 smithi154 podman[74995]: 2024-09-10 22:31:08.775858966 +0000 UTC m=+0.943943705 container cleanup bd31d1cd99cb0a81edc9761ab4c9053d13297e196197bdcb50cdfcd31622d83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T22:31:09.062 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 22:31:08 smithi154 bash[74995]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-y 2024-09-10T22:31:09.317 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 22:31:09 smithi154 podman[75008]: 2024-09-10 22:31:09.060778859 +0000 UTC m=+1.007270704 container remove bd31d1cd99cb0a81edc9761ab4c9053d13297e196197bdcb50cdfcd31622d83c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:31:09.317 INFO:journalctl@ceph.mgr.y.smithi154.stdout:Sep 10 22:31:09 smithi154 systemd[1]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-10T22:31:09.325 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T22:31:09.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:09.399 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T22:31:09.399 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T22:31:09.399 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.z 2024-09-10T22:31:09.748 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 22:31:09 smithi171 systemd[1]: Stopping Ceph mgr.z for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:10.248 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 22:31:09 smithi171 podman[78112]: 2024-09-10 22:31:09.94733796 +0000 UTC m=+0.266187616 container died 105c9e4c7c7653b0b0530e7382ad2f620cde6c672e9facb53c4e17c6f8b1f24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T22:31:10.748 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 22:31:10 smithi171 podman[78112]: 2024-09-10 22:31:10.524789437 +0000 UTC m=+0.843639092 container cleanup 105c9e4c7c7653b0b0530e7382ad2f620cde6c672e9facb53c4e17c6f8b1f24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T22:31:10.748 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 22:31:10 smithi171 bash[78112]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-z 2024-09-10T22:31:11.045 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 22:31:10 smithi171 podman[78125]: 2024-09-10 22:31:10.772747217 +0000 UTC m=+0.812890326 container remove 105c9e4c7c7653b0b0530e7382ad2f620cde6c672e9facb53c4e17c6f8b1f24b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T22:31:11.045 INFO:journalctl@ceph.mgr.z.smithi171.stdout:Sep 10 22:31:10 smithi171 systemd[1]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-10T22:31:11.068 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T22:31:11.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:11.145 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T22:31:11.145 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T22:31:11.146 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.0 2024-09-10T22:31:11.599 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:11 smithi090 systemd[1]: Stopping Ceph osd.0 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:11.856 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:11 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0[48464]: 2024-09-10T22:31:11.597+0000 7f0856400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:11.856 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:11 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0[48464]: 2024-09-10T22:31:11.597+0000 7f0856400640 -1 osd.0 136 *** Got signal Terminated *** 2024-09-10T22:31:11.857 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:11 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0[48464]: 2024-09-10T22:31:11.597+0000 7f0856400640 -1 osd.0 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:31:17.549 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:17 smithi090 podman[115540]: 2024-09-10 22:31:17.172952333 +0000 UTC m=+5.744312066 container stop 5b2057dacd21a60f7fcf23e72b03e529c16d25d8ee8c18b95545963aa2cf4aed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:31:17.550 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:17 smithi090 podman[115540]: 2024-09-10 22:31:17.268239338 +0000 UTC m=+5.839599070 container died 5b2057dacd21a60f7fcf23e72b03e529c16d25d8ee8c18b95545963aa2cf4aed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T22:31:17.856 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:17 smithi090 podman[115540]: 2024-09-10 22:31:17.705323666 +0000 UTC m=+6.276683397 container cleanup 5b2057dacd21a60f7fcf23e72b03e529c16d25d8ee8c18b95545963aa2cf4aed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T22:31:17.856 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:17 smithi090 bash[115540]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0 2024-09-10T22:31:18.356 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:18 smithi090 podman[115571]: 2024-09-10 22:31:18.083937879 +0000 UTC m=+0.836600261 container remove 5b2057dacd21a60f7fcf23e72b03e529c16d25d8ee8c18b95545963aa2cf4aed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T22:31:18.948 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:18 smithi090 podman[115651]: 2024-09-10 22:31:18.526049972 +0000 UTC m=+0.026787880 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:31:18.948 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:18 smithi090 podman[115651]: 2024-09-10 22:31:18.652261239 +0000 UTC m=+0.152999131 container create cdb65b8e84754ad4df0ea3a837b4856e6837a63db94cf1d6a54465f882ac7cb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T22:31:19.356 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:18 smithi090 podman[115651]: 2024-09-10 22:31:18.947242208 +0000 UTC m=+0.447980111 container init cdb65b8e84754ad4df0ea3a837b4856e6837a63db94cf1d6a54465f882ac7cb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T22:31:19.357 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:18 smithi090 podman[115651]: 2024-09-10 22:31:18.953990405 +0000 UTC m=+0.454728298 container start cdb65b8e84754ad4df0ea3a837b4856e6837a63db94cf1d6a54465f882ac7cb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T22:31:19.357 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:19 smithi090 podman[115651]: 2024-09-10 22:31:19.055844855 +0000 UTC m=+0.556582760 container attach cdb65b8e84754ad4df0ea3a837b4856e6837a63db94cf1d6a54465f882ac7cb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T22:31:19.844 INFO:journalctl@ceph.osd.0.smithi090.stdout:Sep 10 22:31:19 smithi090 podman[115674]: 2024-09-10 22:31:19.459017035 +0000 UTC m=+0.026344623 container died cdb65b8e84754ad4df0ea3a837b4856e6837a63db94cf1d6a54465f882ac7cb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T22:31:20.259 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T22:31:20.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:20.335 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T22:31:20.336 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T22:31:20.336 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.1 2024-09-10T22:31:20.668 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:20 smithi154 systemd[1]: Stopping Ceph osd.1 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:21.168 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:20 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1[40185]: 2024-09-10T22:31:20.773+0000 7f65ba600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:21.168 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:20 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1[40185]: 2024-09-10T22:31:20.773+0000 7f65ba600640 -1 osd.1 136 *** Got signal Terminated *** 2024-09-10T22:31:21.169 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:20 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1[40185]: 2024-09-10T22:31:20.773+0000 7f65ba600640 -1 osd.1 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:31:26.574 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:26 smithi154 podman[75119]: 2024-09-10 22:31:26.264250281 +0000 UTC m=+5.637516951 container died e222fd539ceb4ca895bd231d46256322a7f3cbdc1e8c8453cab482152eea60d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T22:31:26.918 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:26 smithi154 podman[75119]: 2024-09-10 22:31:26.78004388 +0000 UTC m=+6.153310548 container cleanup e222fd539ceb4ca895bd231d46256322a7f3cbdc1e8c8453cab482152eea60d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:31:26.918 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:26 smithi154 bash[75119]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1 2024-09-10T22:31:27.418 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:27 smithi154 podman[75136]: 2024-09-10 22:31:27.089192483 +0000 UTC m=+0.792989294 container remove e222fd539ceb4ca895bd231d46256322a7f3cbdc1e8c8453cab482152eea60d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T22:31:27.918 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:27 smithi154 podman[75224]: 2024-09-10 22:31:27.559271561 +0000 UTC m=+0.027290960 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:31:27.919 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:27 smithi154 podman[75224]: 2024-09-10 22:31:27.691395385 +0000 UTC m=+0.159414783 container create a4354a9cf0f3f641c3fb64b3b29c7a5549398ea869b493261323aa18d26562a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T22:31:28.323 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:28 smithi154 podman[75224]: 2024-09-10 22:31:28.011357805 +0000 UTC m=+0.479377204 container init a4354a9cf0f3f641c3fb64b3b29c7a5549398ea869b493261323aa18d26562a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2) 2024-09-10T22:31:28.323 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:28 smithi154 podman[75224]: 2024-09-10 22:31:28.018167472 +0000 UTC m=+0.486186870 container start a4354a9cf0f3f641c3fb64b3b29c7a5549398ea869b493261323aa18d26562a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:31:28.323 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:28 smithi154 podman[75224]: 2024-09-10 22:31:28.111063798 +0000 UTC m=+0.579083197 container attach a4354a9cf0f3f641c3fb64b3b29c7a5549398ea869b493261323aa18d26562a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:31:28.662 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:28 smithi154 podman[75224]: 2024-09-10 22:31:28.322965522 +0000 UTC m=+0.790984921 container died a4354a9cf0f3f641c3fb64b3b29c7a5549398ea869b493261323aa18d26562a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T22:31:29.223 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T22:31:29.274 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:29 smithi154 podman[75247]: 2024-09-10 22:31:29.190986488 +0000 UTC m=+0.854587286 container remove a4354a9cf0f3f641c3fb64b3b29c7a5549398ea869b493261323aa18d26562a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T22:31:29.275 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:29 smithi154 systemd[1]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.1.service: Deactivated successfully. 2024-09-10T22:31:29.275 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:29 smithi154 systemd[1]: Stopped Ceph osd.1 for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T22:31:29.275 INFO:journalctl@ceph.osd.1.smithi154.stdout:Sep 10 22:31:29 smithi154 systemd[1]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.1.service: Consumed 2min 34.917s CPU time. 2024-09-10T22:31:29.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:29.303 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T22:31:29.303 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T22:31:29.303 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.2 2024-09-10T22:31:29.733 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:29 smithi171 systemd[1]: Stopping Ceph osd.2 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:29.998 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:29 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2[40414]: 2024-09-10T22:31:29.732+0000 7f6ad0e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:29.998 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:29 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2[40414]: 2024-09-10T22:31:29.732+0000 7f6ad0e00640 -1 osd.2 136 *** Got signal Terminated *** 2024-09-10T22:31:29.998 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:29 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2[40414]: 2024-09-10T22:31:29.732+0000 7f6ad0e00640 -1 osd.2 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:31:35.641 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:35 smithi171 podman[78238]: 2024-09-10 22:31:35.280651138 +0000 UTC m=+5.682698060 container died 91e94588bee0dc33c0f4046787271454ad42b5f584afd0c3e873e7a5913a1516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:31:35.998 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:35 smithi171 podman[78238]: 2024-09-10 22:31:35.772612063 +0000 UTC m=+6.174658985 container cleanup 91e94588bee0dc33c0f4046787271454ad42b5f584afd0c3e873e7a5913a1516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T22:31:35.998 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:35 smithi171 bash[78238]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2 2024-09-10T22:31:36.497 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:36 smithi171 podman[78268]: 2024-09-10 22:31:36.038071252 +0000 UTC m=+0.749752288 container remove 91e94588bee0dc33c0f4046787271454ad42b5f584afd0c3e873e7a5913a1516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:31:36.840 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:36 smithi171 podman[78353]: 2024-09-10 22:31:36.48418386 +0000 UTC m=+0.027370246 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:31:36.840 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:36 smithi171 podman[78353]: 2024-09-10 22:31:36.709407062 +0000 UTC m=+0.252593448 container create 2b7dc8d18a1348e82fd6f009dcf9f5569b79048607f974ebf65dd208ad6fb904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T22:31:37.131 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:36 smithi171 podman[78353]: 2024-09-10 22:31:36.972723803 +0000 UTC m=+0.515910189 container init 2b7dc8d18a1348e82fd6f009dcf9f5569b79048607f974ebf65dd208ad6fb904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T22:31:37.131 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:36 smithi171 podman[78353]: 2024-09-10 22:31:36.979575466 +0000 UTC m=+0.522761853 container start 2b7dc8d18a1348e82fd6f009dcf9f5569b79048607f974ebf65dd208ad6fb904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T22:31:37.498 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:37 smithi171 podman[78353]: 2024-09-10 22:31:37.130047129 +0000 UTC m=+0.673233515 container attach 2b7dc8d18a1348e82fd6f009dcf9f5569b79048607f974ebf65dd208ad6fb904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T22:31:37.498 INFO:journalctl@ceph.osd.2.smithi171.stdout:Sep 10 22:31:37 smithi171 podman[78376]: 2024-09-10 22:31:37.281587716 +0000 UTC m=+0.026907511 container died 2b7dc8d18a1348e82fd6f009dcf9f5569b79048607f974ebf65dd208ad6fb904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T22:31:38.031 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T22:31:38.107 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:38.107 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T22:31:38.107 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T22:31:38.107 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.3 2024-09-10T22:31:38.538 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:38 smithi090 systemd[1]: Stopping Ceph osd.3 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:38.853 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:38 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T22:31:38.535+0000 7facb7800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:38.853 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:38 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T22:31:38.535+0000 7facb7800640 -1 osd.3 136 *** Got signal Terminated *** 2024-09-10T22:31:38.854 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:38 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T22:31:38.535+0000 7facb7800640 -1 osd.3 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:31:38.854 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:38 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T22:31:38.852+0000 7facb3200640 -1 osd.3 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:15.350682+0000 front 2024-09-10T22:31:15.350710+0000 (oldest deadline 2024-09-10T22:31:38.250910+0000) 2024-09-10T22:31:38.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:38 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:38.731+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:39.106 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:39 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:39.024+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:39.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:39 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:39.738+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:39.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:39 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:39.740+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:40.106 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:39 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T22:31:39.826+0000 7facb3200640 -1 osd.3 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:15.350682+0000 front 2024-09-10T22:31:15.350710+0000 (oldest deadline 2024-09-10T22:31:38.250910+0000) 2024-09-10T22:31:40.106 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:40 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:40.008+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:40.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:40 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:40.697+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:40.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:40 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:40.742+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:41.106 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:40 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T22:31:40.822+0000 7facb3200640 -1 osd.3 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:15.350682+0000 front 2024-09-10T22:31:15.350710+0000 (oldest deadline 2024-09-10T22:31:38.250910+0000) 2024-09-10T22:31:41.106 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:41 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:41.044+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:41.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:40.870+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:41.703 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:41.377+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:41.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:41.781+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:41.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:41.700+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:42.094 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:41 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T22:31:41.803+0000 7facb3200640 -1 osd.3 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:15.350682+0000 front 2024-09-10T22:31:15.350710+0000 (oldest deadline 2024-09-10T22:31:38.250910+0000) 2024-09-10T22:31:42.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:41 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:41.914+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:42.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:42 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:42.093+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:42.687 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:42.408+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:42.687 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:42.685+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:42.997 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:42.788+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:43.048 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:42 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3[55174]: 2024-09-10T22:31:42.794+0000 7facb3200640 -1 osd.3 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:15.350682+0000 front 2024-09-10T22:31:15.350710+0000 (oldest deadline 2024-09-10T22:31:38.250910+0000) 2024-09-10T22:31:43.049 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:43 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:43.046+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:43.418 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:42 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:42.949+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:43.697 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:43.377+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:43.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:43.695+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:43.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:43.806+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:44.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:43 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:43.910+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:44.356 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:44 smithi090 podman[115753]: 2024-09-10 22:31:44.1578988 +0000 UTC m=+5.766949849 container died 0f21e66333ccc9f926b35b25622a771cdf96734da9df3f44bcaa0883e936a8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:31:44.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:44 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:44.051+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:44.715 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:44.410+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:44.856 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:44 smithi090 podman[115753]: 2024-09-10 22:31:44.686694355 +0000 UTC m=+6.295745403 container cleanup 0f21e66333ccc9f926b35b25622a771cdf96734da9df3f44bcaa0883e936a8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T22:31:44.857 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:44 smithi090 bash[115753]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3 2024-09-10T22:31:44.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:44.714+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:44.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:44.761+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:45.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:44 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:44.895+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:45.356 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:44 smithi090 podman[115781]: 2024-09-10 22:31:44.930488698 +0000 UTC m=+0.768842347 container remove 0f21e66333ccc9f926b35b25622a771cdf96734da9df3f44bcaa0883e936a8f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T22:31:45.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:45 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:45.009+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:45.687 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:45.398+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:45.710 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:45 smithi090 podman[115871]: 2024-09-10 22:31:45.362333305 +0000 UTC m=+0.027331985 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:31:45.710 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:45 smithi090 podman[115871]: 2024-09-10 22:31:45.481788261 +0000 UTC m=+0.146786940 container create 47e5cf111339dd74ac00f75213ba44d9c17ed53f44d15673ad58aa05c81379b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T22:31:45.971 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:45 smithi090 podman[115871]: 2024-09-10 22:31:45.709924966 +0000 UTC m=+0.374923645 container init 47e5cf111339dd74ac00f75213ba44d9c17ed53f44d15673ad58aa05c81379b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-10T22:31:45.971 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:45 smithi090 podman[115871]: 2024-09-10 22:31:45.716703774 +0000 UTC m=+0.381702453 container start 47e5cf111339dd74ac00f75213ba44d9c17ed53f44d15673ad58aa05c81379b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:31:45.972 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:45 smithi090 podman[115871]: 2024-09-10 22:31:45.911243092 +0000 UTC m=+0.576241771 container attach 47e5cf111339dd74ac00f75213ba44d9c17ed53f44d15673ad58aa05c81379b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:31:45.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:45.685+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:45.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:45.809+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:46.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:45 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:45.915+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:46.324 INFO:journalctl@ceph.osd.3.smithi090.stdout:Sep 10 22:31:45 smithi090 podman[115871]: 2024-09-10 22:31:45.970386269 +0000 UTC m=+0.635384958 container died 47e5cf111339dd74ac00f75213ba44d9c17ed53f44d15673ad58aa05c81379b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T22:31:46.325 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:46 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:46.040+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:46.706 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T22:31:46.718 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:46.384+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:46.783 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:46.783 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T22:31:46.783 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T22:31:46.783 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.4 2024-09-10T22:31:46.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:46.763+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:46.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:46.716+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:47.169 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:46 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:46.953+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:47.169 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:46 smithi154 systemd[1]: Stopping Ceph osd.4 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:47.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:47 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:47.033+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:47.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:47 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:47.033+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:47.652 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:47 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T22:31:47.245+0000 7ff64bc00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:47.652 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:47 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T22:31:47.245+0000 7ff64bc00640 -1 osd.4 136 *** Got signal Terminated *** 2024-09-10T22:31:47.652 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:47 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T22:31:47.245+0000 7ff64bc00640 -1 osd.4 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:31:47.678 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:47.391+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:47.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:47 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:47.650+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:47.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:47.676+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:47.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:47.759+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:48.276 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:47 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:47.938+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:48.276 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:47 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:47.938+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:48.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:47 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:47.997+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:48.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:47 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:47.997+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:48.630 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T22:31:48.274+0000 7ff647600640 -1 osd.4 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.372321+0000 front 2024-09-10T22:31:24.372360+0000 (oldest deadline 2024-09-10T22:31:47.872551+0000) 2024-09-10T22:31:48.676 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:48.417+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:48.677 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:48.417+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:48.857 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:48 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:48.539+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:48.896 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:48.628+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:48.997 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:48.781+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:48.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:48.674+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:49.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:48.894+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:49.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:48 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:48.894+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:49.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:49 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:49.045+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:49.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:49 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:49.045+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:49.643 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:49 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T22:31:49.236+0000 7ff647600640 -1 osd.4 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.372321+0000 front 2024-09-10T22:31:24.372360+0000 (oldest deadline 2024-09-10T22:31:47.872551+0000) 2024-09-10T22:31:49.694 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:49.419+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:49.694 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:49.419+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:49.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:49 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:49.529+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:49.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:49 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:49.641+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:49.997 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:49.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:49.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:49.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:49.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:49.692+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:49.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:49.692+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:50.188 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:49 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:49.919+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:50.188 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:49 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:49.919+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:50.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:50 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:50.038+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:50.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:50 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:50.038+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:50.606 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:50 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T22:31:50.186+0000 7ff647600640 -1 osd.4 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.372321+0000 front 2024-09-10T22:31:24.372360+0000 (oldest deadline 2024-09-10T22:31:47.872551+0000) 2024-09-10T22:31:50.677 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:50.416+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:50.678 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:50.416+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:50.678 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:50.675+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:50.678 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:50.675+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:50.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:50 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:50.508+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:50.896 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:50 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:50.604+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:50.997 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:50.810+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:50.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:50.810+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:51.152 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:50 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:50.894+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:51.152 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:50 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:50.894+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:51.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:51 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:51.035+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:51.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:51 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:51.035+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:51.418 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:51 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T22:31:51.151+0000 7ff647600640 -1 osd.4 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.372321+0000 front 2024-09-10T22:31:24.372360+0000 (oldest deadline 2024-09-10T22:31:47.872551+0000) 2024-09-10T22:31:51.661 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:51.401+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:51.661 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:51.401+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:51.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:51 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:51.468+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:51.894 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:51 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:51.610+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:51.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:51.847+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:51.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:51.847+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:51.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:51.659+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:51.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:51.659+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:52.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:51 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:51.893+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:52.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:51 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:51.893+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:52.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:52 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:52.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:52.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:52 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:52.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:52.658 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:52 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4[46637]: 2024-09-10T22:31:52.191+0000 7ff647600640 -1 osd.4 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.372321+0000 front 2024-09-10T22:31:24.372360+0000 (oldest deadline 2024-09-10T22:31:47.872551+0000) 2024-09-10T22:31:52.748 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:52.669+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:52.748 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:52.669+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:52.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:52.440+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:52.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:52.440+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:52.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:52 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:52.434+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:52.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:52 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:52.656+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:52.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:52 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:52.874+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:52.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:52 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:52.874+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:52.919 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:52 smithi154 podman[75338]: 2024-09-10 22:31:52.790736657 +0000 UTC m=+5.717842904 container died 8e947c40588866764633c0fceaae3606df63ebea1016d76d4f33b3b70aff506a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:31:53.247 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:52.839+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:53.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:52.839+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:53.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:53.072+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:53.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:53.072+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:53.418 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:53 smithi154 podman[75338]: 2024-09-10 22:31:53.326299402 +0000 UTC m=+6.253405648 container cleanup 8e947c40588866764633c0fceaae3606df63ebea1016d76d4f33b3b70aff506a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:31:53.418 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:53 smithi154 bash[75338]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4 2024-09-10T22:31:53.714 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:53.416+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:53.714 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:53.416+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:53.828 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:53 smithi154 podman[75355]: 2024-09-10 22:31:53.527885739 +0000 UTC m=+0.727973083 container remove 8e947c40588866764633c0fceaae3606df63ebea1016d76d4f33b3b70aff506a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-10T22:31:53.828 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:53 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:53.658+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:53.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:53 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:53.479+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:53.997 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:53.830+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:53.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:53.830+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:53.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:53.712+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:53.998 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:53.712+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:54.168 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:54 smithi154 podman[75438]: 2024-09-10 22:31:53.980474681 +0000 UTC m=+0.027626622 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:31:54.168 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:53 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:53.826+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:54.169 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:53 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:53.826+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:54.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:54 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:54.044+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:54.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:54 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:54.044+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:54.455 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:54 smithi154 podman[75438]: 2024-09-10 22:31:54.170599117 +0000 UTC m=+0.217751051 container create 4e248e384ae48e0c43f8ada2ead585ad66834b3112ea22f00474fdc24e9aadd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T22:31:54.455 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:54 smithi154 podman[75438]: 2024-09-10 22:31:54.454439091 +0000 UTC m=+0.501591025 container init 4e248e384ae48e0c43f8ada2ead585ad66834b3112ea22f00474fdc24e9aadd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T22:31:54.719 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:54 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:54.665+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:54.719 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:54 smithi154 podman[75438]: 2024-09-10 22:31:54.461531433 +0000 UTC m=+0.508683367 container start 4e248e384ae48e0c43f8ada2ead585ad66834b3112ea22f00474fdc24e9aadd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T22:31:54.719 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:54 smithi154 podman[75438]: 2024-09-10 22:31:54.573299692 +0000 UTC m=+0.620451635 container attach 4e248e384ae48e0c43f8ada2ead585ad66834b3112ea22f00474fdc24e9aadd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T22:31:54.747 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:54 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:54.447+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:54.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:54 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:54.447+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:54.748 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:54 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:54.687+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:54.748 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:54 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:54.687+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:54.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:54 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:54.505+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:55.152 INFO:journalctl@ceph.osd.4.smithi154.stdout:Sep 10 22:31:54 smithi154 podman[75438]: 2024-09-10 22:31:54.718926485 +0000 UTC m=+0.766078432 container died 4e248e384ae48e0c43f8ada2ead585ad66834b3112ea22f00474fdc24e9aadd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:31:55.152 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:54 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:54.822+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:55.152 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:54 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:54.822+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:55.247 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:54 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:54.828+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:55.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:54 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:54.828+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:55.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:55.021+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:55.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:55.021+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:55.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T22:31:55.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:31:55.656 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T22:31:55.656 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T22:31:55.657 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.5 2024-09-10T22:31:55.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:55 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:55.529+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:55.866 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:55 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:55.497+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:55.866 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:55 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:55.497+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:55.867 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:55 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:55.701+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:55.867 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:55 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:55.701+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:55.867 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:55 smithi171 systemd[1]: Stopping Ceph osd.5 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:31:55.886 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:55 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:55.615+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:55.886 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:55 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:55.791+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:55.886 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:55 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:55.791+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:56.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:55 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:55.864+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:56.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:55 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:55.864+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:56.248 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:56 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:56.067+0000 7f8f61c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:31:56.248 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:56 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:56.067+0000 7f8f61c00640 -1 osd.5 136 *** Got signal Terminated *** 2024-09-10T22:31:56.248 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:56 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:56.067+0000 7f8f61c00640 -1 osd.5 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:31:56.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:56 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:56.015+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:56.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:56 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:56.015+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:56.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:56 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:56.015+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:31:56.802 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:56 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:56.530+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:56.904 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:56 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:56.542+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:56.905 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:56 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:56.542+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:56.905 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:56 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:56.676+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:56.905 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:56 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:56.676+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:56.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:56 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:56.568+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:56.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:56 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:56.747+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:56.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:56 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:56.747+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:56.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:56 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:56.747+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:31:57.106 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:57 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:57.026+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:57.106 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:57 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:57.026+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:57.106 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:57 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:57.026+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:31:57.247 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:56 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:56.902+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:57.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:56 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:56.902+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:57.855 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:57 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:57.538+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:57.855 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:57 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:57.538+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:57.856 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:57 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:57.713+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:57.856 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:57 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:57.713+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:57.856 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:57 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:57.713+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:34.050722+0000 front 2024-09-10T22:31:34.050752+0000 (oldest deadline 2024-09-10T22:31:57.551057+0000) 2024-09-10T22:31:57.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:57 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:57.538+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:57.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:57 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:57.595+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:57.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:57 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:57.738+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:57.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:57 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:57.738+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:57.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:57 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:57.738+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:31:58.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:57 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:57.853+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:58.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:57 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:57.853+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:58.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:58 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:58.018+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:58.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:58 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:58.018+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:58.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:58 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:58.018+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:31:58.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:58 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:58.521+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:58.883 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:58 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:58.733+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:58.883 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:58 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:58.733+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:58.883 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:58 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:58.733+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:34.050722+0000 front 2024-09-10T22:31:34.050752+0000 (oldest deadline 2024-09-10T22:31:57.551057+0000) 2024-09-10T22:31:58.883 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:58 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:58.576+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:58.884 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:58 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:58.576+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:58.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:58 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:58.615+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:58.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:58 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:58.723+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:58.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:58 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:58.723+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:58.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:58 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:58.723+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:31:59.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:58 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:58.881+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:31:59.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:58 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:58.881+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:31:59.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:58 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:58.989+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:31:59.386 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:58 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:58.989+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:31:59.386 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:31:58 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:31:58.989+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:31:59.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:59 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:59.485+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:31:59.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:31:59 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:31:59.485+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:31:59.891 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:59 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:59.711+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:31:59.891 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:59 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:59.711+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:31:59.891 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:31:59 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:31:59.711+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:34.050722+0000 front 2024-09-10T22:31:34.050752+0000 (oldest deadline 2024-09-10T22:31:57.551057+0000) 2024-09-10T22:31:59.891 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:59 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:59.576+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:31:59.891 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:59 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:59.576+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:31:59.892 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:31:59 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:31:59.576+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:31:59.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:59 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:59.719+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:31:59.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:59 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:59.719+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:31:59.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:31:59 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:31:59.719+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:31:59.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:59 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:59.566+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:31:59.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:31:59 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:31:59.566+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:00.222 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:59 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:59.889+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:00.223 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:31:59 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:31:59.889+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:00.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:00.003+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:00.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:00.003+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:00.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:00.003+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:00.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:00.443+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:00.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:00 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:00.443+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:00.905 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:00 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:00.547+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:00.905 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:00 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:00.547+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:00.905 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:00 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:00.547+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:00.906 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:00 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:32:00.745+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.448282+0000 front 2024-09-10T22:31:16.448330+0000 (oldest deadline 2024-09-10T22:31:38.148074+0000) 2024-09-10T22:32:00.906 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:00 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:32:00.745+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.449748+0000 front 2024-09-10T22:31:23.449849+0000 (oldest deadline 2024-09-10T22:31:49.349632+0000) 2024-09-10T22:32:00.906 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:00 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5[46974]: 2024-09-10T22:32:00.745+0000 7f8f58600640 -1 osd.5 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:34.050722+0000 front 2024-09-10T22:31:34.050752+0000 (oldest deadline 2024-09-10T22:31:57.551057+0000) 2024-09-10T22:32:00.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:00 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:00.590+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:00.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:00 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:00.590+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:00.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:00 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:00.713+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:00.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:00 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:00.713+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:00.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:00 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:00.713+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:01.247 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:00 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:00.903+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:01.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:00 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:00.903+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:01.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:01 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:01.014+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:01.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:01 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:01.014+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:01.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:01 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:01.014+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:01.748 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:01 smithi171 podman[78458]: 2024-09-10 22:32:01.598917124 +0000 UTC m=+5.651527077 container died 729d8cdf342eed3500bb82ece79c7599c8bca064cf532af3eb33389ddab8c7c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T22:32:01.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:01 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:01.551+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:01.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:01 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:01.551+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:01.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:01 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:01.551+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:01.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:01 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:01.469+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:01.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:01 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:01.469+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:01.900 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:01 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:01.579+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:01.900 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:01 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:01.579+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:01.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:01 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:01.670+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:01.900 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:01 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:01.670+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:01.901 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:01 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:01.670+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:02.248 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:02 smithi171 podman[78458]: 2024-09-10 22:32:02.082354391 +0000 UTC m=+6.134964344 container cleanup 729d8cdf342eed3500bb82ece79c7599c8bca064cf532af3eb33389ddab8c7c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:32:02.248 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:02 smithi171 bash[78458]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5 2024-09-10T22:32:02.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:01 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:01.861+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:02.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:01 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:01.861+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:02.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:02 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:02.042+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:02.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:02 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:02.042+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:02.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:02 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:02.042+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:02.747 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:02 smithi171 podman[78486]: 2024-09-10 22:32:02.35939132 +0000 UTC m=+0.743712967 container remove 729d8cdf342eed3500bb82ece79c7599c8bca064cf532af3eb33389ddab8c7c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T22:32:02.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:02 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:02.549+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:02.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:02 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:02.549+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:02.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:02 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:02.549+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:02.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:02 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:02.509+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:02.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:02 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:02.509+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:02.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:02 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:02.621+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:02.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:02 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:02.621+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:02.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:02 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:02.621+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:02.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:02 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:02.533+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:02.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:02 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:02.533+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:03.206 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:02 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:02.852+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:03.206 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:02 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:02.852+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:03.207 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:02 smithi171 podman[78573]: 2024-09-10 22:32:02.796476492 +0000 UTC m=+0.027004397 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:32:03.207 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:02 smithi171 podman[78573]: 2024-09-10 22:32:02.910846298 +0000 UTC m=+0.141374204 container create 09b5bf46f879a091ff0f526cf8d1114a782a83ded2fc2fb8819ac9a33daffc2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:03.207 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:03 smithi171 podman[78573]: 2024-09-10 22:32:03.205587499 +0000 UTC m=+0.436115402 container init 09b5bf46f879a091ff0f526cf8d1114a782a83ded2fc2fb8819ac9a33daffc2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:32:03.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:03 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:03.079+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:03.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:03 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:03.079+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:03.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:03 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:03.079+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:03.469 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:03 smithi171 podman[78573]: 2024-09-10 22:32:03.212738313 +0000 UTC m=+0.443266217 container start 09b5bf46f879a091ff0f526cf8d1114a782a83ded2fc2fb8819ac9a33daffc2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2) 2024-09-10T22:32:03.469 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:03 smithi171 podman[78573]: 2024-09-10 22:32:03.272120442 +0000 UTC m=+0.502648360 container attach 09b5bf46f879a091ff0f526cf8d1114a782a83ded2fc2fb8819ac9a33daffc2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:32:03.748 INFO:journalctl@ceph.osd.5.smithi171.stdout:Sep 10 22:32:03 smithi171 podman[78573]: 2024-09-10 22:32:03.46778895 +0000 UTC m=+0.698316862 container died 09b5bf46f879a091ff0f526cf8d1114a782a83ded2fc2fb8819ac9a33daffc2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T22:32:03.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:03 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:03.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:03.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:03 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:03.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:03.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:03 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:03.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:03.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:03 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:03.460+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:03.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:03 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:03.460+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:03.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:03 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:03.500+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:03.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:03 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:03.500+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:03.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:03 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:03.592+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:03.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:03 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:03.592+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:03.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:03 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:03.592+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:04.247 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:03 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:03.839+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:04.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:03 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:03.839+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:04.340 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T22:32:04.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:04 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:04.066+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:04.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:04 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:04.066+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:04.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:04 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:04.066+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:04.420 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:32:04.420 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T22:32:04.420 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T22:32:04.420 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.6 2024-09-10T22:32:04.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:04 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:04.489+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:04.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:04 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:04.489+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:04.857 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:04 smithi090 systemd[1]: Stopping Ceph osd.6 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:32:04.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:04 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:04.523+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:04.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:04 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:04.523+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:04.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:04 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:04.601+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:04.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:04 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:04.601+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:04.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:04 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:04.601+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:04.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:04 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:04.601+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:04.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:04 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:04.836+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:04.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:04 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:04.836+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:04.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:04 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:04.637+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:04.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:04 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:04.637+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:04.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:04 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:04.637+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:05.356 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:04 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:04.856+0000 7f63ef000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:32:05.357 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:04 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:04.856+0000 7f63ef000640 -1 osd.6 136 *** Got signal Terminated *** 2024-09-10T22:32:05.357 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:04 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:04.856+0000 7f63ef000640 -1 osd.6 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:32:05.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:05 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:05.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:05.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:05 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:05.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:05.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:05 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:05.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:05.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:05 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:05.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:05.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:05 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:05.503+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:05.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:05 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:05.503+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:05.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:05 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:05.609+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:05.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:05 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:05.609+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:05.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:05 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:05.609+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:05.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:05 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:05.609+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:05.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:05 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:05.564+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:05.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:05 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:05.564+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:05.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:05 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:05.564+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:05.997 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:05 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:05.809+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:05.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:05 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:05.809+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:05.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:05 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:05.620+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:05.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:05 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:05.620+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:05.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:05 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:05.620+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:06.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:06 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:06.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:06.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:06 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:06.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:06.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:06 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:06.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:06.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:06 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:06.070+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:06.856 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:06 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:06.592+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:06.856 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:06 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:06.592+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:06.856 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:06 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:06.592+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:06.857 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:06 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:06.467+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:06.857 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:06 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:06.467+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:06.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:06 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:06.530+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:06.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:06 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:06.530+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:06.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:06 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:06.530+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:06.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:06 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:06.644+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:06.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:06 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:06.644+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:06.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:06 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:06.644+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:06.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:06 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:06.644+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:07.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:06 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:06.854+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:07.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:06 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:06.854+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:07.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:06 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:06.854+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:07.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:07 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:07.044+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:07.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:07 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:07.044+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:07.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:07 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:07.044+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:07.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:07 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:07.044+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:07.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:07 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:07.425+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:07.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:07 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:07.425+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:07.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:07 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:07.513+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:07.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:07 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:07.513+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:07.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:07 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:07.513+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:07.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:07 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:07.677+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:07.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:07 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:07.677+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:07.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:07 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:07.677+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:07.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:07 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:07.677+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:07.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:07 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:07.629+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:07.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:07 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:07.629+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:07.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:07 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:07.629+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:07.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:07 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:07.859+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:07.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:07 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:07.859+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:07.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:07 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:07.859+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:08.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:08 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:08.027+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:08.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:08 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:08.027+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:08.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:08 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:08.027+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:08.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:08 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:08.027+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:08.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:08 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:08.421+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:08.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:08 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:08.421+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:08.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:08 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:08.554+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:08.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:08 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:08.554+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:08.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:08 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:08.554+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:08.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:08 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:08.673+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:08.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:08 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:08.673+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:08.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:08 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:08.673+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:08.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:08 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:08.673+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:08.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:08 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:08.621+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:08.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:08 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:08.621+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:08.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:08 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:08.621+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:08.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:08 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:08.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:08.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:08 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:08.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:08.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:08 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:08.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:09.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:09 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:09.004+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:09.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:09 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:09.004+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:09.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:09 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:09.004+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:09.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:09 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:09.004+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:09.855 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:09 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:09.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:09.855 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:09 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:09.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:09.855 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:09 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:09.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:09.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:09 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:09.430+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.847519+0000 front 2024-09-10T22:31:24.847324+0000 (oldest deadline 2024-09-10T22:31:48.347053+0000) 2024-09-10T22:32:09.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:09 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6[61655]: 2024-09-10T22:32:09.430+0000 7f63e5a00640 -1 osd.6 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:28.848544+0000 front 2024-09-10T22:31:28.848745+0000 (oldest deadline 2024-09-10T22:31:59.449218+0000) 2024-09-10T22:32:09.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:09 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:09.584+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:09.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:09 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:09.584+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:09.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:09 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:09.584+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:09.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:09 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:09.663+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:09.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:09 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:09.663+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:09.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:09 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:09.663+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:09.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:09 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:09.663+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:10.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:09 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:09.853+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:10.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:09 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:09.853+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:10.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:09 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:09.853+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:10.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:10 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:10.018+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:10.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:10 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:10.018+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:10.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:10 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:10.018+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:10.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:10 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:10.018+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:10.853 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:10 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:10.562+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:10.853 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:10 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:10.562+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:10.853 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:10 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:10.562+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:10.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:10 smithi090 podman[115974]: 2024-09-10 22:32:10.5076068 +0000 UTC m=+5.788267503 container stop 52d593b1ac107e5a29a2ff57424ea787890ba3175031f800db9f4c6dbe0bcfdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T22:32:10.856 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:10 smithi090 podman[115974]: 2024-09-10 22:32:10.533467673 +0000 UTC m=+5.814128389 container died 52d593b1ac107e5a29a2ff57424ea787890ba3175031f800db9f4c6dbe0bcfdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T22:32:10.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:10 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:10.565+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:10.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:10 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:10.565+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:10.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:10 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:10.565+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:10.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:10 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:10.654+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:10.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:10 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:10.654+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:10.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:10 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:10.654+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:10.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:10 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:10.654+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:11.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:10 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:10.894+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:11.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:10 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:10.894+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:11.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:10 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:10.894+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:11.258 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:10 smithi090 podman[115974]: 2024-09-10 22:32:10.997949064 +0000 UTC m=+6.278609769 container cleanup 52d593b1ac107e5a29a2ff57424ea787890ba3175031f800db9f4c6dbe0bcfdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T22:32:11.258 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:10 smithi090 bash[115974]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6 2024-09-10T22:32:11.259 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:10 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:10.976+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:11.259 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:10 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:10.976+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:11.259 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:10 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:10.976+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:11.259 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:10 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:10.976+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:11.606 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:11 smithi090 podman[115991]: 2024-09-10 22:32:11.257790107 +0000 UTC m=+0.721470472 container remove 52d593b1ac107e5a29a2ff57424ea787890ba3175031f800db9f4c6dbe0bcfdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T22:32:11.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:11 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:11.579+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:11.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:11 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:11.579+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:11.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:11 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:11.579+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:11.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:11 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:11.616+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:11.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:11 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:11.616+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:11.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:11 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:11.616+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:11.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:11 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:11.616+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:11.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:11 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:11.598+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:11.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:11 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:11.598+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:11.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:11 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:11.598+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:12.090 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:11 smithi090 podman[116077]: 2024-09-10 22:32:11.71055505 +0000 UTC m=+0.027064354 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:32:12.090 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:11 smithi090 podman[116077]: 2024-09-10 22:32:11.808966402 +0000 UTC m=+0.125475707 container create 2858293d752dc320a94d29c2b0d4978acfe9c91fa7041e1477ac1fb072bfa126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:32:12.090 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:12 smithi090 podman[116077]: 2024-09-10 22:32:12.088813629 +0000 UTC m=+0.405322934 container init 2858293d752dc320a94d29c2b0d4978acfe9c91fa7041e1477ac1fb072bfa126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:32:12.090 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:11 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:11.947+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:12.090 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:11 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:11.947+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:12.090 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:11 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:11.948+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:12.090 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:11 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:11.948+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:12.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:11 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:11.925+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:12.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:11 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:11.925+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:12.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:11 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:11.925+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:12.346 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:12 smithi090 podman[116077]: 2024-09-10 22:32:12.095752185 +0000 UTC m=+0.412261495 container start 2858293d752dc320a94d29c2b0d4978acfe9c91fa7041e1477ac1fb072bfa126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:12.346 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:12 smithi090 podman[116077]: 2024-09-10 22:32:12.196957676 +0000 UTC m=+0.513466982 container attach 2858293d752dc320a94d29c2b0d4978acfe9c91fa7041e1477ac1fb072bfa126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T22:32:12.606 INFO:journalctl@ceph.osd.6.smithi090.stdout:Sep 10 22:32:12 smithi090 podman[116077]: 2024-09-10 22:32:12.345864956 +0000 UTC m=+0.662374274 container died 2858293d752dc320a94d29c2b0d4978acfe9c91fa7041e1477ac1fb072bfa126 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:12.897 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:12 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:12.583+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:12.898 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:12 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:12.583+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:12.898 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:12 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:12.583+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:12.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:12 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:12.647+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:12.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:12 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:12.647+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:12.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:12 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:12.647+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:12.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:12 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:12.604+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:12.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:12 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:12.604+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:12.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:12 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:12.604+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:12.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:12 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:12.604+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:13.125 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T22:32:13.200 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:32:13.201 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T22:32:13.201 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T22:32:13.201 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.7 2024-09-10T22:32:13.247 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:12 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:12.896+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:13.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:12 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:12.896+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:13.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:12 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:12.896+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:13.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:12 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:12.987+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:13.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:12 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:12.987+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:13.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:12 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:12.987+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:13.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:12 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:12.987+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:13.565 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:13 smithi154 systemd[1]: Stopping Ceph osd.7 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:32:13.861 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:13 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:13.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:13.861 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:13 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:13.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:13.861 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:13 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:13.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:13.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:13.564+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:13.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:13.564+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:13.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:13.564+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:13.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:13.564+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:13.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:13.619+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:13.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:13.619+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:13.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:13.619+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:13.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:13.675+0000 7fd366400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:32:13.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:13.675+0000 7fd366400640 -1 osd.7 136 *** Got signal Terminated *** 2024-09-10T22:32:13.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:13 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:13.675+0000 7fd366400640 -1 osd.7 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:32:14.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:13 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:13.860+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:14.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:13 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:13.860+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:14.248 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:13 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:13.860+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:14.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:13 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:13.961+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:14.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:13 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:13.961+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:14.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:13 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:13.961+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:14.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:13 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:13.961+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:14.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:13 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:13.961+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:14.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:14 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:14.590+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:14.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:14 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:14.590+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:14.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:14 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:14.590+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:14.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:14 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:14.593+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:14.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:14 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:14.593+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:14.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:14 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:14.593+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:14.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:14 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:14.593+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:14.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:14 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:14.855+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:14.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:14 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:14.855+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:14.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:14 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:14.855+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:14.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:14 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:14.620+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:14.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:14 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:14.620+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:14.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:14 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:14.620+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:14.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:14 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:14.620+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:15.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:14 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:14.958+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:15.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:14 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:14.958+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:15.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:14 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:14.958+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:15.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:14 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:14.958+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:15.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:14 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:14.958+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:15.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:15 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:15.567+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:15.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:15 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:15.567+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:15.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:15 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:15.567+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:15.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:15 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:15.567+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:15.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:15 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:15.552+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:15.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:15 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:15.552+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:15.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:15 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:15.552+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:15.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:15 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:15.553+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:51.286366+0000 front 2024-09-10T22:31:51.286402+0000 (oldest deadline 2024-09-10T22:32:15.386489+0000) 2024-09-10T22:32:15.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:15 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:15.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:15.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:15 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:15.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:15.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:15 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:15.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:15.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:15 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:15.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:15.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:15 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:15.822+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:15.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:15 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:15.822+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:15.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:15 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:15.822+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:15.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:15 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:15.822+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:16.106 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:15 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:15.971+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:16.107 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:15 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:15.971+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:16.107 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:15 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:15.971+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:16.107 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:15 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:15.971+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:16.107 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:15 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:15.971+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:16.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:16 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:16.565+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:16.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:16 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:16.565+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:16.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:16 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:16.565+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:16.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:16 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:16.565+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:16.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:16 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:16.554+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:16.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:16 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:16.554+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:16.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:16 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:16.554+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:16.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:16 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:16.554+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:51.286366+0000 front 2024-09-10T22:31:51.286402+0000 (oldest deadline 2024-09-10T22:32:15.386489+0000) 2024-09-10T22:32:16.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:16 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:16.601+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:16.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:16 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:16.601+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:16.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:16 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:16.601+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:16.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:16 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:16.601+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:16.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:16 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:16.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:16.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:16 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:16.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:16.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:16 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:16.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:16.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:16 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:16.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:17.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:17 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:17.021+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:17.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:17 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:17.021+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:17.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:17 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:17.021+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:17.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:17 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:17.021+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:17.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:17 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:17.021+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:17.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:17.549+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:17.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:17.549+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:17.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:17.549+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:17.918 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:17.549+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:51.286366+0000 front 2024-09-10T22:31:51.286402+0000 (oldest deadline 2024-09-10T22:32:15.386489+0000) 2024-09-10T22:32:17.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:17.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:17.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:17.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:17.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:17.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:17.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:17.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:17.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:17.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:17.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:17 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:17.644+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:17.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:17 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:17.644+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:17.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:17 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:17.644+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:17.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:17 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:17.644+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:17.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:17 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:17.842+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:17.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:17 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:17.842+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:17.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:17 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:17.842+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:17.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:17 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:17.842+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:18.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:18.053+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:18.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:18.053+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:18.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:18.053+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:18.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:18.053+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:18.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:18 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:18.053+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:18.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:18.547+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:24.981816+0000 front 2024-09-10T22:31:24.981916+0000 (oldest deadline 2024-09-10T22:31:47.281967+0000) 2024-09-10T22:32:18.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:18.547+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.584664+0000 front 2024-09-10T22:31:33.584818+0000 (oldest deadline 2024-09-10T22:31:59.484881+0000) 2024-09-10T22:32:18.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:18.547+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.485445+0000 front 2024-09-10T22:31:39.485404+0000 (oldest deadline 2024-09-10T22:32:05.385349+0000) 2024-09-10T22:32:18.919 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7[53546]: 2024-09-10T22:32:18.547+0000 7fd361e00640 -1 osd.7 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:51.286366+0000 front 2024-09-10T22:31:51.286402+0000 (oldest deadline 2024-09-10T22:32:15.386489+0000) 2024-09-10T22:32:18.920 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:18.545+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:18.920 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:18.545+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:18.920 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:18.545+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:18.920 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:18.545+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:18.920 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:18 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:18.545+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:18.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:18 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:18.857+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:18.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:18 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:18.857+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:18.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:18 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:18.857+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:18.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:18 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:18.857+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:18.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:18 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:18.637+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:18.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:18 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:18.637+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:18.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:18 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:18.637+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:18.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:18 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:18.637+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:19.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:19 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:19.063+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:19.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:19 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:19.063+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:19.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:19 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:19.063+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:19.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:19 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:19.063+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:19.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:19 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:19.063+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:19.535 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:19 smithi154 podman[75553]: 2024-09-10 22:32:19.242466521 +0000 UTC m=+5.748511988 container died e20038b802b695ed624eee1a5540895442ec48135a45b78fc130caf98335df5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T22:32:19.792 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:19 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:19.533+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:19.792 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:19 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:19.533+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:19.792 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:19 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:19.533+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:19.792 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:19 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:19.533+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:19.792 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:19 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:19.533+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:19.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:19 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:19.664+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:19.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:19 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:19.664+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:19.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:19 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:19.664+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:19.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:19 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:19.664+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:19.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:19 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:19.834+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:19.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:19 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:19.834+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:19.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:19 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:19.834+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:19.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:19 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:19.834+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:20.066 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:19 smithi154 podman[75553]: 2024-09-10 22:32:19.791143717 +0000 UTC m=+6.297189188 container cleanup e20038b802b695ed624eee1a5540895442ec48135a45b78fc130caf98335df5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T22:32:20.066 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:19 smithi154 bash[75553]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7 2024-09-10T22:32:20.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.026+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:20.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.026+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:20.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.026+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:20.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.026+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:20.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.026+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:20.418 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:20 smithi154 podman[75571]: 2024-09-10 22:32:20.065127151 +0000 UTC m=+0.804534137 container remove e20038b802b695ed624eee1a5540895442ec48135a45b78fc130caf98335df5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T22:32:20.840 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:20 smithi154 podman[75654]: 2024-09-10 22:32:20.508498917 +0000 UTC m=+0.027496732 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:32:20.840 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:20 smithi154 podman[75654]: 2024-09-10 22:32:20.690948741 +0000 UTC m=+0.209946555 container create 73ab7eb0db0b967b9f12376b5e547ba5feb3384741e05309da40bc9b10502cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:20.841 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:20 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:20.550+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:20.841 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:20 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:20.550+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:20.841 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:20 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:20.550+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:20.841 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:20 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:20.550+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:20.841 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:20 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:20.550+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:20.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:20 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:20.644+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:20.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:20 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:20.644+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:20.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:20 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:20.644+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:20.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:20 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:20.644+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:20.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:20 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:20.858+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:20.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:20 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:20.858+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:20.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:20 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:20.858+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:20.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:20 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:20.858+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:21.110 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:21 smithi154 podman[75654]: 2024-09-10 22:32:21.006295609 +0000 UTC m=+0.525293424 container init 73ab7eb0db0b967b9f12376b5e547ba5feb3384741e05309da40bc9b10502cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T22:32:21.111 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:21 smithi154 podman[75654]: 2024-09-10 22:32:21.013281972 +0000 UTC m=+0.532279796 container start 73ab7eb0db0b967b9f12376b5e547ba5feb3384741e05309da40bc9b10502cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:21.111 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:21 smithi154 podman[75654]: 2024-09-10 22:32:21.109907646 +0000 UTC m=+0.628905461 container attach 73ab7eb0db0b967b9f12376b5e547ba5feb3384741e05309da40bc9b10502cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T22:32:21.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.978+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:21.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.978+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:21.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.978+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:21.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.978+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:21.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:20 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:20.978+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:21.418 INFO:journalctl@ceph.osd.7.smithi154.stdout:Sep 10 22:32:21 smithi154 podman[75654]: 2024-09-10 22:32:21.27419256 +0000 UTC m=+0.793190377 container died 73ab7eb0db0b967b9f12376b5e547ba5feb3384741e05309da40bc9b10502cdd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T22:32:21.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:21 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:21.537+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:21.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:21 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:21.537+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:21.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:21 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:21.537+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:21.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:21 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:21.537+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:21.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:21 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:21.537+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:21.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:21.645+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:21.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:21.645+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:21.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:21.645+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:21.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:21.645+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:21.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:21.865+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:21.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:21.865+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:21.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:21.865+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:21.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:21 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:21.865+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:22.159 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T22:32:22.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:32:22.236 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T22:32:22.236 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T22:32:22.236 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.8 2024-09-10T22:32:22.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:21 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:21.982+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:22.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:21 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:21.982+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:22.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:21 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:21.982+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:22.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:21 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:21.982+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:22.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:21 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:21.982+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:22.598 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:22 smithi171 systemd[1]: Stopping Ceph osd.8 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:32:22.849 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:22.597+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:22.849 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:22.597+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:22.849 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:22.597+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:22.849 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:22.597+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:22.850 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:22.726+0000 7fcdd9a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:32:22.850 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:22.726+0000 7fcdd9a00640 -1 osd.8 136 *** Got signal Terminated *** 2024-09-10T22:32:22.850 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:22.726+0000 7fcdd9a00640 -1 osd.8 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:32:22.850 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:22.848+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:22.850 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:22.848+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:22.850 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:22.848+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:22.850 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:22 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:22.848+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:22.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:22 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:22.586+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:22.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:22 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:22.586+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:22.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:22 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:22.586+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:22.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:22 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:22.586+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:22.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:22 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:22.586+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:23.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:23 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:23.029+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:23.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:23 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:23.029+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:23.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:23 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:23.029+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:23.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:23 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:23.029+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:23.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:23 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:23.029+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:23.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:23 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:23.554+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:23.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:23 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:23.554+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:23.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:23 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:23.554+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:23.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:23 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:23.554+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:23.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:23 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:23.554+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:23.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:23 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:23.554+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:23.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:23 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:23.584+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:23.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:23 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:23.584+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:23.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:23 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:23.584+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:23.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:23 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:23.584+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:23.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:23 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:23.584+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:23.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:23 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:23.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:23.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:23 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:23.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:23.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:23 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:23.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:23.999 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:23 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:23.813+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:24.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:24 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:24.003+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:24.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:24 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:24.003+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:24.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:24 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:24.003+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:24.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:24 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:24.003+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:24.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:24 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:24.003+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:24.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:24 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:24.003+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:24.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:24 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:24.597+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:24.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:24 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:24.597+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:24.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:24 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:24.597+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:24.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:24 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:24.597+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:24.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:24 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:24.597+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:24.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:24 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:24.597+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:24.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:24 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:24.797+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:24.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:24 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:24.797+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:24.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:24 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:24.797+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:24.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:24 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:24.797+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:24.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:24 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:24.565+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:24.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:24 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:24.565+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:24.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:24 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:24.565+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:24.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:24 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:24.565+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:24.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:24 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:24.565+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:25.307 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:25 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:25.035+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:25.307 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:25 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:25.035+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:25.307 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:25 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:25.035+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:25.307 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:25 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:25.035+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:25.307 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:25 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:25.035+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:25.307 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:25 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:25.035+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:25.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:25 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:25.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:25.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:25 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:25.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:25.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:25 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:25.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:25.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:25 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:25.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:25.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:25 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:25.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:25.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:25 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:25.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:25.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:25.756+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:25.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:25.756+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:25.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:25.756+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:25.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:25.756+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:25.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:25.756+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.216672+0000 front 2024-09-10T22:32:00.216694+0000 (oldest deadline 2024-09-10T22:32:25.516635+0000) 2024-09-10T22:32:25.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:25.518+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:25.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:25.518+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:25.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:25.518+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:25.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:25.518+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:25.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:25 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:25.518+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:26.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:26 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:26.077+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:26.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:26 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:26.077+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:26.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:26 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:26.077+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:26.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:26 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:26.077+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:26.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:26 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:26.077+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:26.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:26 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:26.077+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:26.819 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:26 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:26.544+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:26.820 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:26 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:26.544+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:26.820 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:26 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:26.544+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:26.820 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:26 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:26.544+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:26.820 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:26 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:26.544+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:26.820 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:26 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:26.544+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:26.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:26.745+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:26.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:26.745+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:26.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:26.745+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:26.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:26.745+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:26.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:26.745+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.216672+0000 front 2024-09-10T22:32:00.216694+0000 (oldest deadline 2024-09-10T22:32:25.516635+0000) 2024-09-10T22:32:26.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:26.499+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:26.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:26.499+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:26.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:26.499+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:26.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:26.499+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:26.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:26 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:26.499+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:27.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:27 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:27.059+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:27.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:27 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:27.059+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:27.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:27 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:27.059+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:27.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:27 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:27.059+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:27.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:27 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:27.059+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:27.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:27 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:27.059+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:27.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:27.488+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:27.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:27.488+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:27.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:27.488+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:27.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:27.488+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:27.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:27.488+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:27.748 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:27.723+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:11.605704+0000 front 2024-09-10T22:31:11.605656+0000 (oldest deadline 2024-09-10T22:31:39.206770+0000) 2024-09-10T22:32:27.748 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:27.723+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.308214+0000 front 2024-09-10T22:31:23.308372+0000 (oldest deadline 2024-09-10T22:31:49.109434+0000) 2024-09-10T22:32:27.748 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:27.723+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.313989+0000 front 2024-09-10T22:31:42.313819+0000 (oldest deadline 2024-09-10T22:32:06.314324+0000) 2024-09-10T22:32:27.749 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:27.723+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.115755+0000 front 2024-09-10T22:31:52.115702+0000 (oldest deadline 2024-09-10T22:32:15.615452+0000) 2024-09-10T22:32:27.749 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:27 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8[53463]: 2024-09-10T22:32:27.723+0000 7fcdd5400640 -1 osd.8 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.216672+0000 front 2024-09-10T22:32:00.216694+0000 (oldest deadline 2024-09-10T22:32:25.516635+0000) 2024-09-10T22:32:27.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:27 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:27.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:27.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:27 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:27.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:27.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:27 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:27.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:27.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:27 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:27.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:27.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:27 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:27.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:27.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:27 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:27.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:28.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:28 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:28.062+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:28.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:28 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:28.062+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:28.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:28 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:28.062+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:28.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:28 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:28.062+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:28.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:28 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:28.062+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:28.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:28 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:28.062+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:28.668 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:28 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:28.536+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:28.668 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:28 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:28.536+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:28.668 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:28 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:28.536+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:28.669 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:28 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:28.536+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:28.669 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:28 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:28.536+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:28.669 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:28 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:28.536+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:28.683 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:28 smithi171 podman[78674]: 2024-09-10 22:32:28.292761343 +0000 UTC m=+5.774841649 container died ea05810875ce98bfcca9088da5d03a3e96d3de012a73c8b5cc245d1ae39d8e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:32:28.684 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:28 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:28.524+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:28.684 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:28 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:28.524+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:28.684 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:28 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:28.524+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:28.684 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:28 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:28.524+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:28.684 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:28 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:28.524+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:28.997 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:28 smithi171 podman[78674]: 2024-09-10 22:32:28.823308265 +0000 UTC m=+6.305388569 container cleanup ea05810875ce98bfcca9088da5d03a3e96d3de012a73c8b5cc245d1ae39d8e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T22:32:28.998 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:28 smithi171 bash[78674]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8 2024-09-10T22:32:29.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:29 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:29.098+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:29.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:29 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:29.098+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:29.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:29 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:29.098+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:29.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:29 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:29.098+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:29.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:29 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:29.098+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:29.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:29 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:29.098+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:29.497 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:29 smithi171 podman[78690]: 2024-09-10 22:32:29.158871465 +0000 UTC m=+0.853946324 container remove ea05810875ce98bfcca9088da5d03a3e96d3de012a73c8b5cc245d1ae39d8e34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T22:32:29.852 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:29 smithi171 podman[78777]: 2024-09-10 22:32:29.638010534 +0000 UTC m=+0.027998975 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:32:29.852 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:29 smithi171 podman[78777]: 2024-09-10 22:32:29.851723064 +0000 UTC m=+0.241711504 container create 09253aa718ee9d308f1c63ecde7cad711af6464b25e8ad7e8e13db4c2b5971fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:29.853 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:29 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:29.557+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:29.853 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:29 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:29.557+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:29.853 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:29 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:29.557+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:29.853 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:29 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:29.557+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:29.853 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:29 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:29.557+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:29.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:29 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:29.503+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:29.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:29 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:29.503+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:29.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:29 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:29.503+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:29.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:29 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:29.503+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:29.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:29 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:29.503+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:29.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:29 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:29.503+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:30.423 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:30 smithi171 podman[78777]: 2024-09-10 22:32:30.163171628 +0000 UTC m=+0.553160069 container init 09253aa718ee9d308f1c63ecde7cad711af6464b25e8ad7e8e13db4c2b5971fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:30.423 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:30 smithi171 podman[78777]: 2024-09-10 22:32:30.170565241 +0000 UTC m=+0.560553686 container start 09253aa718ee9d308f1c63ecde7cad711af6464b25e8ad7e8e13db4c2b5971fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T22:32:30.424 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:30 smithi171 podman[78777]: 2024-09-10 22:32:30.280509014 +0000 UTC m=+0.670497457 container attach 09253aa718ee9d308f1c63ecde7cad711af6464b25e8ad7e8e13db4c2b5971fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T22:32:30.606 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:30 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:30.108+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:30.607 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:30 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:30.108+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:30.607 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:30 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:30.108+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:30.607 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:30 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:30.108+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:30.607 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:30 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:30.108+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:30.607 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:30 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:30.108+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:30.748 INFO:journalctl@ceph.osd.8.smithi171.stdout:Sep 10 22:32:30 smithi171 podman[78777]: 2024-09-10 22:32:30.422464122 +0000 UTC m=+0.812452575 container died 09253aa718ee9d308f1c63ecde7cad711af6464b25e8ad7e8e13db4c2b5971fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T22:32:30.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:30 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:30.579+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:30.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:30 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:30.579+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:30.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:30 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:30.579+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:30.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:30 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:30.579+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:30.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:30 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:30.579+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:30.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:30 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:30.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:30.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:30 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:30.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:30.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:30 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:30.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:30.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:30 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:30.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:30.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:30 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:30.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:30.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:30 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:30.515+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:31.219 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T22:32:31.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:32:31.297 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T22:32:31.297 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T22:32:31.297 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.9 2024-09-10T22:32:31.353 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:31.079+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:31.354 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:31.079+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:31.354 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:31.079+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:31.354 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:31.079+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:31.354 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:31.079+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:31.354 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:31.079+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:31.606 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 systemd[1]: Stopping Ceph osd.9 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:32:31.909 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:31 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:31.628+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:31.909 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:31 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:31.628+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:31.909 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:31 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:31.628+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:31.910 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:31 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:31.628+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:31.910 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:31 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:31.628+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:31.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:31 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:31.523+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:31.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:31 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:31.523+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:31.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:31 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:31.523+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:31.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:31 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:31.523+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:31.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:31 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:31.523+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:31.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:31 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:31.523+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:32.061 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:31.726+0000 7f0b23e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:32:32.061 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:31.726+0000 7f0b23e00640 -1 osd.9 136 *** Got signal Terminated *** 2024-09-10T22:32:32.061 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:31 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:31.726+0000 7f0b23e00640 -1 osd.9 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:32:32.061 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:32 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:32.058+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:32.061 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:32 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:32.058+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:32.061 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:32 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:32.058+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:32.061 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:32 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:32.058+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:32.061 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:32 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:32.058+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:32.061 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:32 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:32.058+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:32.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:32 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:32.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:32.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:32 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:32.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:32.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:32 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:32.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:32.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:32 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:32.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:32.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:32 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:32.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:32.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:32 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:32.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:32.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:32 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:32.548+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:32.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:32 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:32.607+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:32.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:32 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:32.607+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:32.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:32 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:32.607+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:32.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:32 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:32.607+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:32.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:32 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:32.607+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:32.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:32 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:32.607+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:33.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:33 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:33.089+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:33.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:33 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:33.089+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:33.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:33 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:33.089+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:33.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:33 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:33.089+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:33.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:33 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:33.089+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:33.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:33 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:33.089+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:33.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:33 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:33.541+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:33.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:33 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:33.541+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:33.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:33 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:33.541+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:33.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:33 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:33.541+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:33.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:33 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:33.541+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:33.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:33 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:33.541+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:33.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:33 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:33.541+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:33.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:33 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:33.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:33.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:33 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:33.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:33.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:33 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:33.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:33.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:33 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:33.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:33.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:33 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:33.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:33.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:33 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:33.602+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:34.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:34.064+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:34.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:34.064+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:34.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:34.064+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:34.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:34.064+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:34.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:34.064+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:34.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:34 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:34.064+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:34.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:34 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:34.520+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:34.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:34 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:34.520+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:34.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:34 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:34.520+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:34.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:34 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:34.520+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:34.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:34 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:34.520+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:34.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:34 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:34.520+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:34.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:34 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:34.520+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:34.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:34 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:34.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:34.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:34 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:34.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:34.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:34 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:34.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:34.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:34 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:34.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:34.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:34 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:34.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:34.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:34 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:34.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:35.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:35 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:35.050+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:35.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:35 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:35.050+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:35.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:35 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:35.050+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:35.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:35 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:35.050+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:35.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:35 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:35.050+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:35.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:35 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:35.050+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:35.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:35 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:35.557+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:35.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:35 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:35.557+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:35.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:35 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:35.557+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:35.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:35 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:35.557+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:35.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:35 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:35.557+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:35.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:35 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:35.557+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:35.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:35 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:35.557+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:35.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:35 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:35.613+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:35.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:35 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:35.613+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:35.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:35 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:35.613+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:35.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:35 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:35.613+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:35.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:35 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:35.613+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:35.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:35 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:35.613+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:36.356 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:36 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:36.046+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.681596+0000 front 2024-09-10T22:31:14.681736+0000 (oldest deadline 2024-09-10T22:31:38.781726+0000) 2024-09-10T22:32:36.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:36 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:36.046+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:25.085016+0000 front 2024-09-10T22:31:25.085266+0000 (oldest deadline 2024-09-10T22:31:46.785218+0000) 2024-09-10T22:32:36.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:36 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:36.046+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:32.086372+0000 front 2024-09-10T22:31:32.086342+0000 (oldest deadline 2024-09-10T22:31:55.586229+0000) 2024-09-10T22:32:36.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:36 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:36.046+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:39.087352+0000 front 2024-09-10T22:31:39.087377+0000 (oldest deadline 2024-09-10T22:32:04.987595+0000) 2024-09-10T22:32:36.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:36 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:36.046+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.888773+0000 front 2024-09-10T22:31:50.888709+0000 (oldest deadline 2024-09-10T22:32:13.788781+0000) 2024-09-10T22:32:36.357 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:36 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9[68588]: 2024-09-10T22:32:36.046+0000 7f0b1f800640 -1 osd.9 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:59.590642+0000 front 2024-09-10T22:31:59.590518+0000 (oldest deadline 2024-09-10T22:32:23.690508+0000) 2024-09-10T22:32:36.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:36 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:36.591+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:36.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:36 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:36.591+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:36.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:36 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:36.591+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:36.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:36 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:36.591+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:36.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:36 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:36.591+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:36.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:36 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:36.591+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:36.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:36 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:36.591+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:36.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:36 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:36.646+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:36.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:36 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:36.646+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:36.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:36 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:36.646+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:36.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:36 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:36.646+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:36.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:36 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:36.646+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:36.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:36 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:36.646+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:37.568 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:37 smithi090 podman[116191]: 2024-09-10 22:32:37.283361972 +0000 UTC m=+5.692697742 container died 33aa1ce867d5d3becf164296292803b25044da5e315027100348afb2b70dd140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:32:37.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:37 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:37.552+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:37.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:37 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:37.552+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:37.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:37 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:37.552+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:37.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:37 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:37.552+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:37.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:37 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:37.552+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:37.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:37 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:37.552+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:37.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:37 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:37.552+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:37.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:37 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:37.617+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:37.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:37 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:37.617+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:37.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:37 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:37.617+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:37.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:37 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:37.617+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:37.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:37 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:37.617+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:37.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:37 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:37.617+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:38.219 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:37 smithi090 podman[116191]: 2024-09-10 22:32:37.883104891 +0000 UTC m=+6.292440659 container cleanup 33aa1ce867d5d3becf164296292803b25044da5e315027100348afb2b70dd140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T22:32:38.219 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:37 smithi090 bash[116191]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9 2024-09-10T22:32:38.606 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:38 smithi090 podman[116208]: 2024-09-10 22:32:38.218208793 +0000 UTC m=+0.904576863 container remove 33aa1ce867d5d3becf164296292803b25044da5e315027100348afb2b70dd140 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=) 2024-09-10T22:32:38.668 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:38 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:38.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:38.669 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:38 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:38.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:38.669 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:38 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:38.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:38.669 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:38 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:38.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:38.669 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:38 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:38.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:38.669 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:38 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:38.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:38.669 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:38 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:38.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:38.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:38 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:38.655+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:38.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:38 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:38.655+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:38.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:38 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:38.655+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:38.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:38 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:38.655+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:38.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:38 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:38.655+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:38.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:38 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:38.655+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:39.033 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:38 smithi090 podman[116297]: 2024-09-10 22:32:38.5892439 +0000 UTC m=+0.026713125 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:32:39.033 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:38 smithi090 podman[116297]: 2024-09-10 22:32:38.702633371 +0000 UTC m=+0.140102563 container create 656cfcd3b66645632da4087738b14ebfb6241f410507f4799432d5f584280d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:39.033 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:39 smithi090 podman[116297]: 2024-09-10 22:32:39.032220242 +0000 UTC m=+0.469689467 container init 656cfcd3b66645632da4087738b14ebfb6241f410507f4799432d5f584280d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:39.288 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:39 smithi090 podman[116297]: 2024-09-10 22:32:39.038950564 +0000 UTC m=+0.476419758 container start 656cfcd3b66645632da4087738b14ebfb6241f410507f4799432d5f584280d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T22:32:39.288 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:39 smithi090 podman[116297]: 2024-09-10 22:32:39.10661787 +0000 UTC m=+0.544087068 container attach 656cfcd3b66645632da4087738b14ebfb6241f410507f4799432d5f584280d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T22:32:39.606 INFO:journalctl@ceph.osd.9.smithi090.stdout:Sep 10 22:32:39 smithi090 podman[116297]: 2024-09-10 22:32:39.286968935 +0000 UTC m=+0.724438150 container died 656cfcd3b66645632da4087738b14ebfb6241f410507f4799432d5f584280d86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:39.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:39 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:39.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:39.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:39 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:39.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:39.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:39 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:39.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:39.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:39 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:39.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:39.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:39 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:39.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:39.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:39 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:39.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:39.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:39 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:39.518+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:39.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:39 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:39.631+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:39.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:39 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:39.631+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:39.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:39 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:39.631+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:39.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:39 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:39.631+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:39.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:39 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:39.631+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:39.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:39 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:39.631+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:40.114 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T22:32:40.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:32:40.192 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T22:32:40.192 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T22:32:40.192 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.10 2024-09-10T22:32:40.507 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 systemd[1]: Stopping Ceph osd.10 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:32:40.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.505+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:40.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.505+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:40.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.505+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:40.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.505+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:40.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.505+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:40.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.505+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:40.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.505+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:40.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.627+0000 7f1cab800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:32:40.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.627+0000 7f1cab800640 -1 osd.10 136 *** Got signal Terminated *** 2024-09-10T22:32:40.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:40 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:40.627+0000 7f1cab800640 -1 osd.10 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:32:40.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:40 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:40.610+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:40.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:40 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:40.610+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:40.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:40 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:40.610+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:40.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:40 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:40.610+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:40.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:40 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:40.610+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:40.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:40 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:40.610+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:41.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:41.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:41.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:41.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:41.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:41.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:41.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:41.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:41.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:41.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:41.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:41.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:41.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:41.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:41.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:41 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:41.474+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:41.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:41 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:41.474+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:41.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:41 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:41.474+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:41.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:41 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:41.474+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:41.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:41 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:41.474+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:41.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:41 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:41.474+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:41.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:41 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:41.474+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:42.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:42 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:42.488+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:42.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:42 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:42.488+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:42.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:42 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:42.488+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:42.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:42 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:42.488+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:42.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:42 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:42.488+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:42.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:42 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:42.488+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:42.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:42 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:42.488+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:42.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:42.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:42.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:42.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:42.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:42.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:42.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:42.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:42.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:42.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:42.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:42.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:42.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:42 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:42.572+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:43.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:43 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:43.478+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:43.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:43 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:43.478+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:43.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:43 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:43.478+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:43.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:43 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:43.478+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:43.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:43 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:43.478+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:43.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:43 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:43.478+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:43.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:43 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:43.478+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:43.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:43.564+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:43.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:43.564+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:43.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:43.564+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:43.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:43.564+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:43.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:43.564+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:43.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:43.564+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:43.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:43 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:43.564+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:44.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:44 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:44.504+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:44.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:44 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:44.504+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:44.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:44 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:44.504+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:44.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:44 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:44.504+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:44.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:44 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:44.504+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:44.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:44 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:44.504+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:44.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:44 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:44.504+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:44.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:44.520+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:44.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:44.520+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:44.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:44.520+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:44.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:44.520+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:44.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:44.520+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:44.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:44.520+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:44.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:44 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:44.520+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:45.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:45 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:45.534+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:16.189341+0000 front 2024-09-10T22:31:16.189207+0000 (oldest deadline 2024-09-10T22:31:40.289341+0000) 2024-09-10T22:32:45.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:45 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:45.534+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:23.191011+0000 front 2024-09-10T22:31:23.191047+0000 (oldest deadline 2024-09-10T22:31:47.792011+0000) 2024-09-10T22:32:45.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:45 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:45.534+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:31.694923+0000 front 2024-09-10T22:31:31.694938+0000 (oldest deadline 2024-09-10T22:31:56.394688+0000) 2024-09-10T22:32:45.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:45 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:45.534+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6814 osd.3 since back 2024-09-10T22:31:42.295980+0000 front 2024-09-10T22:31:42.295996+0000 (oldest deadline 2024-09-10T22:32:04.596063+0000) 2024-09-10T22:32:45.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:45 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:45.534+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:52.197648+0000 front 2024-09-10T22:31:52.197709+0000 (oldest deadline 2024-09-10T22:32:16.798188+0000) 2024-09-10T22:32:45.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:45 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:45.534+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:31:56.798754+0000 front 2024-09-10T22:31:56.798867+0000 (oldest deadline 2024-09-10T22:32:22.698761+0000) 2024-09-10T22:32:45.919 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:45 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10[60112]: 2024-09-10T22:32:45.534+0000 7f1ca7200640 -1 osd.10 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:09.700132+0000 front 2024-09-10T22:32:09.700214+0000 (oldest deadline 2024-09-10T22:32:32.000022+0000) 2024-09-10T22:32:45.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:45.554+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:45.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:45.554+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:45.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:45.554+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:45.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:45.554+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:45.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:45.554+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:45.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:45.554+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:45.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:45.554+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:46.547 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:46 smithi154 podman[75766]: 2024-09-10 22:32:46.22198932 +0000 UTC m=+5.735847588 container died e5190e3d73782db4949fb448fc64621ecd81b37ee36f62e5ae06ef1bf4bd3a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T22:32:46.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:46 smithi154 podman[75766]: 2024-09-10 22:32:46.665337331 +0000 UTC m=+6.179195599 container cleanup e5190e3d73782db4949fb448fc64621ecd81b37ee36f62e5ae06ef1bf4bd3a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T22:32:46.918 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:46 smithi154 bash[75766]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10 2024-09-10T22:32:46.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:46.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:46.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:46.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:46.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:46.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:46.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:46.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:46.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:46.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:46.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:46.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:46.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:46 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:46.582+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:47.418 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:46 smithi154 podman[75783]: 2024-09-10 22:32:46.974910096 +0000 UTC m=+0.744474048 container remove e5190e3d73782db4949fb448fc64621ecd81b37ee36f62e5ae06ef1bf4bd3a84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:47.871 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:47 smithi154 podman[75871]: 2024-09-10 22:32:47.44065652 +0000 UTC m=+0.028078573 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:32:47.871 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:47 smithi154 podman[75871]: 2024-09-10 22:32:47.592899845 +0000 UTC m=+0.180321901 container create e5e597b0f4b66457766540fdb3d9af012bf29cda7e15e53126eb27bd1e70d20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:32:47.871 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:47 smithi154 podman[75871]: 2024-09-10 22:32:47.870290023 +0000 UTC m=+0.457712077 container init e5e597b0f4b66457766540fdb3d9af012bf29cda7e15e53126eb27bd1e70d20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T22:32:47.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:47.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:47.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:47.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:47.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:47.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:47.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:47.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:47.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:47.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:47.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:47.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:47.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:47 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:47.609+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:48.136 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:47 smithi154 podman[75871]: 2024-09-10 22:32:47.877306022 +0000 UTC m=+0.464728076 container start e5e597b0f4b66457766540fdb3d9af012bf29cda7e15e53126eb27bd1e70d20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True) 2024-09-10T22:32:48.136 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:48 smithi154 podman[75871]: 2024-09-10 22:32:48.045367931 +0000 UTC m=+0.632790003 container attach e5e597b0f4b66457766540fdb3d9af012bf29cda7e15e53126eb27bd1e70d20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T22:32:48.418 INFO:journalctl@ceph.osd.10.smithi154.stdout:Sep 10 22:32:48 smithi154 podman[75871]: 2024-09-10 22:32:48.136249896 +0000 UTC m=+0.723671957 container died e5e597b0f4b66457766540fdb3d9af012bf29cda7e15e53126eb27bd1e70d20b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T22:32:48.909 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T22:32:48.985 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:32:48.985 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T22:32:48.986 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T22:32:48.986 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.11 2024-09-10T22:32:48.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:48.659+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:48.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:48.659+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:48.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:48.659+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:48.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:48.659+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:48.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:48.659+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:48.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:48.659+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:48.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:48 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:48.659+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:49.442 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 systemd[1]: Stopping Ceph osd.11 for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.440+0000 7fec2b000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.440+0000 7fec2b000640 -1 osd.11 136 *** Got signal Terminated *** 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.440+0000 7fec2b000640 -1 osd.11 136 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.689+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.689+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.689+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.689+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.689+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.689+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:49.748 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:49 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:49.689+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:50.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:50.657+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:50.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:50.657+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:50.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:50.657+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:50.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:50.657+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:50.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:50.657+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:50.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:50.657+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:50.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:50 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:50.657+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:51.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:51.632+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:51.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:51.632+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:51.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:51.632+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:51.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:51.632+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:51.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:51.632+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:51.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:51.632+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:51.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:51 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:51.632+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:52.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:52.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:52.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:52.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:52.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:52.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:52.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:52.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:52.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:52.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:52.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:52.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:52.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:52.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:52.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:52 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:52.599+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-09-10T22:32:27.121526+0000 front 2024-09-10T22:32:27.121465+0000 (oldest deadline 2024-09-10T22:32:51.721769+0000) 2024-09-10T22:32:53.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:53.555+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-09-10T22:31:14.205500+0000 front 2024-09-10T22:31:14.205476+0000 (oldest deadline 2024-09-10T22:31:40.606336+0000) 2024-09-10T22:32:53.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:53.555+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6804 osd.1 since back 2024-09-10T22:31:20.607240+0000 front 2024-09-10T22:31:20.607047+0000 (oldest deadline 2024-09-10T22:31:47.607639+0000) 2024-09-10T22:32:53.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:53.555+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6804 osd.2 since back 2024-09-10T22:31:33.509083+0000 front 2024-09-10T22:31:33.509049+0000 (oldest deadline 2024-09-10T22:31:58.809343+0000) 2024-09-10T22:32:53.998 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:53.555+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6812 osd.4 since back 2024-09-10T22:31:50.913545+0000 front 2024-09-10T22:31:50.913585+0000 (oldest deadline 2024-09-10T22:32:14.413566+0000) 2024-09-10T22:32:53.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:53.555+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6812 osd.5 since back 2024-09-10T22:32:00.417028+0000 front 2024-09-10T22:32:00.417053+0000 (oldest deadline 2024-09-10T22:32:22.716990+0000) 2024-09-10T22:32:53.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:53.555+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.90:6822 osd.6 since back 2024-09-10T22:32:06.718553+0000 front 2024-09-10T22:32:06.718650+0000 (oldest deadline 2024-09-10T22:32:32.018424+0000) 2024-09-10T22:32:53.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:53.555+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.154:6820 osd.7 since back 2024-09-10T22:32:16.119534+0000 front 2024-09-10T22:32:16.119553+0000 (oldest deadline 2024-09-10T22:32:40.819486+0000) 2024-09-10T22:32:53.999 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:53 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11[60370]: 2024-09-10T22:32:53.555+0000 7fec26a00640 -1 osd.11 136 heartbeat_check: no reply from 172.21.15.171:6820 osd.8 since back 2024-09-10T22:32:27.121526+0000 front 2024-09-10T22:32:27.121465+0000 (oldest deadline 2024-09-10T22:32:51.721769+0000) 2024-09-10T22:32:55.382 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:55 smithi171 podman[78891]: 2024-09-10 22:32:55.087186878 +0000 UTC m=+5.813384592 container died beb107cbc71fdeb449282ca7d3fb7fe969e3184e0e1d59a36af26d5272a3ee6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T22:32:55.733 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:55 smithi171 podman[78891]: 2024-09-10 22:32:55.504790537 +0000 UTC m=+6.230988254 container cleanup beb107cbc71fdeb449282ca7d3fb7fe969e3184e0e1d59a36af26d5272a3ee6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:32:55.733 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:55 smithi171 bash[78891]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11 2024-09-10T22:32:55.997 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:55 smithi171 podman[78907]: 2024-09-10 22:32:55.731386801 +0000 UTC m=+0.637788625 container remove beb107cbc71fdeb449282ca7d3fb7fe969e3184e0e1d59a36af26d5272a3ee6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T22:32:56.586 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:56 smithi171 podman[78994]: 2024-09-10 22:32:56.176989178 +0000 UTC m=+0.027213447 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T22:32:56.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:56 smithi171 podman[78994]: 2024-09-10 22:32:56.316265856 +0000 UTC m=+0.166490111 container create b3f10bbbec653b6a7e896b8bc30f607c3f3d53b83c00bffd0bacac967c89a8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T22:32:56.587 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:56 smithi171 podman[78994]: 2024-09-10 22:32:56.585806186 +0000 UTC m=+0.436030443 container init b3f10bbbec653b6a7e896b8bc30f607c3f3d53b83c00bffd0bacac967c89a8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:32:56.862 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:56 smithi171 podman[78994]: 2024-09-10 22:32:56.592719676 +0000 UTC m=+0.442943940 container start b3f10bbbec653b6a7e896b8bc30f607c3f3d53b83c00bffd0bacac967c89a8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T22:32:56.862 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:56 smithi171 podman[78994]: 2024-09-10 22:32:56.668903793 +0000 UTC m=+0.519128080 container attach b3f10bbbec653b6a7e896b8bc30f607c3f3d53b83c00bffd0bacac967c89a8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:32:57.190 INFO:journalctl@ceph.osd.11.smithi171.stdout:Sep 10 22:32:56 smithi171 podman[78994]: 2024-09-10 22:32:56.861451977 +0000 UTC m=+0.711676238 container died b3f10bbbec653b6a7e896b8bc30f607c3f3d53b83c00bffd0bacac967c89a8db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T22:32:57.584 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T22:32:57.700 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:32:57.701 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T22:32:57.701 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T22:32:57.701 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.a 2024-09-10T22:32:58.106 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:32:57 smithi090 systemd[1]: Stopping Ceph mds.a for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:32:58.606 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:32:58 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-a[75842]: 2024-09-10T22:32:58.196+0000 7f1025000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:32:58.607 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:32:58 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-a[75842]: 2024-09-10T22:32:58.196+0000 7f1025000640 -1 mds.a *** got signal Terminated *** 2024-09-10T22:33:00.606 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:33:00 smithi090 podman[116424]: 2024-09-10 22:33:00.272846685 +0000 UTC m=+2.288191247 container died 922cccf8c017fe65aceb967525fdb6de60a6260a9bb974af33a1de4bda406c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:33:01.037 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:33:00 smithi090 podman[116424]: 2024-09-10 22:33:00.784585002 +0000 UTC m=+2.799929557 container cleanup 922cccf8c017fe65aceb967525fdb6de60a6260a9bb974af33a1de4bda406c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:33:01.038 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:33:00 smithi090 bash[116424]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-a 2024-09-10T22:33:01.302 INFO:journalctl@ceph.mds.a.smithi090.stdout:Sep 10 22:33:01 smithi090 podman[116436]: 2024-09-10 22:33:01.036404727 +0000 UTC m=+0.753788046 container remove 922cccf8c017fe65aceb967525fdb6de60a6260a9bb974af33a1de4bda406c8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T22:33:01.326 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T22:33:01.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:01.400 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T22:33:01.401 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T22:33:01.401 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.d 2024-09-10T22:33:01.606 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:01 smithi090 systemd[1]: Stopping Ceph mds.d for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:02.106 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:01 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d[74559]: 2024-09-10T22:33:01.867+0000 7f1afd400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:02.106 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:01 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d[74559]: 2024-09-10T22:33:01.867+0000 7f1afd400640 -1 mds.d *** got signal Terminated *** 2024-09-10T22:33:06.169 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:05 smithi090 podman[116548]: 2024-09-10 22:33:05.87795389 +0000 UTC m=+4.203228860 container died 1e770cf4225f6cc0a148029703288ba335b45d90c4296024981d89b94df99729 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:33:06.430 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:06 smithi090 podman[116548]: 2024-09-10 22:33:06.408174107 +0000 UTC m=+4.733449075 container cleanup 1e770cf4225f6cc0a148029703288ba335b45d90c4296024981d89b94df99729 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T22:33:06.430 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:06 smithi090 bash[116548]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d 2024-09-10T22:33:06.836 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T22:33:06.856 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:06 smithi090 podman[116563]: 2024-09-10 22:33:06.584837859 +0000 UTC m=+0.674907388 container remove 1e770cf4225f6cc0a148029703288ba335b45d90c4296024981d89b94df99729 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:33:06.856 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:06 smithi090 systemd[1]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.d.service: Deactivated successfully. 2024-09-10T22:33:06.856 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:06 smithi090 systemd[1]: Stopped Ceph mds.d for 815bde3e-6fbe-11ef-bcea-c7b262605968. 2024-09-10T22:33:06.857 INFO:journalctl@ceph.mds.d.smithi090.stdout:Sep 10 22:33:06 smithi090 systemd[1]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.d.service: Consumed 1.914s CPU time. 2024-09-10T22:33:06.906 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:06.907 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T22:33:06.907 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T22:33:06.907 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.g 2024-09-10T22:33:07.350 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:33:07 smithi090 systemd[1]: Stopping Ceph mds.g for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:07.606 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:33:07 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g[75428]: 2024-09-10T22:33:07.348+0000 7fb355800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:07.606 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:33:07 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g[75428]: 2024-09-10T22:33:07.348+0000 7fb355800640 -1 mds.g *** got signal Terminated *** 2024-09-10T22:33:11.774 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:33:11 smithi090 podman[116669]: 2024-09-10 22:33:11.366085674 +0000 UTC m=+4.142208605 container died 98f264980d681bf8737990158f9ca54f2ecb4dd8f8a3cc9fab28bbf9e52ff3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0) 2024-09-10T22:33:12.106 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:33:11 smithi090 podman[116669]: 2024-09-10 22:33:11.896842079 +0000 UTC m=+4.672965010 container cleanup 98f264980d681bf8737990158f9ca54f2ecb4dd8f8a3cc9fab28bbf9e52ff3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:33:12.106 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:33:11 smithi090 bash[116669]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g 2024-09-10T22:33:12.449 INFO:journalctl@ceph.mds.g.smithi090.stdout:Sep 10 22:33:12 smithi090 podman[116681]: 2024-09-10 22:33:12.216494723 +0000 UTC m=+0.844024180 container remove 98f264980d681bf8737990158f9ca54f2ecb4dd8f8a3cc9fab28bbf9e52ff3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD) 2024-09-10T22:33:12.471 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T22:33:12.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:12.541 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T22:33:12.541 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T22:33:12.542 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.j 2024-09-10T22:33:12.856 INFO:journalctl@ceph.mds.j.smithi090.stdout:Sep 10 22:33:12 smithi090 systemd[1]: Stopping Ceph mds.j for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:13.356 INFO:journalctl@ceph.mds.j.smithi090.stdout:Sep 10 22:33:12 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-j[74147]: 2024-09-10T22:33:12.963+0000 7f274d200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:13.356 INFO:journalctl@ceph.mds.j.smithi090.stdout:Sep 10 22:33:12 smithi090 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-j[74147]: 2024-09-10T22:33:12.963+0000 7f274d200640 -1 mds.j *** got signal Terminated *** 2024-09-10T22:33:16.106 INFO:journalctl@ceph.mds.j.smithi090.stdout:Sep 10 22:33:15 smithi090 podman[116790]: 2024-09-10 22:33:15.835737967 +0000 UTC m=+3.023696670 container died ee10b76dd6fd25ee16746879e6bdf904790804c2f2709d7ec0ec12d35e11ffa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T22:33:16.606 INFO:journalctl@ceph.mds.j.smithi090.stdout:Sep 10 22:33:16 smithi090 podman[116790]: 2024-09-10 22:33:16.336910234 +0000 UTC m=+3.524868937 container cleanup ee10b76dd6fd25ee16746879e6bdf904790804c2f2709d7ec0ec12d35e11ffa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:33:16.606 INFO:journalctl@ceph.mds.j.smithi090.stdout:Sep 10 22:33:16 smithi090 bash[116790]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-j 2024-09-10T22:33:16.973 INFO:journalctl@ceph.mds.j.smithi090.stdout:Sep 10 22:33:16 smithi090 podman[116814]: 2024-09-10 22:33:16.713344597 +0000 UTC m=+0.862374786 container remove ee10b76dd6fd25ee16746879e6bdf904790804c2f2709d7ec0ec12d35e11ffa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-j, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T22:33:16.981 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T22:33:17.054 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:17.054 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T22:33:17.054 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T22:33:17.055 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.b 2024-09-10T22:33:17.418 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:33:17 smithi154 systemd[1]: Stopping Ceph mds.b for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:17.918 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:33:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b[65608]: 2024-09-10T22:33:17.459+0000 7f058d200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:17.918 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:33:17 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b[65608]: 2024-09-10T22:33:17.459+0000 7f058d200640 -1 mds.b *** got signal Terminated *** 2024-09-10T22:33:20.542 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:33:20 smithi154 podman[75973]: 2024-09-10 22:33:20.209490783 +0000 UTC m=+2.871079042 container died 60d5877916da954419638508f2989752cdd7dee9c5e1285c1291f483a80956c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T22:33:20.858 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:33:20 smithi154 podman[75973]: 2024-09-10 22:33:20.65610451 +0000 UTC m=+3.317692773 container cleanup 60d5877916da954419638508f2989752cdd7dee9c5e1285c1291f483a80956c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T22:33:20.858 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:33:20 smithi154 bash[75973]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b 2024-09-10T22:33:21.112 INFO:journalctl@ceph.mds.b.smithi154.stdout:Sep 10 22:33:20 smithi154 podman[75998]: 2024-09-10 22:33:20.857384356 +0000 UTC m=+0.637917795 container remove 60d5877916da954419638508f2989752cdd7dee9c5e1285c1291f483a80956c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T22:33:21.121 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T22:33:21.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:21.194 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T22:33:21.194 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T22:33:21.194 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.e 2024-09-10T22:33:21.418 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:33:21 smithi154 systemd[1]: Stopping Ceph mds.e for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:21.918 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:33:21 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e[64990]: 2024-09-10T22:33:21.588+0000 7f9b4ec00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:21.918 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:33:21 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e[64990]: 2024-09-10T22:33:21.588+0000 7f9b4ec00640 -1 mds.e *** got signal Terminated *** 2024-09-10T22:33:24.550 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:33:24 smithi154 podman[76105]: 2024-09-10 22:33:24.232069705 +0000 UTC m=+2.761302184 container died a78c6981b572579b67c94b054e965d2d88b2b6d9801f5022ed66ba20e84b80b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:33:24.903 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:33:24 smithi154 podman[76105]: 2024-09-10 22:33:24.668215377 +0000 UTC m=+3.197447855 container cleanup a78c6981b572579b67c94b054e965d2d88b2b6d9801f5022ed66ba20e84b80b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:33:24.904 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:33:24 smithi154 bash[76105]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e 2024-09-10T22:33:25.168 INFO:journalctl@ceph.mds.e.smithi154.stdout:Sep 10 22:33:24 smithi154 podman[76117]: 2024-09-10 22:33:24.902728158 +0000 UTC m=+0.665324401 container remove a78c6981b572579b67c94b054e965d2d88b2b6d9801f5022ed66ba20e84b80b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:33:25.178 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T22:33:25.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:25.250 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T22:33:25.250 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T22:33:25.250 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.h 2024-09-10T22:33:25.643 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:33:25 smithi154 systemd[1]: Stopping Ceph mds.h for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:25.918 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:33:25 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h[65292]: 2024-09-10T22:33:25.641+0000 7ff96a800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:25.918 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:33:25 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h[65292]: 2024-09-10T22:33:25.641+0000 7ff96a800640 -1 mds.h *** got signal Terminated *** 2024-09-10T22:33:28.499 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:33:28 smithi154 podman[76223]: 2024-09-10 22:33:28.216268088 +0000 UTC m=+2.696370713 container died 54d88274eab6f43ccb4bbf27fe413112d79f9b6bde9113536116f9ffff2ed905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:33:28.882 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:33:28 smithi154 podman[76223]: 2024-09-10 22:33:28.655162708 +0000 UTC m=+3.135265316 container cleanup 54d88274eab6f43ccb4bbf27fe413112d79f9b6bde9113536116f9ffff2ed905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T22:33:28.882 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:33:28 smithi154 bash[76223]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h 2024-09-10T22:33:29.139 INFO:journalctl@ceph.mds.h.smithi154.stdout:Sep 10 22:33:28 smithi154 podman[76235]: 2024-09-10 22:33:28.881369332 +0000 UTC m=+0.659183201 container remove 54d88274eab6f43ccb4bbf27fe413112d79f9b6bde9113536116f9ffff2ed905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T22:33:29.148 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T22:33:29.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:29.217 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T22:33:29.217 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T22:33:29.217 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.k 2024-09-10T22:33:29.418 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:33:29 smithi154 systemd[1]: Stopping Ceph mds.k for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:29.918 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:33:29 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k[64688]: 2024-09-10T22:33:29.619+0000 7f6f79c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:29.918 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:33:29 smithi154 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k[64688]: 2024-09-10T22:33:29.619+0000 7f6f79c00640 -1 mds.k *** got signal Terminated *** 2024-09-10T22:33:32.528 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:33:32 smithi154 podman[76341]: 2024-09-10 22:33:32.217532344 +0000 UTC m=+2.715899895 container died 5f9d0d7e4467a15ac0bf5f423728811f531fdb8ba7586dc55f39571469e6cdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:33:32.910 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:33:32 smithi154 podman[76341]: 2024-09-10 22:33:32.666937182 +0000 UTC m=+3.165304731 container cleanup 5f9d0d7e4467a15ac0bf5f423728811f531fdb8ba7586dc55f39571469e6cdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T22:33:32.910 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:33:32 smithi154 bash[76341]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k 2024-09-10T22:33:33.164 INFO:journalctl@ceph.mds.k.smithi154.stdout:Sep 10 22:33:32 smithi154 podman[76353]: 2024-09-10 22:33:32.909522256 +0000 UTC m=+0.663757149 container remove 5f9d0d7e4467a15ac0bf5f423728811f531fdb8ba7586dc55f39571469e6cdc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:33:33.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T22:33:33.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:33.241 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T22:33:33.241 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T22:33:33.242 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.c 2024-09-10T22:33:33.653 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:33:33 smithi171 systemd[1]: Stopping Ceph mds.c for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:33.972 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:33:33 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c[65189]: 2024-09-10T22:33:33.650+0000 7fbb96400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:33.973 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:33:33 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c[65189]: 2024-09-10T22:33:33.650+0000 7fbb96400640 -1 mds.c *** got signal Terminated *** 2024-09-10T22:33:36.459 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:33:36 smithi171 podman[79097]: 2024-09-10 22:33:36.170179491 +0000 UTC m=+2.642268281 container died 9a12e81784aee4aa27a91d478c54be1050f9dd2a5792b32b819eaf3f4ee3f27a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T22:33:36.748 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:33:36 smithi171 podman[79097]: 2024-09-10 22:33:36.556751508 +0000 UTC m=+3.028840294 container cleanup 9a12e81784aee4aa27a91d478c54be1050f9dd2a5792b32b819eaf3f4ee3f27a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T22:33:36.748 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:33:36 smithi171 bash[79097]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c 2024-09-10T22:33:37.034 INFO:journalctl@ceph.mds.c.smithi171.stdout:Sep 10 22:33:36 smithi171 podman[79123]: 2024-09-10 22:33:36.850221558 +0000 UTC m=+0.674721603 container remove 9a12e81784aee4aa27a91d478c54be1050f9dd2a5792b32b819eaf3f4ee3f27a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-c, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T22:33:37.058 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T22:33:37.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:37.132 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T22:33:37.132 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T22:33:37.132 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.f 2024-09-10T22:33:37.497 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:33:37 smithi171 systemd[1]: Stopping Ceph mds.f for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:37.997 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:33:37 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-f[64885]: 2024-09-10T22:33:37.539+0000 7f0ea6000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:37.998 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:33:37 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-f[64885]: 2024-09-10T22:33:37.539+0000 7f0ea6000640 -1 mds.f *** got signal Terminated *** 2024-09-10T22:33:40.498 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:33:40 smithi171 podman[79230]: 2024-09-10 22:33:40.179630343 +0000 UTC m=+2.776608257 container died 917ee409f917486539e0f6d415be23649f7a53601e7ebffab11fb062bf4e714d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T22:33:40.888 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:33:40 smithi171 podman[79230]: 2024-09-10 22:33:40.678196103 +0000 UTC m=+3.275174017 container cleanup 917ee409f917486539e0f6d415be23649f7a53601e7ebffab11fb062bf4e714d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-09-10T22:33:40.889 INFO:journalctl@ceph.mds.f.smithi171.stdout:Sep 10 22:33:40 smithi171 bash[79230]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-f 2024-09-10T22:33:41.052 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T22:33:41.123 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:41.123 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T22:33:41.123 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T22:33:41.123 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.i 2024-09-10T22:33:41.498 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:33:41 smithi171 systemd[1]: Stopping Ceph mds.i for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:41.998 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:33:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i[65488]: 2024-09-10T22:33:41.509+0000 7f24d8800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:41.998 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:33:41 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i[65488]: 2024-09-10T22:33:41.509+0000 7f24d8800640 -1 mds.i *** got signal Terminated *** 2024-09-10T22:33:44.443 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:33:44 smithi171 podman[79351]: 2024-09-10 22:33:44.163986195 +0000 UTC m=+2.762715266 container died 6a13ab59feeb787efe45618f9d1d5c5ef2595806bdaefb6bfdb368e5d1fe1b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T22:33:44.748 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:33:44 smithi171 podman[79351]: 2024-09-10 22:33:44.540930052 +0000 UTC m=+3.139659122 container cleanup 6a13ab59feeb787efe45618f9d1d5c5ef2595806bdaefb6bfdb368e5d1fe1b47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T22:33:44.748 INFO:journalctl@ceph.mds.i.smithi171.stdout:Sep 10 22:33:44 smithi171 bash[79351]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-i 2024-09-10T22:33:45.028 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T22:33:45.098 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:45.098 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T22:33:45.098 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T22:33:45.099 DEBUG:teuthology.orchestra.run.smithi171:> sudo systemctl stop ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.l 2024-09-10T22:33:45.498 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:33:45 smithi171 systemd[1]: Stopping Ceph mds.l for 815bde3e-6fbe-11ef-bcea-c7b262605968... 2024-09-10T22:33:45.997 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:33:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l[64579]: 2024-09-10T22:33:45.623+0000 7f7611c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T22:33:45.998 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:33:45 smithi171 ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l[64579]: 2024-09-10T22:33:45.623+0000 7f7611c00640 -1 mds.l *** got signal Terminated *** 2024-09-10T22:33:48.466 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:33:48 smithi171 podman[79472]: 2024-09-10 22:33:48.186534771 +0000 UTC m=+2.814407827 container died a4454f43cfa3b2b24247eef599880154f5df09abc3ee6532b74bd4a0e56e5200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T22:33:48.747 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:33:48 smithi171 podman[79472]: 2024-09-10 22:33:48.579694144 +0000 UTC m=+3.207567200 container cleanup a4454f43cfa3b2b24247eef599880154f5df09abc3ee6532b74bd4a0e56e5200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T22:33:48.748 INFO:journalctl@ceph.mds.l.smithi171.stdout:Sep 10 22:33:48 smithi171 bash[79472]: ceph-815bde3e-6fbe-11ef-bcea-c7b262605968-mds-l 2024-09-10T22:33:48.987 DEBUG:teuthology.orchestra.run.smithi171:> sudo pkill -f 'journalctl -f -n 0 -u ceph-815bde3e-6fbe-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T22:33:49.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T22:33:49.057 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T22:33:49.057 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T22:33:49.330 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T22:33:53.353 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T22:33:53.628 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T22:33:57.680 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T22:33:57.953 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T22:34:01.839 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:34:01.903 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:34:01.970 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T22:34:02.028 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T22:34:02.030 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695/remote/smithi090/crash 2024-09-10T22:34:02.030 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/crash -- . 2024-09-10T22:34:02.112 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T22:34:02.112 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-09-10T22:34:02.116 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695/remote/smithi154/crash 2024-09-10T22:34:02.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/crash -- . 2024-09-10T22:34:02.187 INFO:teuthology.orchestra.run.smithi154.stderr:tar: /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T22:34:02.187 INFO:teuthology.orchestra.run.smithi154.stderr:tar: Error is not recoverable: exiting now 2024-09-10T22:34:02.191 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695/remote/smithi171/crash 2024-09-10T22:34:02.191 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/crash -- . 2024-09-10T22:34:02.263 INFO:teuthology.orchestra.run.smithi171.stderr:tar: /var/lib/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T22:34:02.263 INFO:teuthology.orchestra.run.smithi171.stderr:tar: Error is not recoverable: exiting now 2024-09-10T22:34:02.266 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T22:34:02.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-10T22:34:02.470 INFO:tasks.cephadm:Compressing logs... 2024-09-10T22:34:02.471 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T22:34:02.476 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T22:34:02.480 DEBUG:teuthology.orchestra.run.smithi171:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T22:34:02.535 INFO:teuthology.orchestra.run.smithi171.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T22:34:02.535 INFO:teuthology.orchestra.run.smithi171.stderr:: No such file or directory 2024-09-10T22:34:02.535 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T22:34:02.536 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T22:34:02.536 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T22:34:02.537 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-10T22:34:02.537 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T22:34:02.538 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T22:34:02.538 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T22:34:02.539 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T22:34:02.540 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T22:34:02.540 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T22:34:02.541 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T22:34:02.541 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T22:34:02.542 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.cephadm.log: 90.3% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T22:34:02.542 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T22:34:02.542 INFO:teuthology.orchestra.run.smithi171.stderr: 82.7% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T22:34:02.542 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T22:34:02.543 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.11.log: /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log 2024-09-10T22:34:02.543 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T22:34:02.544 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T22:34:02.544 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T22:34:02.545 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T22:34:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T22:34:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log 2024-09-10T22:34:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T22:34:02.546 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T22:34:02.547 INFO:teuthology.orchestra.run.smithi154.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T22:34:02.547 INFO:teuthology.orchestra.run.smithi154.stderr:: No such file or directory 2024-09-10T22:34:02.547 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T22:34:02.547 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T22:34:02.547 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T22:34:02.547 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T22:34:02.547 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T22:34:02.549 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T22:34:02.549 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T22:34:02.550 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T22:34:02.550 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T22:34:02.550 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T22:34:02.550 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mgr.x.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-10T22:34:02.550 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T22:34:02.550 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-10T22:34:02.552 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T22:34:02.552 INFO:teuthology.orchestra.run.smithi090.stderr: 87.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T22:34:02.553 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T22:34:02.554 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log 2024-09-10T22:34:02.554 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T22:34:02.554 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T22:34:02.554 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T22:34:02.554 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T22:34:02.554 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T22:34:02.554 INFO:teuthology.orchestra.run.smithi171.stderr: 87.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T22:34:02.554 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.c.log/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.8.log: 2024-09-10T22:34:02.554 INFO:teuthology.orchestra.run.smithi171.stderr:gzip -5 --verbose -- /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T22:34:02.555 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T22:34:02.559 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T22:34:02.560 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 95.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T22:34:02.562 INFO:teuthology.orchestra.run.smithi090.stderr: 95.0% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T22:34:02.563 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.c.log: /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T22:34:02.574 INFO:teuthology.orchestra.run.smithi171.stderr:/var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T22:34:02.595 INFO:teuthology.orchestra.run.smithi171.stderr: 93.3% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T22:34:02.695 INFO:teuthology.orchestra.run.smithi090.stderr: 84.7% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T22:34:02.718 INFO:teuthology.orchestra.run.smithi154.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T22:34:02.726 INFO:teuthology.orchestra.run.smithi154.stderr: 82.8% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T22:34:02.807 INFO:teuthology.orchestra.run.smithi090.stderr: 91.0% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T22:34:02.847 INFO:teuthology.orchestra.run.smithi090.stderr: 90.2% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T22:34:03.015 INFO:teuthology.orchestra.run.smithi154.stderr: 90.9% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T22:34:03.038 INFO:teuthology.orchestra.run.smithi154.stderr: 90.3% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T22:34:03.117 INFO:teuthology.orchestra.run.smithi154.stderr: 87.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T22:34:03.233 INFO:teuthology.orchestra.run.smithi090.stderr: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T22:34:03.356 INFO:teuthology.orchestra.run.smithi171.stderr: 91.7% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T22:34:04.951 INFO:teuthology.orchestra.run.smithi090.stderr: 90.5% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T22:34:06.010 INFO:teuthology.orchestra.run.smithi171.stderr: 94.6% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T22:34:06.502 INFO:teuthology.orchestra.run.smithi171.stderr: 95.0% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T22:34:07.015 INFO:teuthology.orchestra.run.smithi171.stderr: 94.9% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T22:34:10.327 INFO:teuthology.orchestra.run.smithi090.stderr: 90.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T22:34:17.055 INFO:teuthology.orchestra.run.smithi090.stderr: 94.8% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T22:34:24.844 INFO:teuthology.orchestra.run.smithi154.stderr: 92.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T22:34:27.337 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T22:34:27.833 INFO:teuthology.orchestra.run.smithi090.stderr: 94.5% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T22:34:30.131 INFO:teuthology.orchestra.run.smithi171.stderr: 91.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T22:34:30.958 INFO:teuthology.orchestra.run.smithi171.stderr: 91.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T22:34:38.354 INFO:teuthology.orchestra.run.smithi171.stderr: 91.0% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T22:35:20.829 INFO:teuthology.orchestra.run.smithi090.stderr: 91.4% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T22:35:28.451 INFO:teuthology.orchestra.run.smithi090.stderr: 91.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T22:35:33.611 INFO:teuthology.orchestra.run.smithi090.stderr: 91.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T22:35:37.167 INFO:teuthology.orchestra.run.smithi090.stderr: 91.0% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T22:35:52.059 INFO:teuthology.orchestra.run.smithi171.stderr: 94.2% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T22:35:52.065 INFO:teuthology.orchestra.run.smithi171.stderr: 2024-09-10T22:35:52.065 INFO:teuthology.orchestra.run.smithi171.stderr:real 1m49.550s 2024-09-10T22:35:52.065 INFO:teuthology.orchestra.run.smithi171.stderr:user 3m38.258s 2024-09-10T22:35:52.065 INFO:teuthology.orchestra.run.smithi171.stderr:sys 0m17.573s 2024-09-10T22:36:56.400 INFO:teuthology.orchestra.run.smithi154.stderr: 91.5% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T22:36:59.972 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T22:38:34.696 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T22:39:01.526 INFO:teuthology.orchestra.run.smithi090.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-10T22:39:01.530 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-09-10T22:39:01.530 INFO:teuthology.orchestra.run.smithi090.stderr:real 4m59.025s 2024-09-10T22:39:01.530 INFO:teuthology.orchestra.run.smithi090.stderr:user 5m4.529s 2024-09-10T22:39:01.530 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m30.095s 2024-09-10T22:39:11.936 INFO:teuthology.orchestra.run.smithi154.stderr: 91.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T22:48:11.809 INFO:teuthology.orchestra.run.smithi154.stderr: 93.3% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T22:49:59.487 INFO:teuthology.orchestra.run.smithi154.stderr: 94.1% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T22:52:04.655 INFO:teuthology.orchestra.run.smithi154.stderr: 93.9% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T22:53:41.027 INFO:teuthology.orchestra.run.smithi154.stderr: 94.9% -- replaced with /var/log/ceph/815bde3e-6fbe-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T22:53:41.053 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-09-10T22:53:41.053 INFO:teuthology.orchestra.run.smithi154.stderr:real 19m38.520s 2024-09-10T22:53:41.053 INFO:teuthology.orchestra.run.smithi154.stderr:user 8m6.729s 2024-09-10T22:53:41.053 INFO:teuthology.orchestra.run.smithi154.stderr:sys 1m8.949s 2024-09-10T22:53:41.054 INFO:tasks.cephadm:Archiving logs... 2024-09-10T22:53:41.055 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695/remote/smithi090/log 2024-09-10T22:53:41.056 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:54:32.110 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695/remote/smithi154/log 2024-09-10T22:54:32.111 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:55:42.293 DEBUG:teuthology.misc:Transferring archived files from smithi171:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695/remote/smithi171/log 2024-09-10T22:55:42.294 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:56:12.490 INFO:tasks.cephadm:Removing cluster... 2024-09-10T22:56:12.491 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 --force 2024-09-10T22:56:12.824 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T22:56:13.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 --force 2024-09-10T22:56:14.151 INFO:teuthology.orchestra.run.smithi154.stdout:Deleting cluster with fsid: 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T22:56:15.992 DEBUG:teuthology.orchestra.run.smithi171:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 815bde3e-6fbe-11ef-bcea-c7b262605968 --force 2024-09-10T22:56:16.268 INFO:teuthology.orchestra.run.smithi171.stdout:Deleting cluster with fsid: 815bde3e-6fbe-11ef-bcea-c7b262605968 2024-09-10T22:56:17.451 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T22:56:17.451 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T22:56:17.492 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T22:56:17.533 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T22:56:17.569 INFO:tasks.cephadm:Teardown complete 2024-09-10T22:56:17.569 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T22:56:17.581 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T22:56:17.581 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:56:17.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:56:17.591 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:56:17.844 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T22:56:17.856 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T22:56:17.857 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:56:17.862 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:56:17.867 DEBUG:teuthology.orchestra.run.smithi171:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:56:17.893 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:56:17.899 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:56:17.900 INFO:teuthology.orchestra.run.smithi171.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:56:17.907 INFO:teuthology.orchestra.run.smithi171.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:56:17.907 INFO:teuthology.orchestra.run.smithi171.stdout:=============================================================================== 2024-09-10T22:56:17.907 INFO:teuthology.orchestra.run.smithi171.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -58us[ -65us] +/- 43ms 2024-09-10T22:56:17.907 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 +673us[ +666us] +/- 58ms 2024-09-10T22:56:17.907 INFO:teuthology.orchestra.run.smithi171.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -668us[ -675us] +/- 66ms 2024-09-10T22:56:17.907 INFO:teuthology.orchestra.run.smithi171.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:56:18.029 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:56:18.029 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-09-10T22:56:18.030 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv01.front.sepia.ceph.com 3 6 377 12 -59us[ -65us] +/- 43ms 2024-09-10T22:56:18.030 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +674us[ +668us] +/- 58ms 2024-09-10T22:56:18.030 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -669us[ -676us] +/- 66ms 2024-09-10T22:56:18.030 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:56:18.039 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:56:18.039 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-09-10T22:56:18.039 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv01.front.sepia.ceph.com 3 6 377 16 -58us[ -72us] +/- 43ms 2024-09-10T22:56:18.039 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 +668us[ +655us] +/- 58ms 2024-09-10T22:56:18.040 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -678us[ -678us] +/- 66ms 2024-09-10T22:56:18.040 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:56:18.042 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T22:56:18.055 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T22:56:18.056 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T22:56:18.066 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T22:56:18.299 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T22:56:18.488 DEBUG:teuthology.orchestra.run.smithi171:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T22:56:18.612 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:56:18.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:56:18.674 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:56:18.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:56:18.738 DEBUG:teuthology.orchestra.run.smithi171:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:56:18.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:56:18.799 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T22:56:18.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T22:56:18.821 INFO:teuthology.task.internal:Duration was 4449.211882 seconds 2024-09-10T22:56:18.821 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T22:56:18.837 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T22:56:18.838 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:56:18.843 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:56:18.847 DEBUG:teuthology.orchestra.run.smithi171:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:56:18.935 INFO:teuthology.orchestra.run.smithi171.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:56:18.949 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:56:18.960 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:56:19.455 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T22:56:19.455 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-09-10T22:56:19.456 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:56:19.505 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-09-10T22:56:19.506 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:56:19.558 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi171.front.sepia.ceph.com 2024-09-10T22:56:19.558 DEBUG:teuthology.orchestra.run.smithi171:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:56:19.621 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T22:56:19.621 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:56:19.625 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:56:19.629 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:56:19.687 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T22:56:19.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:56:19.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:56:19.696 DEBUG:teuthology.orchestra.run.smithi171:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:56:20.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T22:56:20.668 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T22:56:20.669 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:56:20.731 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:56:20.791 DEBUG:teuthology.orchestra.run.smithi171:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:56:20.849 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T22:56:20.862 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:56:20.867 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:56:20.872 DEBUG:teuthology.orchestra.run.smithi171:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:56:20.942 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-09-10T22:56:20.942 INFO:teuthology.orchestra.run.smithi171.stdout:kernel.core_pattern = core 2024-09-10T22:56:20.961 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-09-10T22:56:21.017 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:56:21.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:56:21.051 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:56:21.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:56:21.086 DEBUG:teuthology.orchestra.run.smithi171:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:56:21.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:56:21.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T22:56:21.131 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T22:56:21.132 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695/remote/smithi090 2024-09-10T22:56:21.133 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:56:21.236 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695/remote/smithi154 2024-09-10T22:56:21.236 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:56:21.303 DEBUG:teuthology.misc:Transferring archived files from smithi171:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899695/remote/smithi171 2024-09-10T22:56:21.303 DEBUG:teuthology.orchestra.run.smithi171:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:56:21.370 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T22:56:21.371 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:56:21.375 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:56:21.379 DEBUG:teuthology.orchestra.run.smithi171:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:56:21.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T22:56:21.427 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T22:56:21.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T22:56:21.438 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T22:56:21.438 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:56:21.451 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:56:21.455 DEBUG:teuthology.orchestra.run.smithi171:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:56:21.481 INFO:teuthology.orchestra.run.smithi090.stdout: 393255 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:56 /home/ubuntu/cephtest 2024-09-10T22:56:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:56 /home/ubuntu/cephtest 2024-09-10T22:56:21.490 INFO:teuthology.orchestra.run.smithi171.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:56 /home/ubuntu/cephtest 2024-09-10T22:56:21.492 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T22:56:21.503 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T22:56:21.528 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 4449.211881637573 flavor: default mon.a-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a mon.b-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a mon.c-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a owner: scheduled_hyelloji@teuthology success: true 2024-09-10T22:56:21.528 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T22:56:21.624 INFO:teuthology.run:pass